Good Practice for Patient Registries (Text Version)

Slide Presentation from the AHRQ 2008 Annual Conference

On September 9, 2008, Richard Gliklich, M.D., made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (1.1 MB).

Slide 1

Good Practice for Patient Registries

AHRQ Annual Meeting
Bethesda, MD
September 9, 2008

Speaker: Richard Gliklich, MD
Principal Investigator (PI), Outcome Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Center
President, Outcome
Cambridge, MA

Slide 2

Premise

Registries can provide unique outcomes information on populations under real-world conditions that are not studied in clinical trials.

• Reality.
• Generalizability.
• Applicability.
• Availability.

What makes a registry good?

Slide 3

Current guidelines for pharmacoepidemiologic research do not fully address registries

• Experimental Research.
  • CONSORT Statement (JAMA 2001; 285:1987-91).
• Observational Research.
  • Guidelines for good pharmacoepidemiology practices (Pharmacoepidemiology & Drug Safety 2005:14:589-95).
  • Quality of Reporting of Observational Longitudinal Research (AJE 2005:161:280-8).
  • Guidance for Industry: Good pharmacovigilance practices & pharmacoepi. assessment. U.S. Department of Health and Human Services (HHS), March 2005.
  • Guidance for Industry: Establishing Pregnancy Exposure Registries, HHS, August 2002.

Slide 4

Registries for Evaluating Patient Outcomes: A User's Guide

• Commissioned by the Agency for Healthcare Research and Quality (AHRW) and the Centers for Medicare and Medicaid Services (CMS) in the United States.
• Purpose: To promote development of high-quality evidence that can be relied on for public and private purposes.
• Product: Handbook addresses the creation, operation, analysis, interpretation, reporting and evaluation of registries designed to evaluate patient outcomes.

Note: To the right of the text is a screen shot of the cover page of the publication.

Slide 5

Registries for Evaluating Patient Outcomes: A User's Guide (continued)

PIs: RE Gliklich, NA Dreyer, Outcome DEcIDE

Collaborative effort

• Outcome Sciences DEcIDE center.
• Duke University Evidence-based Practice Center (EPC).
• CMS Coverage and Analysis Group.
• 39 contributors from industry, academia, health plans, physician societies, U.S. government, National Institute for Clinical Effectiveness (NICE).
• 35 invited peer reviewers and public comment, including Office for Civil Rights (OCR), Office of Human Research Protections (OHRP), Institute of Medicine (IOM), among others.

Example driven: ~23 case studies illustrating specific challenges and solutions.

Available online and in print at https://effectivehealthcare.ahrq.gov

Note: To the right of the text there is a screen shot of the publication cover entitled, "Registries for Evaluating Patient Outcomes: A User's Guide."

Gliklich RE, Dreyer NA, eds.
Registries for Evaluating Patient Outcomes: A User's Guide.
Prepared by Outcome DEcIDE Center
AHRQ Publication No. 07-EHC001-1. Rockville, MD:
Agency for Healthcare Research and Quality. April 2007

Slide 6

What is a Patient Registry?

A patient registry:

• Is an organized system that uses observational study methods to collect uniform data (clinical and other).
• Evaluates specified outcomes for a population defined by a particular disease, condition, or exposure, and that
• Serves a predetermined scientific, clinical, or policy purpose.

Note: Gliklich RE, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. Prepared by Outcome DEcIDE Center, AHRQ Publication No. 07-EHC001-1. Rockville, MD: Agency for Healthcare Research and Quality. April 2007.

Slide 7

Key Characteristics of Registries

• Data are collected in a naturalistic manner.
• Registry is designed to fulfill specific purposes, and these purposes are defined in advance of collecting and analyzing the data.
• Registry captures data elements with specific and consistent data definitions.
• Data are collected in a uniform manner for every patient.
• Data collected derive from and are reflective of the clinical status of the patient (by history, examination, laboratory test, or patient reported).
• At least one element of registry data collection is active, meaning that some data are collected specifically for the purpose of the registry.

Slide 8

Typical Goals of Patient Registries

• Track natural history of a disease process.
• Measure or monitor safety and harm.
• Evaluate clinical, comparative or cost effectiveness.
• Measure and/or improve quality of care.

Note: To the right of the text is a screen shot of an opened page to an article entitled, "Controversy on Drug Eluting Stents Widens," from CMS's Web site.

Slide 9

Taxonomy of Registries: Product

May include all or a subset of patients exposed to a drug, device or biologic.

• Device registries:
  • Implantable cardioverter defibrillators (ICD).
  • Stents.
  • Orthopedic devices.
• Pharmaceutical product registries:
  • Cox 2 inhibitors.
  • Thalidomide.
• Pregnancy registries:
  • Exposed population = mother and fetus.

Note: To the right of the text is a screen shot of of a sample product registry database.

Slide 10

Taxonomy of Registries: Service

Targeting a health care service, procedure or clinical encounter.

• Procedure registries:
  • Primary coronary intervention.
  • Normal pressure hydrocephalus registry.
  • Society Thoracic Surgeons (STS) database.
• Clinical service (and quality measurement) registries:
  • Hospitalization registries.
  • Pay-for-Performance (P4P).

Note: To the right of the text is a screen shot showing a portion of a sample service registry.

Slide 11

Taxonomy of Registries: Disease/Event

Patients enrolled share a common disease or event experience, regardless of treatment or other exposures.

• Acute diseases or events:
  • National Registry Myocardial Infarction (NRMI).
  • Organ transplant registries.
• Chronic diseases:
  • End stage renal disease (ESRD) registry.
  • Heart failure registry.
  • Cancer registries (SEER).
• Rare diseases:
  • Lysosomal storage disorders.
  • Cystic fibrosis.
  • Hemophilia.

Note: To the right of the text is a screen shot of the home page of the SEER Web site.

Slide 12

Registry types and perspective

The slide shows a Venn diagram for "Disease Registry."

Large, orange circle represents: Clinical event registry
Large, yellow circle represents: Procedure registry
Small, orange circle represents: Device
Large, green oval represents: Drug registry
Small, yellow oval represents: Procedure B
Small, lavender oval represents: Pregnancy registry, nested inside large, green oval

Slide 13

Sections of Handbook

• Creating a Registry.
• Operating Registries.
• Evaluating Registries.

Slide 14

Sections of Handbook (continued)

• Creating a Registry
  • Planning
  • Design
  • Data Elements
  • Other Data Sources
  • Ethical and Legal Issues
• Operating Registries
• Evaluating Registries

Slide 15

Planning

• State the purpose.
• Identify stakeholders.
• Establish governance.
• Define scope of registry.
• Define target population.
• Assess feasibility.
• Secure funding.

Slide 16

Design

• Design registry with respect to its major purposes:
  • Different levels of rigor are necessary for different types of registries.
  • The specific clinical questions of interest will guide definitions of study subjects, exposures, and outcome measures.
• Choose a study design.
• Select data sources, populations, comparison groups.
• Determine whether sampling is needed.
• Identify possible sources of bias (systematic error) and address them to the extent that is practical and achievable.

Slide 17

Data Elements

• Select based on importance and relationship to the primary outcome.
• Consider data collection burden and incremental costs for collection.
• Whenever possible, use established standards and common data definitions or validated instruments.
• Weigh pros/cons of using patient identifiers.
• Use pilot testing to assess feasibility and burden as well as reliability, validity, and potential for missing data.

Slide 18

Data Sources

• Registries can include data from many sources.
• Primary data are collected for direct purposes of the registry.
• Secondary data were originally collected for other purposes:
  • Medical records.
  • Institutional or organizational databases.
  • Administrative and claims data.
  • Death and birth records.
  • Census databases.
  • Existing registry databases.
• When selecting data sources, consider cost, timeliness, structure, availability and quality.

Slide 19

Ethics, data ownership, and privacy

• Review ethical and data privacy requirements early in planning phase to ensure compliance:
  • Common Rule.
  • Health Insurance Portability and Accountability Act (HIPAA).
  • Local requirements.
• "The research purpose of a registry, the status of its developer, and the extent to which registry data are individually identifiable largely determine applicable regulatory requirements."
• Also important: registry transparency, oversight/governance and data ownership.

Slide 20

The table presents the "Extent an Individual May be Identified from Health Information" and "Waiver of Authorization, Documentation of Consent, or Consent Process" for a "Registry Developer or Purpose of Registry." "Extent an Individual May be Identified from Health Information" is broken down into (1) "De-identified health information**," (2) "Health information excludes direct identifiers," and (3) "Health information includes direct identifiers."

Slide 21

Sections

• Creating a Registry.
• Operating Registries:
  • Recruitment and management of providers and patients.
  • Data collection and quality assurance.
  • Adverse event collection and management.
  • Data analysis and interpretation.
• Evaluating Registries.

Slide 22

Patient and Provider Recruitment & Management

• Recruitment occurs at many levels.
  • Facilities (hospital, practice, pharmacy).
  • Providers.
  • Patients.
• Motivation for participation at each level differs.
  • Relevance, importance, scientific credibility, risks, burdens, incentives.
• Goals for recruitment, retention and follow-up should be explicit and deviations continuously evaluated for risk of introducing bias.

Slide 23

Data Collection & Quality Assurance

• Data collection:
  • Includes collecting, cleaning, storing, monitoring and reporting registry data.
  • Broad range of data collection procedures and systems available.
• Critical factors in data quality:
  • Data element structure and definition, training of personnel, how data problems are handled.
• Quality assurance:
  • Define requirements at registry creation.
  • Risk-based approach.
    • Most important or likely sources of error or potential lapses in procedures that may impact quality in the context of intended purpose.

Slide 24

Adverse Event (AE) Detection, Processing, & Reporting

• Important for any registry, especially with direct patient and/or physician contact.
• Collection of spontaneously reported or solicited safety information:
  • Protocol-defined procedures strongly recommended.
  • Investigators and site staff appropriately trained.
• Processing, coding and managing of AE data.
• Reporting and regulatory requirements.

Slide 25

Best Practices for Adverse Event Reporting to FDA in Registries of Post-Marketed Products

The diagram presents the adverse event reporting requirements by product type, level of interaction and sponsorship.

Gliklich R, Dreyer N, eds. Registries for Evaluating Patient Outcomes: A User's Guide. AHRQ Pub. 07-EHC001-01 2007.

Slide 26

Analysis and Interpretation

Analysis

• Importance of a statistical analysis plan.
  • Analytic plans and statistical techniques for primary and secondary objectives.
• Report on characteristics of the patient population, exposures of interest and endpoints.

Interpretation

• Who was studied?
  • Is the actual population representative of the target population?
• How were the data collected, edited and verified?
  • Completeness of data collection and data quality.
  • How were missing data handled and reported.
• How were the analyses performed?

Slide 27

Case Examples

23 case examples are included to illustrate practical challenges and how they were addressed. E.g.:

• National Registry of Myocardial Infarction (NRMI)—"Creating a Registry to Fulfill Multiple Purposes and Using a Publications Committee to Review Data Requests".
• Get With The Guidelines (GWTG)-Stroke—"Using Performance Measures to Develop a Data Set".
• BPH—"Developing and Validating a Patient-Administered Questionnaire".
• GWTG—"Using Recognition Programs to Recruit Sites".
• OPTIMIZE-HF—"Using Registry Tools to Recruit Sites".
• ESCF—"Using Registry Data to Evaluate Outcomes by Practice.

Slide 28

How do you know what makes a registry "good enough"?

Slide 29

Evaluating Registries

• Although all registries can provide useful information, there are levels of rigor that enhance validity and make the information from some registries more useful for guiding decisions.
• We adapted quality evaluations used for trials, that address the confidence that the design, conduct and analysis of the registry protect against systematic errors and errors in inference.

Slide 30

Evaluating Registries (continued)

• Quality component analysis.
  • Research quality (scientific process).
    • Planning; design; data elements & data sources; ethics, privacy and governance.
  • Evidence quality (data/findings).
    • Patients; data elements & data sources; quality assurance (QA); analysis; reporting.
• Components classified as:
  • Basic Practice (draft names).
  • Future Directions/Potential Enhancement (draft names).

Slide 31

Research Quality: Basic

• Planning: A written study plan documents: goals, design, study population, recruitment, data collection, human subject protection, data element, sources, & review/QA. Feasibility is considered at the outset.
• Plans address how data will be evaluated, incl. what comparative information, if any, will be used to support study hypotheses or objectives.
• Design: The size required to detect an effect, should it exist, or achieve a desired level of precision is acknowledged, whether or not met.
• Follow-up time needed to detect events of interest is acknowledged, whether or not feasible to achieve. To the extent feasible, follow-up time is adequate to address the main objective.
• Data Elements: Outcomes are clinically meaningful and relevant, i.e., useful to the medical community for decision-making.
• Ethics, Privacy, and Governance: Registry has received review by required oversight committees.
• Communication Plan: for results is addressed.

Slide 32

Research Quality: Potential Enhancements

• Planning: Formal study protocol with review from key stakeholders prior to finalization. Pilot studies are useful when studying hard to reach populations or when sensitive data are sought.
• Design: Use of concurrent comparators in situations where treatments are evolving rapidly.
• Formal statistical calculations to support sample size, whether or not that size is achievable within practical constraints.
• Data Elements & Sources: Multiple methods of data collection may be required for some purposes.
• Use validated scales and tests when such tools exist for purpose needed.
• Adapt levels of QA based on observed performance.
• Coding consistent with nationally approved coding systems; standardized data dictionaries, validated assessment tools...
• Publication policies are specified in advance of collecting data.

Slide 33

Evidence Quality: Basic

• Patients: Participants are similar to the target population; attention is paid to minimize selection bias to the extent feasible.
• For safety studies, registry personnel are trained to ask about AEs in a consistent, clear & specific manner, and know how to report.
• Data Elements & Sources: Data are reasonably complete.
• QA: Reasonable efforts have been expended to assure that appropriate patients have been systematically enrolled and followed in as unbiased a manner as possible; reasonable efforts have been devoted to minimize losses to follow-up. Data are checked using range and consistency checks.
• Analysis: Accepted analytic techniques are used; they may be augmented by new or novel approaches.
• Reporting: Results are reported for all main objectives; follow-up time is described so readers can assess its impact on conclusions drawn; report clearly states any conclusions drawn and implications of results, as appropriate.

Slide 34

Evidence Quality: Potential Enhancements

• Patients: External validity is described; for comparative effectiveness and safety, contemporaneous comparison data are collected.
• Data: Results that can be confirmed by an unbiased observer enhance accuracy and reliability (e.g., death, test results, scores from validated measurements for PRO or clinical rating scales).
• QA: Potential sources of errors relating to accuracy and falsification are rigorously evaluated and quantified (e.g., through database and site reviews). For studies of safety, effectiveness and comparative effectiveness, a sample of data are compared with patient records.
• Analysis: Loss to follow-up is characterized at all stages of study conduct. For safety studies, risks and/or benefits of "exposure" are evaluated quantitatively, beyond statistical significance. Sensitivity analyses are used to examine the effect of varying the inclusion/exclusion criteria & other assumptions.
• Reporting: Inferences about causality are based on a variety of factors, including the strength of the association, biases, temporal relation, etc.

Slide 35

Some Unanswered Questions

• When should a registry end?
• What are the privacy implications of linking multiple data sources into registries?
• How can different registries or other data sources working together, interfacing.
• How should a registry be designed specifically for adverse event collection?

Slide 36

Thank You

Contact Information:
Richard Gliklich MD
President
Outcome
201 Broadway
Cambridge, MA 02139
richg@outcome.com
617-621-1600 x530
www.outcome.com