Developing a Metric Set for Measuring and Reporting Ambulatory Quality of Care in the Setting of Health IT with HIE

Slide Presentation from the AHRQ 2008 Annual Conference

On September 8, 2008, Lisa M. Kern, MD, MPH; Rina V. Dhopeshwarkar, MPH; and Rainu Kaushal, MD, MPH, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (185 KB).

Slide 1

Developing a Metric Set for Measuring and Reporting Ambulatory Quality of Care in the Setting of Health IT with HIE

Lisa M. Kern, MD, MPH
Rina V. Dhopeshwarkar, MPH
Rainu Kaushal, MD, MPH
HITEC
Cornell University
New York-Presbyterian Hospital
September 2008

Slide 2

Background

• ¾ of States are pursuing development and implementation of health information exchange (HIE).
• New York State is investing $250 million in infrastructure for health information technology (health IT) and HIE
  • Largest State-based investment of taxpayer dollars

• Note: www.staterhio.org, www.health.state.ny.us/technology

Slide 3

HEAL NY Program

• HEAL NY: Healthcare Efficiency and Affordability Law for New Yorkers Capital Grants Program
  • Funding distributed in waves:
    • 1st wave in 2006
    • 2nd wave in 2008
  • Projects include both health IT and HIE.
  • Grantees were required to evaluate the effects of their projects.

Slide 4

HITEC: The Health Information Technology Evaluation Collaborative

• A formal collaborative of 4 universities in New York (Cornell, Columbia, SUNY Albany, University of Rochester).
• Established with the endorsement of the New York State Department of Health.
• Established to conduct rigorous evaluations of HEAL NY projects in order to maximize learning and produce generalizable results.

Slide 5

How do you measure the impact on health care quality of health IT with HIE?

Slide 6

Limitations of Existing Metric Sets

• Existing metric sets developed to evaluate the quality of healthcare delivered in an ambulatory care setting:
  • Rely on manual chart review (expensive and laborious)
  • Claims data (lack clinical nuance)
  • Do not presume communication between health care providers
  • Not designed to take into account incremental effect of receiving clinical data from outside sources

Slide 7

Specific Aims

1. Develop a modified set of quality metrics that can be retrieved electronically and is sensitive to the types of improvements in quality that health IT with HIE may contribute to the ambulatory care setting.
2. Validate the modified quality metrics set through review by a panel of national experts in quality measurement and national experts in HIE.
3. To test the reliability of electronic retrieval of the modified quality metrics set, by comparing electronic retrieval to manual retrieval.
4. To evaluate the long-term effects of using health IT with HIE on improving health care quality, using the modified metric set.

Slide 8

Conceptual Framework

• "Sensitivity to EHR [electronic health records] with HIE"
  • Clinical Data Residing Elsewhere
  • HIE
  • Electronic Receipt of Clinical Data by Health Care Provider with an EHR at or Before the Point of Care
  • Medical Decision Making
• "Suitability for Electronic Reporting"
  • Medical Decision Making
  • Electronic Reporting
  • Quality Report

Slide 9

Overall Methodology in Brief

• Conduct a literature review for existing ambulatory care quality metric sets.
• Determine if any of the metrics retrieved should be excluded.
• Articulate the domains and assumptions upon which each metric would be rated.
• Rate the existing metrics.
• Develop novel metrics.

Slide 10

Inclusion Criteria for Metric Sets

• Included metric sets had to be:
  • Endorsed by:
    • A national quality organization.
    • A national professional organization.
    • A national research organization.

    • OR

  • Specifically address quality of transitions across health care settings
• Included metric sets could be general or disease-specific

Slide 11

Exclusion Criteria

• Not in the ambulatory setting:
  • Emergency department care was excluded.
• Not adult primary care:
  • Obstetrics, pediatrics, cancer care and HIV care were excluded.
• Provider, practice or health plan characteristics
• Satisfaction or experience of patients or providers

Slide 12

Metric Selection (continued)

• 17 metric sets=1064 metrics:
  • Extract 139 duplicates=925 metrics
  • Extract 423 excluded=502 metrics

Slide 13

Rating Process: Round One

• Each metric was reviewed by 2 raters on 2 dimensions, each on a scale from 0-6.
  • Impact of HIE on medical decisionmaking
  • Suitability for electronic reporting
• Ratings were summed across dimensions and averaged across raters
• 59 metrics scored high (≥9 out of 12)

Slide 14

Rating Process: Round Two

• Each metric was reviewed by several raters on 5 dimensions, each on a scale from 0-6.
  • Feasibility of delivering data electronically
  • Impact on medical decision making
  • Clinical importance
  • Feasibility of reporting data electronically
  • Global rating (4-7 raters for each metric)
• Ratings were averaged across raters for the global rating
• 18 scored high (≥4 out of 6)

Slide 15

Diseases Represented by Top-Scoring Existing Metrics

• Asthma (1 metric)
• Cardiovascular Disease (3)
• Congestive Heart Failure (1)
• Diabetes (4)
• Medication/Allergy Management (2)
• Mental Health (1)
• Osteoporosis (1)
• Prevention (5)

Slide 16

Novel Metrics

• Developed through an iterative process with national quality experts.
• Cover topics related to efficiency and coordination of care:
  • Test Ordering (3 metrics)
  • Medications (4)
  • Referrals (2)
  • Revisits (3)

Slide 17

Next Step

• Test reliability of electronic reporting vs. manual chart review for selected existing metrics and for novel metrics

Slide 18

Blank Slide

Slide 19

Assumptions for Determining EHR+HIE Sensitivity

• We assumed the perspective of a primary care physician who has the following characteristics:
  • Is board-certified and competent
  • Has been in a community-based practice x 10 years
  • Has a relatively stable panel of patients
  • Has an electronic health record (EHR), which is linked only to generalist partners
  • Is in a practice with the technical capacity to participate in an HIE

Slide 20

Factors Relevant for Rating EHR+HIE Sensitivity

1. Whether needed data elements would be missing in the absence of HIE (Relevance)
2. Ease of electronic transmission of data elements to the provider (Feasibility)
3. Impact of electronic transmission (Impact) on:
   • Processes of care and/or patient outcomes
   • Utilization

Slide 21

Factors Relevant for Rating Suitability for eReporting

1. How commonly this metric appears in other quality metric sets (Importance)
2. How often the data needed for this metric are currently structured (Feasibility)
3. If data are not currently structured, how easy would it be technically to create a structured format (Feasibility)

Slide 22

Factors Relevant for Rating Suitability for eReporting

4. How much electronic reporting would rely on providers' style of documentation (Physician burden)
5. How valid an electronically reported version of this metric would be (Validity)