Guidelines for Consumer-focused Public Reporting (Text Version)

Slide Presentation from the AHRQ 2008 Annual Conference

On September 10, 2008, Carol Cronin, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (82 KB).

Slide 1

Guidelines for Consumer-focused Public Reporting

• AHRQ Quality Indicators (QI) Users Meeting
• Carol Cronin
  Informed Patient Institute
• September 10, 2008

Slide 2

Background

• National Quality Forum (NQF) Public Reporting Technical Advisory Panel (TAP)?
• Part of NQF Steering Committee (SC)—Hospital Care 2007 project.
• AHRQ-funded.

Slide 3

TAP Charge

• "Recommend a Web-based approach for public reporting of acute hospital quality data that can be used, at minimum, to report the AHRQ Quality Indicators."

Slide 4

Scope

• Focuses on reporting healthcare quality data from acute care hospitals in a Web-based format; though generalizable to other types of settings.
• Intended primarily for use by sponsors of consumer-focused sites to enable reports that support consumer understanding and participation in care decisions.

Slide 5

TAP Members

• Chair: Carol Cronin-Informed Patient Institute, Annapolis, MD.
• Katherine Brown, Hospital Quality Alliance, Washington, DC.
• Susan Dragoo, INTEGRIS Health, Oklahoma City, OK.
• Judy Hibbard, University of Oregon, Eugene, OR.
• Art Levin, Center for Medical Consumers, New York, NY.
• Denise Love, National Assoc. of Health Data Orgs., Salt Lake City, UT.
• Jeanne McGee, Mcgee & Evers Consulting, Vancouver, WA.
• Deirdre Mylod, Press Ganey Assoc., Inc., South Bend, IN.
• Ramesh Sachdeva, Children's Hospital and Health System, Milwaukee, WI.
• Maribeth Shannon, California HealthCare Foundation, Oakland, CA.
• Bruce Spurlock, CA Hospital Assesment & Reporting Taskforce, Roseville, CA.

• Project Staff: Melinda Murphy-NQF Consultant.

Slide 6

Approach

• Literature review.
• Review of existing Web-based resources—Talkingquality.gov, Usability.gov.
• Structured interviews with researchers, report sponsors.
• Iterative development.

Slide 7

AHRQ Model Report

• Designed to report comparative hospital QIs.
• Developed by researchers including Shoshanna Sofaer—Baruch College.
• Included testing with consumers.
• Two resulting model reports:
  • Health Topics—individual QI indicators grouped into categories.
  • Composites—QI composites.

Slide 8

Report Sponsor Challenges

• Understanding what constitutes a useful report.
• Assuring reports support consumer understanding of quality and healthcare choice.
• Conveying information that provides the right amount of information at the right time in a way that resonates with target audiences.
• Customization of results to the audience.

Slide 9

TAP Principles

• Public and other stakeholders have right to access objective measures of quality provided by organizations in/from which they receive care, deliver care, purchase care, or provide funding/regulation and to receive the information in an understandable format.
• Use of the public reporting guidance can/should provide incentives for quality improvement.
• Information should be displayed for an array of common, cross-cutting conditions, all ages and be available across organizational settings/service lines and over time.
• Awareness and understanding of inherent values and biases is important to responsible reporting.

Slide 10

Goals of Consumer-focused Public Reporting

• Increase consumer motivation to use public reports by making reports more understandable and relevant.
• Provide objective, unbiased, actionable and evaluable performance information to the public.
• Improve quality of care provided across the industry; and
• Stimulate further evolution of the quality and comparability of public reporting at the organization, state and national levels.

Slide 11

Guidance for Consumer-focused Public Reporting—Overview

• TAP identified guidelines for public reporting in 7 areas.
• Also included Implementation Considerations that amplify the guidelines.
• Assessed AHRQ Model Reports against guidelines.
• Includes AHRQ Model reports in Appendix.

Slide 12

Guideline 1: Purpose of Web-based Report

• Identify the purpose of the Web-based report, its intended main consumer audience(s), and how the report will be made known to the audience.
  • Nature and purpose.
  • Who is the audience? What are their info needs?
  • Who are secondary audiences and how will their unique needs be addressed?

Slide 13

Guideline 2: Process of Developing the Web-based Report

• Develop the Web-based report using a transparent process that involves consumers and other relevant stakeholders.
  • Stakeholders include: sponsors, consumers, organizations being measured.
  • Establish governance/decision making rules.
  • Opportunity for those being measured to preview and correct errors.
  • Consumer usability testing before, during, after.

Slide 14

Guideline 3:Introduce concept of quality

• At the beginning of the report, set the stage by communicating what quality is, how quality varies and how making quality comparisons can be of value to consumers.
  • Explain that quality varies within and across orgs.
  • Use consistent, simple and familiar language.

Slide 15

Guideline 4: Meaningful Measures

• Ensure that measures included are meaningful to consumers, transparent, and meet widely accepted, rigorous criteria including importance, scientifically acceptable, feasible and usable.
  • Relevant to consumers.
  • Demonstrate variation.
  • Provide information that reflects overall quality.

Slide 16

Guideline 5: Data Presentation

• Present and explain data clearly and objectively in ways that help consumers understand and use it.
  • Data should be evaluable.
  • Consistent reporting (low or high scores better)�
  • Use stories to illustrate meaning.
  • Allow users to see what they want.

Slide 17

Guideline 5: Data Presentation (cont.)

• In presenting comparative quality information.
  • Use tools such as rank ordering, symbols that help user quickly discern performance.
  • Include benchmarks if possible.
  • Provide risk-adjusted rates into categories using words such as "better", "average."
  • Label indicators in everyday language.
  • Limit use of statistics that are difficult for consumers to understand.

Slide 18

Guideline 5: Data Presentation (cont.)

• In presenting information from composite measures:
  • Report all measures that comprise the composite.
• In providing contextual/decision support:
  • Provide clear contextual information.
  • Provide clear explanation for missing data.
  • Consumer test.
  • Use reasonably current data and display dates covered.
• In presenting technical documentation:
  • Include detailed measure definitions, specs etc.
  • Provide details about the methodology.

Slide 19

Guideline 6: Report Usability

• Ensure that design and navigation features enhance report usability.
  • Organize information to let users make choices.
  • Provide engaging format/easy navigation.
  • Easy to skim.
  • Easy to print.
  • Test with audience.

Slide 20

Guideline 7: Regularly Review and Improve Reports

• Regularly review and assess reports to ensure their effectiveness and currency.
  • Assess use and impact.
  • Involve stakeholders in revisions.
  • Use learnings to drive improvement and usefulness of performance measures and public reporting field.

Slide 20

Research Recommendations

• Impact of public reporting.
• Content of public reporting.
• Unintended consequences.
• Understanding issues related to cultural and linguistic needs.
• Effect of public reporting on quality improvement.

Slide 22

Project Status

• Public comment period: June 2008.
• Final NQF member voting: Aug. 2008.
• Consensus Standards Approval Committee (CSAC)/Board consideration: Sept. 2008.
• Publicly available: Late 2008.

Slide 23

For more information:

• National Quality Forum: www.qualityforum.org.
  • "Guidelines for Consumer-focused Public Reporting."
• AHRQ Model Reports
  • Send request to: support@qualityindicators.ahrq.gov
• Talking Quality: https://talkingquality.ahrq.gov