УTalking about hospital quality data: reactions and reality from patients and doctorsФ American Institutes for Research (AIR) Ц Summary of project goals and methods 10th National CAHPS User Group Meeting, Baltimore, Maryland, March 29-31, 2006 (This document provides background on project goals and methods to accompany a presentation of our research findings at the User Group Meeting. Please do not cite or quote it. For more information, contact Kristin Carman.) Project team AIR Project Director: Kristin L. Carman, PhD Co-investigators (consultants): Kelly J. Devers, PhD Jeanne McGee, PhD Judith H. Hibbard, DrPH Richard M. Frankel, PhD AIR Project Staff: Pamela Dardess Karen Frazier Anna Levin (For contact information, see the end of this handout) Partner sites Х The Alliance (Employer Health Care Cooperative) in Madison, Wisconsin (Cynthia Schlough, Manager, Strategic Connections) Х Virginia Commonwealth University, Department of Family Medicine in Richmond, Virginia (Anton J. Kuzel, MD, M.H.P.E. Professor and Chairman) Funded by Х Agency for Healthcare Research and Quality, CAHPS II Cooperative Agreement Х Centers for Medicare & Medicaid Services Purposes of this study Х Compare how physicians and patients feel about engaging in discussions about hospital quality information Х Compare how both groups would use hospital quality information to assess hospitals and make hospital choices Х Identify approaches and tools to help physicians and patients discuss and use hospital quality information productively Potential long-term outcomes Х Physicians and patients gain greater understanding of hospital quality information and the implications for referral and choice Х Discussions of hospital quality information between physicians and patients become more common and candid; there is improved communication between physicians and patients Х Decisions about hospital choice become more fully informed and more transparent Х Patients maintain or increase their trust in physicians and the medical care system Overview of study design -Ten focus groups with patients and with doctors. We conducted five groups with patients; three with primary care doctors; and two with specialists. -Background information on quality measurement and reporting. To introduce our topic and provide a common starting point for all focus group participants, we started each session with a presentation of background information on quality measurement and reporting. We used a slide show with taped narration to standardize this introductory material. -Innovative focus group method using audiotaped vignettes as trigger devices. To raise issues and trigger discussion about use of hospital quality data by doctors and patients, we created three brief doctor-patient conversations. These vignettes were recorded by professional actors (see below), and we played them one at a time during the groupsessions, with group discussion after each one. In the patient groups, we gave participants a comment sheet to fill out after each tape had been played but before the group discussion began. This gave them an opportunity to think about the vignette and record their impressions privately before hearing the reactions and opinions of others in the group. These written comment sheets allowed us to compare peopleТs initial private impressions with the views that evolved during group discussion. We concluded the sessions with a more general group discussion about quality reports and how doctors and patients might use them. -Two locations. We conducted our focus groups in Madison, Wisconsin, and Richmond, Virginia. These sites were chosen to differ on extent of exposure to publicly reported quality data, hospital and physician alignment, and other factors that might affect physician referrals such as payment arrangements. About the audiotaped vignettes -Brief simulated doctor-patient conversations that include discussion of hospital quality reports. Each vignette is a two or three minute excerpt from a fictional conversation that takes place at the doctorТs office. In each vignette, the patient has a medical condition that will require hospitalization, and the patient brings up the topic of hospital quality reports. -Three carefully selected medical conditions. Each vignette covered a different reason for hospitalization: hip replacement; coronary artery bypass graft; pregnancy. All three (a) are among the most common reasons for hospitalization; (b) are conditions for which measures of quality exist; (c) typically do not require immediate hospitalization and therefore permit patients to consult quality information as part of selecting a hospital. -Vignettes featured common situations and trade-offs that might be made in choosing a hospital. Since the purpose of the vignettes was to trigger group discussion, each was crafted to emphasize typical factors that might come up in using quality information to help make a decision about which hospital to use (see below for details). Each vignette ended before the decision was made. -Extensively pretested and professionally produced. To verify realism and refine the scripts, we conducted multiple rounds of pretesting with both consumers and doctors. We recorded the final vignettes in a professional studio using experienced actors. -To avoid confounding influences, we held certain doctor and patient characteristics constant for all vignettes. Our study was not large enough to systematically examine the impact of doctor and patient demographics. Therefore, to minimize possible confounding, each vignette had a female patient and male doctor. To minimize the impact of physical appearance, we chose to audiotape the vignettes rather than videotape them. We selected professional actors with voices that were age- appropriate, with no obvious regional accents. Vignette # 1 --Hip replacement -The situation: Mrs. Egan, a 68 year old patient with Medicare, needs a hip replacement. She is talking with Dr. Norris, her family doctor, about where to have it done. -The hospital choices: Two local hospitals are Annandale and Westside. Х Dr. Norris usually refers hip replacements to a surgeon who practices only at Annandale Hospital. He has never sent a hip replacement patient to Westside. Х Mrs. EganТs granddaughter was born at Westside last month and Mrs. Egan was impressed by Westside. Х Mrs. Egan saw a report that compares quality ratings for both hospitals. Westside looks better for hip and knee replacements. -How the patient got the hospital quality report: Her son who Уknows all about computersФ found it on a government website and showed it to her -Hospital quality measures that are mentioned: Х Measures of clinical quality: giving antibiotics to prevent infection after surgery (hospitals are about the same); excess bleeding (Westside is better); unplanned readmission (Westside is better). Х Measures based on patient experiences: discharge information: УЕ giving patients information when theyТre ready to leave the hospital. Like telling them what problems to watch out forЕФ (Westside is better). -Main themes in this vignette: Х Doctor usually refers to a surgeon at a local hospital vs. patientТs family has had good experiences at a different local hospital that has better quality scores Vignette # 2 Ц Heart bypass surgery (coronary artery bypass graft or CABG) -The situation: Mrs. Bailey, a 62 year old patient, needs heart bypass surgery. She is talking with Dr. Addison, her family doctor, about where to have it done. -The hospital choices: For bypass surgery, the two hospital options are Valley View and Oakdale: Х Valley View is the local hospital. It is more familiar to Mrs. Bailey and would be more convenient for her family and friends to visit. She knows some of the staff and likes the idea that her family doctor might be able to look in on her. Х Oakdale is the distant hospital; it takes an hour and a half to drive there. The government report on hospital quality shows that Oakdale does better than the local hospital in areas relevant to heart surgery. -How the patient got the hospital quality report: A neighbor gave her a copy of a government report that compares the two hospitals. -Hospital quality measures that are mentioned: Х Measures of clinical quality: infection control (Oakdale is better). Х Measures based on patient experiences: patient ratings of nursing care (Oakdale is better); how well staff explain medicines before giving them to patients (hospitals are about the same). Х Other: volume (Dr. Addison notes that surgeons at Oakdale do more bypass surgeries than Valley View); Mrs. Bailey notes that length of stay for bypass surgery is shorter at Oakdale (Dr. Addison says this is a Уgood sign;Ф Mrs. Bailey says sheТs confused and asks Dr. Addison to explain how it can be a good sign to send patients home sooner). -Main themes in this vignette: Х Local hospital is familiar and convenient vs. distant hospital has better quality scores Х Patient asks doctor to explain something thatТs confusing in the quality report (length of stay Ц itТs counterintuitive to her) Vignette # 3 Ц Pregnancy -The situation: Mrs. Thomas, a 30 year old patient, is four months pregnant. She is talking with Dr. Welker, her obstetrician. She chose him based on recommendations from family and friends. She has seen him several times and is happy with the care sheТs getting. -Two hospitals are compared: Х Hawthorne Hospital is the one Dr. Welker uses for all of his deliveries. Х In the quality report, Morrison Hospital looks better for labor and delivery. -How the patient got the hospital quality report: Just recently got it from her employer: УLast week at my office, they sent around a government report that compares the quality of hospitals in our area.Ф -Hospital quality measures that are mentioned: Х Measures of clinical quality: severe lacerations during delivery (Уbad tearing and bleedingФ) (Hawthorne is worse); pain control for women in labor (Hawthorne is better). Х Measures based on patient experiences: patientsТ ratings of how well nurses communicate (Hawthorne is worse). -Main themes in this vignette: Х Four months pregnant; sheТs happy with her doctor vs. she sees a quality report and is concerned that the hospital he uses scores low on quality Х Doctor tells patient about how the hospital is working to improve quality; patient is concerned about how long that might take, since her baby is due in five months. Data analysis -Unique opportunity to systematically compare how patients and doctors react to the same vignettes. Since we used the same background presentation about quality reports and the same audiotaped vignettes in all of our focus groups, we are able to compare the perspectives of patients and doctors. We are also able to compare reactions of primary care doctors and specialists. -Multiple sources of data. Data sources include (a) written transcripts of the focus group discussions; (b) written comment sheets filled out by participants in the patient focus groups after each vignette was played, and before group discussion began; and (c) debriefings conducted by the project team after each session (based on written notes and audiotapes). -Analysis uses coding that was developed deductively and inductively. We developed a preliminary code list based on project objectives, then refined this list by having several team members code a full transcript for a doctor group and a patient group. The basic code structure included domains such as: feelings about the doctor or patientТs communication; barriers to and facilitators of discussions about hospital quality data; usefulness of hospital quality reports; best (and worst) possible outcome for the situation in the vignette; best (and worst) possible outcome for the doctor-patient relationship; and recommendations or advice one would give to others about using hospital quality data. Codes were then applied to all transcripts at the paragraph level using computer software designed to facilitate such analysis (Atlas.ti). All codes for a certain domain were then retrieved and analyzed for general themes. We also created code families and/or code combinations to assess similarities and differences by vignette, by type of participant (patients, primary care doctors, specialists), and by patient demographics (such as comparing reactions by level of education). We used a variety of well-established qualitative data analysis techniques to draw our conclusions and assess the rigor of our findings. Findings and implications -Our presentation at the CAHPS User Group Meeting will share the findings from this study and discuss their implications. We will identify the major themes and illustrate them with quotations from the focus group participants. Contact information Kristin L. Carman, PhD Principal Research Scientist American Institutes for Research 1000 Thomas Jefferson St., NW Washington, DC 20007 202-403-5090 kcarman@air.org Kelly J. Devers, PhD Associate Professor Department of Health Administration and Family Medicine Virginia Commonwealth University Grant House 1008 East Clay Street P.O. Box 980203 Richmond, VA 23298-0203 804-828-5509 kjdevers@mail1.vcu.edu Jeanne McGee, PhD President, McGee & Evers Consulting, Inc. 1924 NW 111th Street Vancouver WA 98685 360-574-4744 jcmgee@pacifier.com Pamela Dardess Research Analyst American Institutes for Research 101 Conner Drive, Suite 301 Chapel Hill, NC 27514 919-918-2311 pdardess@air.org Anna Levin Research Assistant American Institutes for Research 101 Conner Drive, Suite 301 Chapel Hill, NC 27514 919-918-2321 alevin@air.org Judith H. Hibbard, DrPH Professor University of Oregon Department of Planning, Public Policy, & Management 119 Hendricks Hall Eugene, OR 97403-1209 541-346-3364 jhibbard@oregon.uoregon.edu Richard M. Frankel, PhD Professor of Medicine, Indiana University School of Medicine Regenstrief Institute RG-6 1050 Wishard Blvd Indianapolis, IN 46202 317-630-8434 rfrankel@iupui.edu