The Evolution of CAHPS: A 20 Year Perspective

Slide 0

The Evolution of CAHPS: A 20 Year Perspective

Christine Crofton Agency for Healthcare Research and Quality
Susan Edgman-Levitan John D. Stoeckle Center for Primary Care Innovations, Massachusetts General Hospital
Caren Ginsberg Agency for Healthcare Research and Quality

Monday October 5, 2015, 1:30 to 3:00
Crystal Gateway Marriott Hotel and Convention Center

Slide 1

Agenda

• What are the major lessons across the past 20 years?
• How has CAHPS changed patient assessment and patient-centered care?
• Taking stock: Where are we now?

Slide 2

Evolution of CAHPS, Part I

What are the major lessons learned across the past 20 years?

Christine Crofton

Slide 3

Evolution of CAHPS

	1995	2015
CAHPS data collected from:	10M	Over 146M people
N of surveys:	1 Health Plan	6+ Ambulatory care 10+ Facility care 6+ Supp item sets

 

Slide 4

Evolution of CAHPS, cont'd

	1995	2015
Organizations Collecting CAHPS data:	NCQA CMS Medicare	NCQA CMS Medicare CMS CMMI CMS Healthcare Exchanges State Medicaid agencies US OPM US DOD Acute care hospitals Hemodialysis facilities Home health care agencies

 

Slide 5

Evolution of CAHPS, cont'd

	1995	2015
Organizations collecting CAHPS data:		Healthcare Exchange insurers Outpatient surgical centers Accountable care orgs

Coming soon:

Emergency Department
Hospice
In-center rehabilitation facilities
Cancer care
Long-term care facilities

Slide 6

 

Evolution of CAHPS, cont'd

	1995	2015
Uses of CAHPS data:	Consumer choice Large purchasers Accreditation	Consumer choice Large purchaser Accreditation Pay for Performance Quality Improvement Outcome measurement Policy decisions
Communication of survey results:	Print media	Electronic media

 

Slide 7

Lesson 1: Design Principles

Develop Design Principles

• To ensure reliable and valid data
• To promote transparency
• To enable other organizations to produce high quality CAHPS data

Slide 8

Design Principles

• Emphasis on consumers/patients
• Extensive testing with consumers
• Reporting about actual experiences
• Standardization across materials, procedures
• Multiple versions for diverse populations

Slide 9

Principle 1: Emphasis on Patients

Only the patient knows:

• How well their pain was controlled during a hospital stay
• Whether a provider explained things in a way that was easy to understand
• How often the provider’s office staff treated him or her with courtesy and respect.

Slide 10

Discovering What Patients Want to Know

• Focus groups with members of target population
• Focus groups with other individuals
• Literature reviews
• Environment scans

Slide 11

Principle 2: Extensive Testing with Consumers

Cognitive testing

• Confirms that items, response options are understood as developer intended
• Is conducted in iterative rounds
• In English and in Spanish
• Participant ‘thinks out loud’ while completing the questionnaire or
• Participant is interviewed in detail after completing the questionnaire

Slide 12

Principle 2: Extensive Testing with Consumers, cont'd

Field testing

• To assess the effectiveness and feasibility of survey administration procedures and guidelines
• To determine validity, reliability and other psychometric properties

Slide 13

Principle 3: Reporting About Actual Experiences

Survey focus = Patient experience of care rather than simple satisfaction

Slide 14

Principle 3: Reporting About Actual Experiences, cont'd

Reports of experience are more:

• Actionable
• Understandable
• Specific
• Objective

than general ratings.

Slide 15

Principle 3: Reporting About Actual Experiences, cont'd

How satisfied were you?
vs.
How often did this provider:

• Explain things in a way you could understand?
• Treat you with courtesy and respect?
• Listen carefully to you?
• Spend enough time with you?
• See you within 15 minutes of appointment time?

Slide 16

Principle 4: Standardization

Instrument

• Every user administers items the same way

Protocol

• Sampling, communicating with potential respondents, and data collection procedures are standardized

Analysis

• Standardized programs and procedures

Reporting

• Standard reporting composites and presentation guidelines

Slide 17

Principle 5: Multiple Versions for Diverse Populations

Designed for all types of users

• Medicare
• Medicaid
• Commercial population

In English and Spanish

Slide 18

Lesson 2: Identify and include stakeholders

• Include key stakeholders in every phase of the design and development process

Slide 19

Who are the key stakeholders in CAHPS?

CAHPS Consortium

Grantees—RAND and Yale
User Network Contractor—Westat
AHRQ CAHPS team

High-volume CAHPS users

CMS
NCQA

Slide 20

Key CAHPS stakeholders, cont’d

Consumers

Published research articles
Published survey results
Focus Groups
Cognitive Testing
Consumer advocacy organizations
Public comment process

Slide 21

Key CAHPS stakeholders, cont’d

Technical expert panel
Content specialists
Co-funders
Field test sites
Data vendors
Government organizations (OMB, HHS, Congress)
Gatekeepers to target audience
Professional associations
Dissemination and promotion team

Slide 22

Standardized Procedures and Analyses Ensure High-Quality, Comparable Survey Data

• Implementation procedures:
  • Authorized survey vendors must meet minimum business requirements and complete training
  • Vendors must follow detailed guidelines regarding sampling protocols, modes of survey administration, and data coding and data file preparation
• Case-mix adjustment aims to “level the playing field”:
  • To remove predictable effects of differences in patient characteristics, statistical models predict what each provider’s score would be for a standard patient population

Slide 23

Evolution of CAHPS, Part II

How has CAHPS changed patient assessment and patient-centered care?

Susan Edgman-Levitan

Slide 24

Impact on the Patient’s Experience of Care

• CAHPS Improvement Guide published in 2003:
  • Most popular item on the AHRQ CAHPS website
  • Currently being updated

Slide 25

CAHPS Improvement Guide

Image: Screen shot of the older version of The CAHPS Improvement Guide main page.

Slide 26

Topics Across the Clinician & Group and Health Plan Surveys

• Access to care
• Provider communication
• Customer service
• Care coordination
• Shared decision making
• Comprehensiveness
• Health promotion and education
• Self-management
• Access to specialists
• Cultural competence
• Plan information
• Cost of care
• Overall rating

Slide 27

Impact of Public Reporting and VBP

Image: Line graph titled "Hospitals are improving patient experience measures". The graph shows improvements from 2008 to 2012 on a variety of measures.

Slide 28

CAHPS Health Plan Survey Improvements

Images: Screen shots of sample tables showing comparisons of different health plan ratings of 8, 9, or 10 and 9 or 10 from 2006 to 2012.

Slide 29

CG-CAHPS Improvement

Image: Bar chart titled 2011-2012 Clinician & Group Visit Survey showing slight improvements in access, provider communication, office staff, and provider rating from 2011 to 2012.

Slide 30

Internal Organizational Factors to Support Improvement

1. Top leadership engagement,
2. A strategic vision clearly and constantly communicated to every member of the organization,
3. Involvement of patients and families at multiple levels,
4. A supportive work environment for all employees,
5. Systematic measurement and feedback,
6. The quality of the built environment; and,
7. Supportive information technology.

Shaller D. “Patient-Centered Care: What Does It Take?” New York: The Commonwealth Fund. Publication No. 1067, November 2006.

Slide 31

External Factors to Support Improvement

1. Public reporting of standardized measures
2. Value-based purchasing,
3. Accreditation and certification requirements, and;
4. Growing demand for accountability and transparency by consumers and patients

Slide 32

Do Healthcare Leaders Care?

Image: Screen shot of chart titled: HCAHPS scores and leadership are key factors in providing great PX. Six different measures are rated.

Slide 33

Better Care Experiences are Associated with Better Patient Adherence

• Zolneriak & Dimatteo (2009) meta-analysis of 127 studies shows:
  • Higher non-adherence among patients whose physicians communicate poorly
  • Substantial improvements in adherence among patients whose physician participated in communication skills training
• Better patient-reported provider communication related to higher:
  • Diabetics’ adherence to hypoglycemic medication (Ratanawongsa et al., 2013)
  • Veterans’ diabetes self-management (Heisler et al. 2002)
  • Blacks’ hypertension medication adherence (Schoenthaler et al. 2009)
  • Breast cancer patients’ adherence to tamoxifen (Kahn et al. 2007; Liu et al. 2013)
  • Rates of colorectal cancer screening (Carcaise et al. 2008)
  • Preventive health screening and health counseling services (Flocke et al. 1998)
• Greater patient trust in physician related to:
  • Better adherence to diabetes care recommendations (Lee & Lin 2009)
  • More preventive services among low-income Black women (O’Malley et al. 2004)

Slide 34

Better Care Experiences are Often Associated with Better Care Processes

• Jha et al. (2008) found that hospitals with highest HCAHPS scores did better on clinical processes of care measures, including acute myocardial infarction (AMI), congestive heart failure (CHF), pneumonia, and surgery than hospitals with lowest scores.
• Patients’ overall ratings of hospitals were positively associated with hospital performance on pneumonia, CHF, AMI, and surgical care (Isaac et al. 2010) and process indicators for 19 different conditions (Llanwarne et al. 2013).
• Overall ratings and willingness to recommend hospital were lower in hospitals that consistently perform poorly on cardiac process measures (Girota et al. 2012).
• Findings regarding associations between outpatient experiences of care and care processes are mixed.

Slide 35

Better Care Experiences are Often Associated with Better Clinical Outcomes

• Positive patient experiences may provide unique benefit to clinical outcomes for AMI patients over and above clinical quality performance:
  • Meterko et al. (2010): Better patient-centered hospital care associated with better 1-year survival, controlling for comorbidity, clinical, and demographic factors
  • Glickman et al. (2010): Higher patient ratings associated with lower hospital inpatient mortality, controlling for hospitals’ clinical performance
• One much-publicized study (Fenton et al. 2013) reported a negative relationship between patient-provider communication with all providers seen in the last year and total health care and prescription drug spending, inpatient admissions, and mortality.

Slide 36

No Inherent Trade-Off Between Strong Performance on Patient Experience and Other Quality Performance

Among dozens of studies examined in a recent systematic review, the vast majority found either positive or null associations between patient experiences and best practice clinical processes, lower hospital readmissions, and desirable clinical outcomes.

Anhang Price R, Elliott MN, et al. 2014. "Examining the role of patient experience surveys in measuring health care quality." Medical Care Research & Review. 71(5):522-54

Slide 37

.

Beyond Public Reporting and Pay for Performance, There is a Business Case for Patient Experience

• Patients keep or change providers based upon their experiences of care:
  • Lied et al. (2003) reported that the mean voluntary disenrollment rate was 4 times higher for health plans in the lowest 10% of overall plan ratings compared to those in the highest 10% in the CAHPS Health Plan survey.
• Better patient-reported experiences correlate with lower medical malpractice risk:
  • Fullman et al. (2009) found that for each drop in minimum satisfaction along a five-step scale of “very good” to “very poor,” the likelihood of being named in a malpractice suit increased by 21.7%.
• Efforts to improve patient experience may also result in greater employee satisfaction, reducing turnover:
  • Rave et al. (2003) described how a focused endeavor to improve patient experience at one hospital also resulted in a 4.7% reduction in employee turnover.

Slide 38

Part III – Taking Stock: Where Are We Now?

Caren Ginsberg, PhD
AHRQ

Slide 39

Where Are We Now?

• Tremendous growth over the past 20 years:
  • Number of surveys
  • Uses for the surveys
  • Languages
  • Patients reached
  • Facilities/health plans covered
• All with using the same CAHPS design principles
• Demonstrable improvements

Slide 40

Taking Stock

• Consumer use of CAHPS data
• Managing requests for new surveys
• Education about the value of patient experience
• Keeping surveys current
• Data collection
• AHRQ’s CAHPS Consortium’s unique role

Slide 41

Use of CAHPS Data for Consumer Choice

• Are consumers using CAHPS information?
• What information are consumers looking for?
• What information are consumers using?
  • Patient experience scores
  • Narrative comments

Slide 42

Managing Requests for New Surveys

• Prioritizing need for new instruments vs. use of existing core and supplemental items:
  • Examples: PCMH, HIT, Health Literacy

Slide 43

v

Maximizing Education about the Value of Patient Experience Feedback

• Ongoing need to educate healthcare leaders, clinicians, administrators and staff about the value of patient experience feedback:
  • Patient experience vs patient satisfaction
  • Myths about CAHPS surveys
  • VBP and public reporting

Slide 44

Keeping Surveys Current

• Updating survey items, sampling, and data collection options across multiple stakeholders
• Goal: avoid disruption in reporting and ongoing survey efforts/ consider budget and time constraints

Slide 45

Data Collection

• Electronic Survey Administration:
  • Is it feasible?
  • What will it look like?
  • What are our priorities?

Slide 46

AHRQ’s CAHPS Consortium Unique Role

• Neutral convener
• Science partner
• Manages broad stakeholder input
• Maintains integrity of products

Slide 47

Questions?

Comments?

Return to 2015 AHRQ Research Conference Page