Psychometric Properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey

Select to access the PowerPoint® presentation (470 KB).

Slide 0

Psychometric Properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey

September 11, 2012
Naomi Dyer, PhD, Joann Sorra, PhD, Scott Smith, PhD, Westat
Paul Cleary, PhD, Yale
Ron Hays, PhD, RAND

Slide 1

Background

• CAHPS is a multiyear initiative of AHRQ to support and promote the assessment of consumers' experiences with health care.
• Numerous CAHPS surveys have been created for different health care settings such as:
  • Health Plan.
  • Surgical Care.
  • Home Health Care.
  • Hospital.
  • In-center Hemodialysis.
  • Nursing Home.
  • Clinician & Group..
  .

Slide 2

Background

• Westat, under contract with AHRQ, houses the database for Clinician and Group CAHPS (CG-CAHPS) and reports on these data.

Image: A screenshot shows sample Percentile Top Box Scores from a CAHPS report, and the back of a viewer's head.

Slide 3

Background

• There are several CG-CAHPS Surveys:
  • CG-CAHPS 12-month Survey (Adult and Child).
  • CG-CAHPS 12 month Survey with Patient-Centered Medical Home.
  • CG-CAHPS Visit Survey (Adult and Child).
• CG-CAHPS Database is the newest component of the CAHPS Databases.
  • The CG-CAHPS Adult Visit Survey database received the most data in 2011.

Slide 4

Background

• The CG-CAHPS Visit surveys were created to allow patients to report on and evaluate their experiences during their most recent outpatient medical visit:
  • The 12 month versions ask about their experiences in the previous year.
• Surveys and related material available at: http://cahps.ahrq.gov/clinician_group/.

Slide 5

Survey Development

• Goal: Develop a survey to assess patients' experiences with medical groups and clinicians.
• Development of the Visit Survey began in 2008.
• Field testing in 2009 to compare with the 12-month survey version:
  • Results led to the Access to Care items in the survey reverting to a 12-month reference period rather than visit-specific.

Slide 6

CG-CAHPS Composites

• 3 composites:
  1. Access to Care (5 items).
  2. Doctor Communication (6 items).
  3. Courteous/Helpful Staff (2 items).
  .
• 2 overall rating items:
  1. Overall Doctor Rating (11-point scale; 0 = "Worst doctor possible", 10 = "Best doctor possible").
  2. Recommend the Doctor's Office to family and friends (3-point scale; "No" to "Yes, definitely").
  .
• Various socio-demographics including overall health, age, gender, and education.

Slide 7

Access to Care Composite

• All 5 items reference a 12-month period.
• 4-point scale (1 = Never, 2 = Sometimes, 3 = Usually, 4 =Always).

Slide 8

Access to Care Composite

• In the last 12 months:
  • How often did you get an appointment for care you needed right away as soon as you needed?.
  • How often did you get an appointment for routine care as soon as you needed?.
  • When you phoned this doctor's office during regular office hours, how often did you get an answer to your medical question that same day?.
  • When you phoned this doctor's office after regular office hours, how often did you get an answer to your medical question as soon as you needed?.
  • How often did you see this doctor within 15 minutes of your appointment time?.

Slide 9

Doctor Communication Composite

• All six items reference the most recent office visit.
• 3-point scale (1 = Yes, definitely; 2 = Yes, somewhat; 3 = No).
• During your most recent visit, did this doctor:
  • Explain things in a way that was easy to understand?.
  • Listen carefully to you?.
  • Give you easy to understand information about your health questions or concerns?.
  • Seem to know the important information about your medical history?.
  • Show respect for what you had to say?.

Slide 10

Courteous/Helpful Staff Composite

• Both items reference the office visit.
• 3-point scale (1 = Yes, definitely; 2 = Yes, somewhat; 3 = No).
• During your most recent visit, were clerks and receptionists at this doctor's office:
  • As helpful as you thought they should be?.
  • Treat you with courtesy and respect?.

Slide 11

CG-CAHPS Analysis Dataset

• Adult Visit Survey Data from the CG-CAHPS Comparative Database:
  • 103,442 responses.
  • 469 practice sites.
• There were 5 screener questions that determined if the composite item was to be answered by respondents:
  • 4 screener questions for Access to Care.
  • 1 screen question Doctor Communication.
• Only non-missing data for the composite items were included in the analysis.

Slide 12

CG-CAHPS Analysis Dataset

• 93% of respondents did not phone their doctor after regular hours:
  • Because of the high percentage of skips, "How often did you get an appointment for care you needed right away as soon as you needed?" was dropped from analysis.
  • Remaining 4 Access to Care composite items had responses between 46% to 98% of respondents.
• Final CG-CAHPS Analysis Dataset:
  • 450 practice sites.
  • 21,318 responses.

Slide 13

Characteristics of Analysis Dataset

• All respondents received paper/mail surveys.
• 89% of sites were Family Practice or Internal Medicine.
• 69% of sites were owned by a hospital or integrated delivery system.
• 67% of respondents were female.
• 81% were 45 years or older.

Slide 14

Psychometric Analyses and Criterion

• Individual and Multilevel Confirmatory Factor Analysis (CFA):
  • Factor loadings above 0.40.
  • Acceptable model fit indices:
    • CFI >0.95.
    • RMSEA <0.06.
    • SRMR <0.08 (Individual, Between- and Within-).
• Individual Internal Consistency Reliability Analysis:
  • Cronbach's a ≥0.70.

Slide 15

Psychometric Analyses and Criterion

• Practice Site Reliability Analysis:
  • Reliability ≥0.70.
• Examined reliability by practice size categories:
  • 1 clinician.
  • 2 to 3 clinicians.
  • 4-9 clinicians.
  • 10-13 clinicians.
  • 14-19 clinicians.
  • 20 or more clinicians.
• Correlation analysis among the composites and global ratings:
  • Individual and practice site levels.

Slide 16

Individual Level CFA Results

• All items within composites had individual factor loadings above 0.40 with average loadings of:
  • Access to Care: 0.68.
  • Doctor Communication: 0.76.
  • Courteous/Helpful Staff: 0.86.
• All model fit indices met criteria:
  • CFI: 0.97.
  • RMSEA: 0.05.
  • SRMR: 0.04.

Slide 17

Multilevel CFA Results

• All items at practice site level had factor loadings above 0.40:
  • Between site factor loadings range: 0.59 to 0.99.
  • Within site factor loadings rage: 0.45 to 0.99.
• All model fit indices met criteria except between-practice site SRMR:
  • CFI: 0.97.
  • RMSEA: 0.03.
  • Between SRMR: 0.10.
  • Within SRMR: 0.05.

Slide 18

Internal Consistency Reliability

• All items at individual level had Cronbach's alpha above 0.70:
  • Access to Care a = 0.77.
  • Doctor Communication a = 0.89.
  • Courteous/Helpful Staff a = 0.85.

Slide 19

Practice Site Reliability

• Practice site reliability estimate were acceptable for all site with at least 4 clinicians:
  • For a site with 1 clinician:
    • Only Access to Care had reliability above 0.70.
  • For a site with 2-3 clinicians:
    • Access to Care and Courteous/Helpful Staff had reliability above 0.70.
• The average number of respondents for 1 clinician and 2-3 clinicians was less than 100:
  • Smaller sites need more respondents per practice to increase reliability to acceptable levels.

Slide 20

Individual Correlations

• Strongest correlation was between Doctor Communication and Overall Global Rating Items.
• All correlations were significant (p <0.05).

Slide 21

Practice Site Level Correlations

• Strongest correlation was between Doctor Communication and Recommend Doctor’s Office.
• All correlations were significant (p <0.05).

Slide 22

Summary

• Overall, both the individual level and multilevel confirmatory factor analysis results provided support for the survey’s three composites.
• The CG-CAHPS composites have acceptable individual-level internal consistency reliability.
• The reliability remains acceptable across sites with four to twenty or more clinicians.

Slide 23

Email: naomidyer@westat.com

Image: A yellow note is shown with the words "Thank you!" written on it.