Survey of Patient Safety Culture in U.S. Hospitals: External Validity Analyses

Slide presentation from the AHRQ 2008 conference showcasing Agency research and projects.

Slide Presentation from the AHRQ 2008 Annual Conference

On September 10, 2008, Russ Mardon, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (215 KB).

Slide 1

Survey of Patient Safety Culture in U.S. Hospitals: External Validity Analyses

Russ Mardon, Ph.D.
Westat
2008 AHRQ Annual Conference
Westat
1650 Research Blvd.
Rockville, MD 20850
russmardon@westat.com
301-294-2037

Slide 2

Acknowledgements

Westat:
- Joann Sorra, Ph.D.
- Kabir Khanna, M.A.
- Naomi Dyer, Ph.D.
- Theresa Famolaro, M.P.S.
- AHRQ:
- Jim Battles, Ph.D.
- Mary Beth Farquhar, R.N., M.S.N.
- Claudia Steiner, M.D.
- Advisors:
  - Ron Hays, Ph.D.
  - Patricia Merryweather
Slide 3

HSOPS External Validity
- Hospitals worldwide are administering the AHRQ hospital patient safety culture survey (HSOPS)
- How is patient safety culture related to:
  - Patient harm?
  - Good clinical processes?
  - Patient satisfaction with care?
Slide 4

HSOPS External Validity
- AHRQ funded external validity analyses to explore correlations between HSOPS and:
  1. AHRQ Patient Safety Indicators (PSIs).
  2. Hospital Quality Alliance (HQA) Core Measures.
  3. Consumer Assessment of Healthcare Providers & Systems (CAHPS®) Hospital Survey.
Slide 5

Limitations
- Small and self-selected group of hospitals.
- Possible unmeasured confounding variables.
- Time periods for data sources did not exactly coincide.
- Limitations of administrative data sources.
Slide 6

Analysis 1:
- HSOPS & AHRQ Patient Safety Indicators (PSIs).
Slide 7

AHRQ Patient Safety Indicators (PSIs)
- Based on inpatient discharge data.
- Measures rates of potential complications or adverse events following:
- Surgery.
- Procedures.
- Adjusted for case-mix differences.
- Hospital-level rates of adverse events per 1,000 patients.
- http://www.qualityindicators.ahrq.gov/psi_overview.htm
Slide 8

PSIs Selected for Analysis
- We examined 11 PSIs plus overall PSI composite.
  - Decubitus ulcer (PSI 3).
  - Iatrogenic pneumothorax (PSI 6).
  - Selected infections due to medical care (PSI 7).
  - Postoperative:
    - Hip fracture (PSI 8).
    - Hemorrhage or hematoma (PSI 9).
    - Physiologic & metabolic derangements (PSI 10).
    - Respiratory failure (PSI 11).
    - Pulmonary embolism or deep vein thrombosis (PSI 12).
    - Sepsis (PSI 13).
    - Wound dehiscence in abdominopelvic surgical patients (PSI 14).
  - Accidental puncture and laceration (PSI 15).
Slide 9

Analysis Methods
- Examined partial correlations between HSOPS composite scores & AHRQ PSIs.
- Controlled for bed size, teaching status & government ownership.
- Data from 179 hospitals, matched using AHA ID.
- HSOPS data from 2005 & 2006.
- 12 composites, grade, # events, overall composite.
- 12 PSIs (adjusted rates) computed from 2005 Healthcare Cost & Utilization Project (HCUP) data for hospitals in 28 States.
- A total of 180 partial correlations (15 x 12).
Slide 10

Results: HSOPS & AHRQ PSIs
- Out of 180 possible correlations, 33 or 18% statistically significant (p<.05).
- "r"s ranged from -.17 to -.29.
- Negative relationships indicated hospitals with better patient safety cultures had lower rates of adverse events.
Slide 11

Results: HSOPS & AHRQ PSIs
- HSOPS composites with more significant correlations:
- Handoffs & Transitions (6 PSIs).
- Teamwork Across Units (4 PSIs).
- PSIs with more significant correlations:
  - Physiologic & metabolic derangements (PSI 10).
  - Pulmonary embolism or deep vein thrombosis (PSI 12).
  - Sepsis (PSI 13).
  - Wound dehiscence in abdominopelvic surgical patients (PSI 14).
Slide 12

Summary: HSOPS & AHRQ PSIs
- Most relationships were negative (i.e. in the right direction), supporting external validity of HSOPS.
- Findings point to safety culture areas to focus on for quality improvement.
- However, only 18% of possible relationships were statistically significant.
Slide 13

Analysis 2:
- HSOPS &
  Hospital Quality Alliance (HQA)
  Core Measures.
Slide 14

Hospital Quality Alliance (HQA) Core Measures—
- Hospitals submit data on hospital care processes to Centers for Medicare and Medicaid Services (CMS).
- Many HQA measures are same/similar to ORYX measures reported to the Joint Commission.
- 24 measures, we examined 20:
  - Heart attack (acute myocardial infarction, AMI)—6.
  - Heart failure (HF)—4.
  - Pneumonia (PN)—7.
  - Surgical care/infection prevention (SCIP/INF)—3.
Slide 15

Hospital Quality Alliance (HQA) Core Measures
- Rates:
  - # eligible patients who received recommended care
    divided by
    total # patients eligible to receive the care.
- Only included rates where total # eligible patients ≥30 (excluded 1 rate).
- Only included rates with a full year of data (excluded 3 rates).
- http://www.hospitalqualityalliance.org/hospitalqualityalliance/qualitymeasures/qualitymeasures.html
Slide 16

Analysis Methods
- Examined partial correlations between HSOPS composite scores & HQA core measures.
- Controlled for bed size, teaching status & government ownership.
- Data from 182 hospitals matched using American Hospital Association (AHA) ID.
- HSOPS data from 2005 & 2006.
- 12 composites, grade, # events, overall composite.
- 20 HQA core measures from 2006 & 2007.
- A total of 300 partial correlations (15 x 20).
Slide 17

Results: HSOPS & HQA Core Measures
- Out of 300 correlations, 28 or 9% were positive & statistically significant.
- "r"s ranged from .16 to .35, average = .24
- Positive relationships indicate hospitals with better patient safety cultures had higher rates of providing recommended care to patients.
- One core measure—smoking cessation counseling (AMI-4)—had 7 significant negative relationships with HSOPS.
Slide 18

Results: HSOPS & HQA Core Measures
- HSOPS composites with more significant correlations:
- Teamwork Across Units (6 measures).
- Handoffs & Transitions (5 measures).
- Core measures with more significant positive correlations:
- PN-7 Flu vaccination related to 5 HSOPS composites.
- HF-1 Discharge instruction related to 3 HSOPS composites.
- SCIP/INF-2 Antibiotic selection related to 3 HSOPS composites.
Slide 19

Summary: HSOPS & HQA Core Measures
- Most relationships positive, in the right direction.
- Findings point to same safety culture areas to focus on as AHRQ PSIs.
- However, only 9% of possible relationships were statistically significant.
Slide 20

Analysis 3:
- HSOPS &
  Consumer Assessment of Healthcare Providers and Systems (CAHPS®)
  Hospital Survey.
- https://cahps.ahrq.gov
Slide 21

Hospital CAHPS®
- Patient satisfaction survey asking about hospital care.
- 9 Hospital CAHPS® composites.
  1. Communication with nurses.
  2. Communication with doctors.
  3. Communication about medicines.
  4. Responsiveness of hospital staff.
  5. Discharge information.
  6. Pain management.
  7. Hospital environment (clean & quiet).
  8. Overall rating of hospital (0 worst to 10 best).
  9. Willingness to recommend to family & friends.
Slide 22

Analysis Methods
- Examined partial correlations between HSOPS composite scores & Hospital CAHPS.
- Controlled for bed size, teaching status & government ownership.
- Data from 75 hospitals.
- HSOPS data from 2005 & 2006.
- 12 composites, grade, # events, overall composite.
- 9 Hospital CAHPS composites from patients discharged in 2005 & 2006.
- A total of 135 partial correlations (15 x 9).
Slide 23

Results: HSOPS & Hospital CAHPS
- Out of 135 correlations, 35 or 26% were statistically significant.
- All significant correlations were positive, indicating that hospitals with better patient safety cultures had patients who rated the hospital higher on quality of care.
  - Positive "r"s ranged from .24 to .46, average = .31
Slide 24

Results: HSOPS & Hospital CAHPS
- HSOPS composites with more significant correlations:
  - Teamwork within units (5 HCAHPS® measures).
  - Organizational learning, Staffing, Patient safety grade (4 HCAHPS® measures).
- HCAHPS® composites with more significant correlations:
  - Hospital environment (9 HSOPS composites).
  - Communication with nurses (8 HSOPS composites).
  - Responsiveness of staff (6 HSOPS composites).
Slide 25

Summary: HSOPS & Hospital CAHPS
- Most relationships positive, in the right direction.
- Findings point to different safety culture areas to focus on compared to first 2 analyses.
- However, only 26% of possible relationships were statistically significant and some of these were negative.
Slide 26

Take-Away Messages
- Most relationships in all three analyses were in the right directions, lending support to HSOPS external validity.
- Associations with PSIs and HQA measures were strongest for HSOPS Handoffs and Teamwork across units measures.
- Associations with HCAHPS® measures were strongest for Teamwork within units, staffing, and safety grade measures.
- Data sets are cross-sectional and limited, therefore they cannot support causal inference.
Current as of January 2009

Internet Citation:

Survey of Patient Safety Culture in U.S. Hospitals: External Validity Analyses. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). January 2009. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/about/annualmtg08/091008slides/Mardon.htm

Current as of February 2009

Internet Citation: Survey of Patient Safety Culture in U.S. Hospitals: External Validity Analyses . February 2009. Agency for Healthcare Research and Quality, Rockville, MD. https://archive.ahrq.gov/news/events/conference/2008/Mardon.html

Survey of Patient Safety Culture in U.S. Hospitals: External Validity Analyses

Slide Presentation from the AHRQ 2008 Annual Conference

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

Slide 22

Slide 23

Slide 24

Slide 25

Slide 26