Nordic Experience With Safety Culture Survey (Text Version)

Slide Presentation from the AHRQ 2008 Annual Conference

On September 10, 2008, John Ovretveit, Ph.D., made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (3.7 MB).

Slide 1

Nordic Experience With Safety Culture Survey

John Øvretveit
Director of Research, Professor, Karolinska Medical Management Centre Sweden and Professor of Health Management, Faculty of Medicine, Bergen University

Resources download from: http://homepage.mac.com/johnovr/FileSharing2.html

Slide 2

Map of Europe

Legend:

• Europe in Green.
• Scandinavia is Sweden, Norway, and Denmark.
• Sweden in blue.
• Finland in white.
• Norway in orange.
• Denmark in red.

Slide 3

Map of Scandinavia

Countries included are:

• Iceland.
• Norway.
• Finland.
• Sweden.
• Denmark.

On bottom of screen are images of Image Nordiska Hälsovårdshögskolan and 2008 Nordiska Ministerrådet.

Slide 4

Karolinska Institute Medical University—Largest in NE

This slide shows the home page of the Karolinska Institute Medical University Web site.

Slide 5

Report

Draws on roles as coordinator for:

• Nordic patient safety research network &
• International quality improvement research network.

• Sweden: five hospitals approximately 2,000 people surveyed.
• Norway: Stavanger & Akershus studies.
• Denmark: Copenhagen hospitals study.
• Finland: Considering AHRQ versus Institute for Healthcare Improvement (IHI) versions.

Slide 6

Why We Wanted a Survey Tool

• Diagnosis to show possible intervention points and problem areas.
• Compare units/organisations to discover and learn from "best cultures/practice."
• Assess impact of changes on before/after responses to questions.

Slide 7

Sweden: Study Requirements

1. Describe the different "pictures" which different surveys provide—differences and strengths and weaknesses of each.
2. Describe what the surveys really measure, by comparing aggregate data from units which should be the same to find out if same data is produced, by comparing different instruments and asking subjects views about validity and in other ways.
3. Assess reliability by comparing matched subjects responses and test-retest reliability.
4. Assess sensitivity to registering different changes.
5. Describe how data from these related to real changes and activities (comparing survey data to observational and interview data).

Slide 8

Criteria for assessing surveys include:

Practical:

• Value assessed by managers for action and making more informed decisions.
• Use for identifying best practice units.
• Time and cost to administer and analyse.

Scientific:

• Value for research use to studying change and influences on performance.
• Validity (global).
• Sensitivity.

Slide 9

How to Measure?

• Typological approaches:
  • Competing Values Framework.
  • Harrison's Organizational Ideology Questionnaire.
  • Quality Improvement Implementation Survey.
• Dimensional Instruments:
  • Organizational Culture Inventory.
  • Hospital Culture Questionnaire.
  • Nursing Unit Culture Assessment Tool.
  • Practice Culture Questionnaire.
  • MacKenzie's Culture Questionnaire.
  • Survey of Organizational Culture.
  • Corporate Culture Questionnaire.
  • Core Employee Opinion Questionnaire.
  • Hofstede's Organizational Culture Questionnaire.
  • Organizational Culture Survey.
  • (Scott et al. (2003) HSR).

Slide 10

Publicly Available Safety Culture Survey Instruments

• California 15 hospitals study (Singer et al. 2003 (Stanford/Patient Safety Center of Inquiry [PSCI] survey 2001).
• AHRQ 2004 survey.
• IHI survey.
• Weingart et al. 2004 employee survey instrument.
• Manchester patient safety tool (MaPSaT) 2005.

Slide 11

Metaphors for Culture—The Iceberg

Slide has image of iceberg with blue waterline to delineate the ice above and ice below the water.

• Pointing to iceberg above water: Tip: What people say in response to questions.
• Pointing to iceberg at waterline: Observable behaviors, practices, and discourse: This is the way we are doing things around here.
• Pointing to the iceberg below the waterline: Below: underlying beliefs, attitudes, values, philosophies, and taken-for-granted, aspects of workplace life: why we do the things we do around here.

Slide 12

Previous Research—John's Assessment

• Most are attitude surveys:
  • Tell us about climate not culture.
• Often without theoretical basis.
• Rarely build on previous instruments.
• More research combining ethnographic and survey pictures needed.
• Unclear for which purpose each instrument most useful and perspective it gives.

Slide 13

What Research Can Contribute

To safer patient care:

• Is there a link between culture and safety performance?
• Can you predict safety performance using culture assessment?
• Theory explaining any causal pathways.
• What changes which aspects of culture in which situations?

Slide 14

UK-Sweden Senior Leaders' Views

Slide includes graphic of bar chart with title "Here is a list of some possible steps that could be used to improve patient safety. Please select the five you think would be most beneficial."

There is a scale from 0-100 showing the percentage of positive responses.

• An organizational culture that encourages reporting and avoids blame:
  • UK—83
  • Sweden—95
• More emphasis on infection control, including hand washing:
  • UK—73
  • Sweden—47
• Better communication between staff and patients:
  • UK—64
  • Sweden—65
• Better training of health professionals:
  • UK—56
  • Sweden—54
• Standardization of medical equipment and devices:
  • UK—49
  • Sweden—45
• Using computerized patient records:
  • UK—34
  • Sweden—39
• Making greater use of information technology (IT) in dispensing and administering medication:
  • UK—31
  • Sweden—35
• Including a pharmacist on hospital rounds:
  • UK—29
  • Sweden—5
• Increasing the number of hospital nurses:
  • UK—22
  • Sweden—9
• Stronger leadership from board members:
  • UK—20
  • Sweden—23
• Confidential reporting of patient safety incidents to an independent agency:
  • UK—14
  • Sweden—23
• Reducing junior doctors' hours to avoid fatigue:
  • UK—13
  • Sweden—12
• Relaxation of penalties for staff who make mistakes:
  • UK—4
  • Sweden—15
• Stronger penalties for staff who make mistakes:
  • UK—4
  • Sweden—1
• None of these:
  • UK—0
  • Sweden—1

Slide 15

Swedish Experience So Far

Pilot:

• One hospital 100 questionnaires in 10 departments (10 staff at each unit) (acute care, primary healthcare, psychiatrics, and geriatrics).
• Interviews with 20.
• Longer than IHI, but easier to fill in.

Main study: AHRQ approximately 2,000 administered—currently analysing:

• Large variations.
• Similarities and differences to AHRQ averages possibly significant.
• Preliminary findings—next slides.
• Contact Jessica.lindberg@karolinska.se.

Slide 16

In Swedish

Shows tables and bar charts with results.

Slide 17

Biggest Difference to USA

Senior Management

Survey Questions (translated from Swedish):

• 1. Hospital management provides a work climate that promotes patient safety.
• 8. The actions of hospital management show that patient safety is a top priority.
• 9. Hospital management seems interested in patient safety only after an adverse event happens.

Average: 38
Median: 36
Highest: 88
Lowest: 16
AHRQ average: 70

Slide 18

Reporting AHRQ Section D

Survey Questions:

• 1. When a mistake is made but is caught and corrected before affecting the patient, how often is this reported?
• 2. When a mistake is made but has no potential harm to the patient, how often is this reported?
• 3. When a mistake is made that could harm the patient but does not, how often is this reported?

To the right of the text is a bar chart with the results translated:

Average: 45
Median: 43
Highest: 83
Lowest: 23
AHRQ average: 60

Slide 19

Learning Organisation

(Text in Swedish).

Average: 54
Median: 55
Highest: 88
Lowest: 20
AHRQ average: 70

Slide 20

Biggest Variation Between Sweden Hospitals

Bar graph showing:"General view of how safe patients are in your service area".

Average: 61
Median: 66
Highest: 100
Lowest: 18
AHRQ average: 72

Slide 21

Overall safety situation (e.g.) We have patient safety problems in this unit

Average: 65
Median: 66
Highest: 96
Lowest: 33
AHRQ average: 64

Slide 22

My Supervisor

Survey Questions (Translated from Swedish):

• 1. My supervisor/manager says a good word when he/she sees a good job done according to established patient safety procedures (scale 1-5)
• 2. My supervisor/manager seriously considers staff suggestions for improving patient safety (scale 1-5)
• 3. Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts (scale 1-5)
• 4. My supervisor/manager overlooks patient safety problems that happen over and over (scale 1-5)

Average: 62
Median: 64
Highest: 91
Lowest: 38
AHRQ average: 75

Slide 23

Cooperation

Average: 76
Median: 79
Highest: 97
Lowest: 17
AHRQ average: 79

Slide 24

Norway: Stavanger Studies

• 400 bed hospital 10 departments.
• 1919 questionnaires returned (55% rr).
• Investigate fit of proposed factor structure using Confirmatory factor analysis (CFA).
• Internal consistency of dimensions: Cronbach's alpha.
• Discriminant validity: intercorrelation among concepts and multivariate analysis of variance (MANOVA.

Slide 25

Concurrent Validity: Degree to Which Dimensions Influenced Outcome Variables Included in Hospital Survey on Patient Safety Culture (HSOPSC)

Slide shows Unit level and hospital level dimensions in a box with an arrow pointing toward the outcome measures.

Unit Level—7 dimensions

• Supervisor/manager expectations and actions promoting safety.
• Organizational learning—continuous improvement.
• Teamwork within units.
• Communication openness.
• Feedback & communication about error.
• Nonpunitive response to error.
• Staffing.

Hospital Level—3 dimensions

• Hospital management support for patient safety.
• Teamwork across hospital units.
• Hospital handoffs and transitions.

Outcome Measures (Single- and Multiple-Item Scales)

• Single-item outcome measures:
  • Number of events reported.
  • Overall patient safety grade.
• Multiple-item scales:
  • Overall perceptions of safety.
  • Frequency of events reported.

Slide 26

Stavanger Studies: Findings

• Factor analyses: factorial model fitted the data well.
• Poor internal consistency: "Organizational learning—continuous improvement," improved when "mistakes have led to positive changes here" removed.
• Surprise: safety culture dimension exerted several negative influences on "Number of events reported (last 12 months)," suggests this outcome variable is invalid.

Slide 27

Stavanger Studies: Findings (continued)

• Can be used in Norwegian hospital settings.
• Psychometric properties satisfactory.
• "Number of events reported" is not a good outcome measure.

Research challenges:

• Getting high/unbiased survey response rate.
• Using correct statistical methods to avoid Type I and Type II errors.

Reference: Olsen in: Øvretveit, J; Sousa, P (2008) Quality and Safety Improvement Research, Portugal School of Public Health, Publisher: Lisbon

Slide 28

Denmark: Copenhagen hospitals study (n=10k)

Slide depicts two charts, with bottom five and top five most negative and most positive answers to survey questions.

Bottom 5—Where the staff was most negative (fig 2)

Data is expressed in percentages.

• Units involved in the patient pathway coordinate well: 9756 respondents, 38.6 positive answers, 31.4 neutral answers, 29.9 negative answers.
• Cooperation grows stronger when many tasks must be done in a hurry: 10261 respondents, 38.3 positive answers, 30.3 neutral answers, 31.4 negative answers.
• We often work in a way where too much must be done too quickly: 10240 respondents, 36.9 positive answers, 17.9 neutral answers, 45.2 negative answers.
• Busyness is no threat to patient safety: 9744 respondents, 27.0 positive answers, 21.5 neutral answers, 51.5 negative answers.
• Information often gets lost when patients are transferred: 9538 respondents, 17.6 positive answers, 30.3 neutral answers, 52.1 negative answers.

Top 5—Where the staff was most positive (fig 1)

• We speak up when we observe something that may threat patient safety: 10288 respondents, 89.3 positive answers, 8.5 neutral answers, 2.2 negative answers.
• We help each other and cooperate: 10360 respondents, 88.1 positive answers, 7.5 neutral answers, 4.4 negative answers.
• Generally my leader has great confidence in his/her employees: 10385 respondents, 85.7 positive answers, 8.7 neutral answers, 5.6 negative answers.
• One can be quite easy about reporting adverse events: 10224 respondents, 84.4 positive answers, 12.5 neutral answers, 3.1 negative answers.
• The managers have expressed they want us to discuss adverse events: 10395 respondents, 81.9 positive answers, 9.0 neutral answers, 9.0 negative answers.

Slide 29

Denmark: Copenhagen hospitals study (continued)

Slide depicts bar graph, scale 0%-45%.

Differences between Professions (fig 3)

Question: Is it only by sheer chance no more serious accidents occur?

• Doctors: 17.9%
• Nurses: 23.5%
• Social and Health Assistants: 29.6%
• Physiotherapists and Occupational Therapists: 8.2%
• Laboratory Technologists: 7.9%
• Medical Secretaries: 18.0%
• Midwives: 40.5%
• Radiographer: 13.6%
• Others: 11.0%

Slide 30

Denmark: Copenhagen hospitals study (continued)

Slide depicts bar graph, scale 0%-100%.

Differences between departments (fig 4)

• Anesthesiology: 28 fall between 30-60%.
• Diagnostics: 46 fall between 10-57%.
• Gynecology/Obstetrics: 46 fall between 10-57%.
• Medicine: 82 fall between 8-90%.
• Surgery: 65 fall between 9 and 75%.
• Neurology: 38 fall between 33-70%.
• Orthopedic surgery: 49 fall between 36-85%.
• Psychiatry: 35 fall between 28-63%.
• Pediatric: 22 fall between 55-77%.

Slide 31

Conclusions: Copenhagen hospitals study

• The culture is open and non-sanctioning towards reporting adverse events. This is very positive and crucial to obtain a more mature safety culture.
• However, the staff encounters safety problems relating to the planning of work.
• The confidence and openness can be attributed to the Danish Patient Safety Act, which protects the staff against sanctions in spite of reporting an event.
• The trends of the Copenhagen Region Survey also seem to appear in the AHRQ's preliminary benchmarks on the hospital survey on patient safety culture.

Slide 32

Denmark: Copenhagen hospitals study (continued)

• Comparisons with similar regions in DK & comparison over time in a repeat survey in 2 years.
• Comparing with data from clinical databases and surveys of patient satisfaction.
• Results given to department managers -required to report their follow-up.
• Provided: tools and ideas how to react on problem areas.
• Results contribute to the Region's patient safety action plan & used in leadership development.

Acknowledgements to: Marlene Madsen Dansk mdyrloev@ruc.dk Institut for Medicinsk Simulation, Amtssygehuset i Herlev

Slide 33

Norway: Akershus Sexton (SAQ 2006) study

Slide 34

Norway: Akershus Sexton (SAQ 2006) study

Six patient safety dimensions:

• Teamwork Climate.
• Safety Climate.
• Stress Recognition.
• Working Conditions.
• Job Satisfaction.
• Perceptions of Management (Hospital Mgmt and Unit Mgmt).

Slide 35

"Safety Attitudes: Frontline Perspective from this Patient Care Area"

Screen shot of a form from The University of Texas at Austin, copyright 2004.

Slide 36

Percentage of Units Reporting "Good Patient Safety Climate"

This slide is a bar chart with a scale of 0 to 100. It shows the range of percentage of units reporting good patient safety climate from approximately 5 percent to approximately 87 percent.

Slide 37

Acceptability:

• Relatively high response rate.
• Relatively quickly answered (but not as quickly as promised).
• Not obviously threatening (but some units did decline to participate).
• Met with interest (by nurses, not by all doctors).

Discriminatory ability

• Good—Variation across professions, departments, wards.

Slide 38

Validity of Safety Attitudes Scores?

1. Reasonable confirmation of factor structure
   But can it be improved?
   • By reformulating translated questions?
   • By re-reversing the three reversed questions?
   • By emphasizing: your general evaluation (not just "this morning").
2. Employees recognized their own ward/department.
3. Quality department recognized "safer" wards/departments.
4. Department SAQ-averages correlated with number of errors discovered by "Trigger Tool"—revision of patient records.
5. Department SAQ-scores correlate with (same time) average patient response to questions on.
   1. Suspected treatment errors.
   2. Poor organization of hospital work.

Slide 39

Slide is in untranslated Swedish

Slide 40

Slide is in untranslated Swedish

Slide 41

Slide is in untranslated Swedish

Slide 42

Slide is in untranslated Swedish

Slide 43

Coordination

Average: 47
Median: 48
Highest: 67
Lowest: 27
AHRQ average: 57

Slide 44

Coordination (continued)

Average: 48
Median: 498
Highest: 72
Lowest: 22
AHRQ average: 48