Development and Use of Ambulatory Adverse Event Trigger Tools (Text Version)

---

On September 14, 2009, Amy K. Rosen, PhD made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (776 KB).

---

Slide 1

 

Development and Use of Ambulatory Adverse Event Trigger Tools

Amy K. Rosen, PhD
AHRQ Conference
Sept. 14, 2009

Boston University School of Public Health, Boston, MA,
VA Center for Healthcare Quality, Outcomes, and Economic Research (CHQOER), Bedford MA
akrosen@bu.edu

 

Slide 2

 

Acknowledgements

Sponsored by AHRQ Contract No. HHSA290200600012, Task Order Officer Amy Helwig, MD

• PI Amy Rosen, PhD
• Co-PI Jonathan Nebeker, MD, MS
• Co-Investigators:
  • Stephan Gaehde, MD
  • Haytham Kaafarani, MD, MPH
  • Brenna Long, MA
  • Hillary Mull, MPP
  • Brian Nordberg, BS
  • Steve Pickard, MS
  • Peter Rivard, PhD
  • Lucy Savitz, PhD, MBA
  • Chris Shanahan, MD, MPH
  • Stephanie Shimada, PhD

 

Slide 3

 

Project Goal and Settings

• Goal: Develop adverse event (AEs) triggers for the outpatient setting
  • Outpatient surgery
  • Outpatient adverse drug events (ADEs)
• Three sites for patient data:
  • Boston Medical Center (BMC)
  • Intermountain Healthcare
  • Veterans Health Administration (VA)

 

Slide 4

 

Background

• Triggers are algorithms that use electronic patient data to identify patterns consistent with a possible adverse event
  • e.g. , the combination of a lab value threshold and an active prescription
• Global vs. AE specific trigger:
  • Flags the chart for the suspicion of occurrence of any AE or the occurrence of a specific AE
• Interventionist triggers:
  • Mostly ADEs
  • Gives providers a chance to respond and avoid alert overload

 

Slide 5

 

Methods

Literature Review
Clinical
Input
Focus Groups
Clinical Advisory Panel
Modified
Delphi Panel
Final List of Triggers
Document existing triggers
Establish prevalence of outpatient AEs
Establish primary causes of outpatient AEs
 

 

Slide 6

 

Methods

Literature Review
Clinical
Input
Focus Groups
Clinical Advisory Panel
Modified
Delphi Panel
Final List of Triggers
Document existing triggers
Establish prevalence of outpatient AEs
Establish primary causes of outpatient AEs
Review epidemiological basis for AEs
Input clinical knowledge and data needed into trigger rules
 

 

Slide 7

 

Methods

Literature Review
Clinical
Input
Focus Groups
Clinical Advisory Panel
Modified
Delphi Panel
Final List of Triggers
Document existing triggers
Establish prevalence of outpatient AEs
Establish primary causes of outpatient AEs
Research data limitations
Determine priority areas for trigger development
Develop methods to critique triggers
Review epidemiological basis for AEs
Input clinical knowledge and data needed into trigger rules
 

 

Slide 8

 

Methods

Literature Review
Clinical
Input
Focus Groups
Clinical Advisory Panel
Modified
Delphi Panel
Final List of Triggers
Document existing triggers
Establish prevalence of outpatient AEs
Establish primary causes of outpatient AEs
Research data limitations
Determine priority areas for trigger development
Develop methods to critique triggers
Review epidemiological basis for AEs
Input clinical knowledge and data needed into trigger rules
Refine rules/trigger logic
Refine priority areas priority areas for trigger development
 

 

Slide 9

 

Methods

Literature Review
Clinical
Input
Focus Groups
Clinical Advisory Panel
Modified
Delphi Panel
Final List of Triggers
Document existing triggers
Establish prevalence of outpatient AEs
Establish primary causes of outpatient AEs
Research data limitations
Determine priority areas for trigger development
Develop methods to critique triggers
Review epidemiological basis for AEs
Input clinical knowledge and data needed into trigger rules
Refine rules/trigger logic
Refine priority areas priority areas for trigger development
Rate priority of AE causes
Rate priority of AEs
Rate triggers based on system- and patient-level perspectives
 

 

Slide 10

 

Methods

Obtained de-indentified clinical data from each site.

Boston Medical Ctr
Intermountain
VA (VISN 19)

Combined the data fields from each site into a SQL database.

Trigger Database
Mock EMR

Created a mock electronic medical record (EMR) interface to enable case classification.

 

Slide 11

 

Global Trigger Tools—Outpatient Surgery

Trigger Name	Description
Emergency Department (ED)	ED visits within 21 days of an outpatient surgery are likely to include visits due to complications from surgery
Re-Admit	Hospital admissions within 30 days of an outpatient surgery may be due to complications from surgery
Procedure (Proc)	Reoperation within 30 days and certain procedures on the same day as a scheduled same-day surgery are indicative that a complication may have occurred
Length of stay (LOS)	There is a higher probability of complications in same-day surgeries that result in a longer-than-planned stay in the hospital

Slide 12

 

AE-Specific Trigger Tools—Outpatient Surgery

Trigger Name	Description
Pulmonary Embolism/Deep Vein Thrombosis (PE/DVT)	Detect cases of PE/DVT that result within 30 days of an outpatient surgery.
Surgical Site Infection (SSI) 1	Detect cases of surgical site infection that occur as a complication of outpatient surgery by looking for cases requiring wound care.
Hematocrit (Hema)	Detect the occurrence of postoperative hematoma/hemorrhage by looking for evidence of postoperative blood loss in same-day surgeries (otherwise expected to have minimal blood loss).

Slide 13

 

Surgery Trigger Logic: Procedure

• Fire if:
  • Same-day surgery
  • AND
  • procedure (interventional radiological OR urological OR cardiac OR gastroenterological)
  • OR re-operation ≤ 30 days

Slide 14

 

AE-Specific Trigger Tools—ADE

Trigger Name	Description
Warfarin (Warf)	Detect rapid or excessive anticoagulation to prevent bleed by looking for over- anticoagulation and no evidence of rechecking within reasonable window.
Potassium reducer (K-low)	Detect hypokalemia to prevent further decline and arrhythmia by looking for dropping potassium without evidence of adjustments to medication.
Potassium raiser (K-high)	Detect hyperkalemia to prevent further increase and arrhythmia by looking for rising potassium without evidence of adjustments to medication.

Slide 15

 

AE-Specific Trigger Tools—ADE (cont'd)

Trigger Name	Description
Change in renal clearance (Creat)	Detect decreased renal function to prevent reactions from other drugs that are renally cleared by looking for a decrease in creatinine clearance to a concerning level that occurs soon after starting a drug that might decrease creatinine, after confirming that the new drug has not been decreased.
Bone Marrow Toxin (BMT)	Detect early signs of myelosuppression to prevent more severe cases by looking for a decrease in cells after non-cancer drug without evidence that the drug has been decreased in response.
Hypnotic	Detect impairment in consciousness and cognition to improve quality of life by looking for patients on psychotropic drugs with a subsequent decline in consciousness or cognition.

Slide 16

 

ADE Trigger Logic: Change in Renal Clearance

• Fire if:
  • Subsequent increase in creatinine > 33% and dose > than dose prior to creatinine measurement (This is the reference creatinine level) AND
  • NOT (trimethoprim started in interval between 1 day prior to creatinine measurement and after reference creatinine level) AND
  • NOT (all GFR reducers and renal toxins discontinued or expired > 3 months prior to triggering value)
• Remove trigger if response taken within window:
  • Renal toxin discontinued or GFR reducer dose reduced 0-6 days after firing criteria satisfied OR
  • Creatinine resulted 0-6 days after firing criteria satisfied

Slide 17

 

Data Challenges—Accessing Data

• Political/Logistical Barriers
  • Gaining permission to access the data
    • Developed de-identification algorithm
    • Challenge meeting HIPAA compliance
    • Administrative barriers to obtaining access
  • Encrypting/ ensuring safe transfer of data between sites
  • Safe storage of data from multiple institutions
• IT Resources
  • Availability of personnel for data pulls
  • Computing infrastructures
    • Pulling notes too resource intensive

Slide 18

 

Data Challenges—Data Elements

• "IT Black Box"
  • Researchers reliant on IT staff's programming, no way to ascertain completeness of data
• Inconsistencies in coding across institutions
  • Same information, different coding:
    • Gender: M/F vs 1/2/3
    • Units of measure: metric vs US vs missing
    • ICD-9-CM codes stored with or without periods
    • ICD-9-CM procedure codes were unavailable for some procedures
    • Lab titles inconsistent across settings
  • Lack of documentation re: coding practices
  • Numeric results within text data

Slide 19

 

Data Challenges—Data Elements (cont'd)

• Missing data
  • Loss of information from text de-identification algorithm
    • Fuzzy pattern and word matching removed some key clinical terms from clinical notes
    • De-identification made notes difficult to read
    • Removal of dates resulted in loss of information about clinical order
  • Missing National Drug Codes (NDCs) in pharmacy data
  • Free text vs. standardized daily dosage information
    • TAKE ONE-HALF TABLET BY MOUTH EVERY DAY FOR 2 WEEKS, THEN TAKE � ONE-HALF TABLET TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE ONE TABLET � TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE TWO TABLETS TWO (2) TIMES A � DAY FOR 2 WEEKS, THEN TAKE THREE TABLETS TWO (2) TIMES A DAY FOR 2 � WEEKS, THEN TAKE FOUR TABLETS TWO (2) TIMES A DAY INCREASE DOSE � GRADUALLY. � WHEN GOING FROM 25 TO 50 MG START WITH INCREASING THE AM � DOSE FOR 2WEEKS, THEN THE AM AND PM DOSE. � DO THIS WHEN INCREASING � FROM 50 TO 75 AND 75 TO 100. � IF QUESTIONS PLEASE CALL.
  • Lack of units in lab data

Slide 20

 

Next Steps

• Case classification
  • RNs classifying surgery AE trigger-flagged cases
  • Pharmacists classifying ADE trigger-flagged cases
• Calculate positive predictive value (PPV) for each trigger
• Conduct a second round of focus groups at each institution
• Hold phone call with trigger experts to review logic and discuss results

Slide 21

 

Dissemination to Date

• Triggers and Targeted Injury Detection Systems (TIDS)Expert Panel Meeting , Rockville, MD. June 2008. See proceedings at https://www.ahrq.gov/QUAL/triggers/
• Mull HJ & Nebeker, JR. Informatics Tools for the Development of Triggers for Outpatient Adverse Drug Events. AMIA Annual Symposium Proceedings. Nov 2008, 6:505-9.
• Kaafarani H, Rosen AK, et al. What is a Trigger Tool to a Surgeon: Designing Trigger Tools for Surveillance of Adverse Events in Ambulatory Surgery. Massachusetts Chapter of the American College of Surgeons 55th Annual Meeting, Boston, MA. Dec 2008.
• Kaafarani H, Rosen AK, et al. Development of Trigger Tools for Surveillance of Adverse Events in Ambulatory Surgery. VA HSR&D QUERI National Meeting, Phoenix, AZ. Dec 2008.
• Kaafarani H, Rosen AK, et al. Developing Trigger Tools for Surveillance of Adverse Events in Same-Day Surgery: A Literature-Based, End-User Inspired & Expert-Evaluated Methodology. VA HSR&D Annual Meeting, Baltimore, MD. Feb 2009.
• Shimada S, Rivard P, et al. Priorities & Preferences of Potential Ambulatory Trigger Tool Users. AcademyHealth Annual Research Meeting, Chicago, IL. June 2009.
• Kaafarani H, Rosen AK, et al. Developing Trigger Tools for Surveillance of Adverse Events in Same-Day Surgery: A Literature-Based, End-User Inspired and Expert-Evaluated Methodology. AHRQ Annual Meeting, Bethesda, MD. Sept 2009.
• Kaafarani H, Rosen AK, et al. Development of Trigger Tools for Surveillance of Adverse Events in Ambulatory Surgery. Quality and Safety in Health Care. (forthcoming)