Instrumental Variables for Comparative Effectiveness Research: A Review of applications

AHRQ 2008 Annual Conference

Slide Presentation from the AHRQ 2008 Annual Conference

On September 9, 2008, M. Alan Brookhart, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (153 KB).

Slide 1

Instrumental variables for comparative effectiveness research: a review of applications

M. Alan Brookhart, Ph.D.
Division of Pharmacoepidemiology,
Brigham & Women's Hospital, Harvard Medical School.

Slide 2

Overview of Lecture

• Brief introduction to instrumental variable analysis.
• Examples of instrumental variables, some characteristics.
• Role of IV in observational studies of medical interventions.

Slide 3

The Challenge of Observational Studies of Intended Effects

• Confounding by indication is strong.
• Patients who need treatment are more likely to receive treatment.
• Indications unmeasured or poorly measured.
  • Greater unmeasured confounding bias.

Slide 4

Instrumental Variables

• Can permit estimation of causal effects even when important confounders are unmeasured.
• Instrument should be correlated with treatment.
• Instrument should be related to outcome only through association with treatment (often termed the exclusion restriction).
  • Empirically unverifiable, but can be explored in observed data.

Slide 5

Confounding and Instrumental Variables (IV)

Example: Randomized Controlled Trial with Non-Compliance.

• Z: Instrument Treatment Arm Assignment.
• C: Confounders.
• Y: Outcome.
• X: Received Treatment.

The diagram presents a square-like formation composed of arrows with an "X" being formed by two arrows in the center of the square. The upper corners, represented by "Z" on the left and "C" on the right, have an arrow pointing both ways between them. A red circle with a slash through it sits atop this arrow with another arrow pointing to "Randomization." The bottom corners, represented by "X" on the left and "Y" on the right, have an arrow pointing from X to Y. An arrow points to both X and Y from Z and an arrow points to both Y and X from C. Another red circle with a slash through it sits atop of the arrow pointing to Y from Z with another arrow pointing to "Blinding."

Slide 6

Intention-to-treat (ITT) and (Wald) IV Estimator

ITT Estimator =   E[Y|Z=1] - E[Y|Z=0]

                            E[Y|Z=1] - E[Y|Z=0]
IV Estimator =   -------------------------
                            E[X|Z=1] - E[X|Z=0]

           Effect of the Instrument on the Outcome
=    ---------------------------------------------
          Effect of the Instrument on the Exposure

Slide 7

Interpretation of an IV

• When treatment effects are heterogeneous, IV estimator may be biased for average treatment effect (ATE).
• IV estimates a weighted average of causal treatment effects.
• Subgroups of patients whose treatment status is more likely to be influenced by the IV are weighted up.
• Empirical data, subject-matter knowledge may be used to anticipate direction of bias in IV relative to ATE.

Slide 8

IVs For Comparative Effectiveness

Slide 9

Preference-based Instrumental Variables

• Substantial variation in medical practice across regions, hospitals, physicians.
• Differences in medical practice may represent a natural experiment.
• Suggests IVs defined at level of provider.

Slide 10

Observational Study of Non-steroidal Anti-Inflammatory Drugs and GI bleeding risk in an elderly population (Brookhart et al, Epidemiology 2006)

• Compare short-term risk of Gastrointestinal (GI) outcomes between.
  • Non-selective Nonsteroidal Antiinflammatory Drugs (NSAIDs).
  • COX-2 selective NSAIDs.
• Coxibs are slightly less likely to cause GI problems.
• Coxibs are likely to be selectively prescribed to patients at increased GI risk.
• Classic problem of confounding by indication.

Slide 11

Characteristics of Cohort

The table presents the percentages for "Coxib" and "NS NSaid" for various "Variables."

• Female Gender: 86%; 81%
• Age greater than 75: 75%; 65%
• Charlson Score greater than 1: 76%; 71%
• History of Hospitalization: 31%; 26%
• History of Warfarin Use: 13%; 7%
• History of Peptic Ulcer Disease: 4%; 2%
• History of GI Bleeding: 2%; 1%
• Concomitant GI drug use: 5%; 4%
• History GI drug use: 27%; 20%
• History of Rheumatoid Arthritis: 5%; 3%
• History of Osteoarthritis: 49%; 33%

Slide 12

Patient's GI Risk: The slide shows three images of a sick man with "Low," "Moderate," and "High" over each one of them and "Marginal Patient" below them.

• COX-2 Preferring Physician: NS NSAID; COXIB; COXIB.
• NS NSAID Preferring Physician: NS NSAID; NS NSAID; COXIB.

Slide 13

Estimating Preference

• Volume of NSAID prescribing varies considerably among physicians.
• Our approach: use the type of the last NSAID prescription written by each physician as a measure of current preference.
• If for last patient, physician wrote a coxib prescription, for the current patient he is classified as a "coxib preferring physician" other he is classified as a "non-selective NSAID preferring physician."

Slide 14

The diagram shows an image of a doctor with two images of the sick man, from slide 12, forming a triangle. The man on the left represents "Previous Patient treated with NSAIDs," and the man on the right represents "Index Patient." One arrow points from the doctor to the previous patient showing "Treatment." Another arrow points from the doctor to the index patient showing "Treatment=?" An arrow points from the previous patient to the index patient showing "Index Patient's IV is Previous Patient's Treatment." Underneath the diagram is an arrow pointing to the right showing "Time" from previous patient to index patient.

Slide 15

Instrument should be related to treatment

The table presents the "Current Prescription-Actual Treatment" for the "Last NSAID Prescription-IV."

• Coxib Z=1
  • Coxib X=1: (73%)
  • Non-Selective NSAID X=0: (27%)
• Non-Selective NSAID Z=0
  • Coxib X=1: (50%)
  • Non-Selective NSAID X=0: (50%)

Slide 16

Instrument should be unrelated to observed patient risk factors

The table presents the percentages for "Coxib Pref Z=1" and "NS NSAID Pref Z=0" for various "Variables."

• Female Gender: 84%; 84%
• Age greater than 75: 73%; 72%
• Charlson Score greater than 1: 75%; 73%
• History of Hospitalization: 29%; 27%
• History of Warfarin Use: 12%; 10%
• History of Peptic Ulcer Disease: 3%; 3%
• History of GI Bleeding: 1%; 1%
• Concomitant GI drug use: 5%; 5%
• History GI drug use (e.g., PPIs): 25%; 24%
• History of Rheumatoid Arthritis: 4%; 4%
• History of Osteoarthritis: 45%; 41%

Slide 17

IV estimate of the effect of coxib exposure on GI outcome

IV Estimate
E[Y|Z=1]-E[Y|Z=0]               -0.21%
-------------------------   =   --------  =   -0.92%
E[X|Z=1]-E[X|Z=0]                  22.8%

Crude
E[Y|X=1]-E[Y|X=0]    =    +0.03%

After multivariable adjustment
      =   -0.04%

Slide 18

Other examples of preference-based instrument

• Clinic, hospital as IV
  • Johnston SC, J Clin Epi.
  • Schneeweiss, Seeger, Walker NEJM 2008: Aprotinin during CABG.
• Geographic region as instrument.
  • Wen, J & Kramer J Clin Epi 1997.
  • Brooks et al, HSR, Breast cancer treatment.
  • Stuckel T, et. al JAMA—Cardiac catheterization.
• Generally available, but vulnerable to case-mix bias, concomitant treatments associated with the IV

Slide 19

Distance to Specialty Provider as IV

McClellan, M., B. McNeil and J. Newhouse, JAMA 1994. "Does More Intensive Treatment of Acute Myocardial Infarction Reduce Mortality?"

• Medicare claims data identify admissions for AMI, 1987-91.
• Treatment: Cardiac catheterization (marker for aggressive care).
• Outcome: Survival to 1 day, 30 days, 90 days, etc.
• Instrument: Indicator of whether the hospital nearest to a patient's residence does catheterizations.

Slide 20

Are assumptions valid?

1. Is IV associated with treatment?
   • 26.2% get cath if nearest hospital does caths.
   • 19.5% get cath if nearest hospital does not do caths.
2. Is IV associated with outcome other than through it effect on treatment?
   • They demonstrated IV is largely unassociated with observed patient characteristics.

Slide 21

McClellan, et al. results McClellan, et al. results

1. Conventional methods.
   • Crude estimate -30% (17% 1-year mortality if catheterized vs. 47%).
   • OLS estimate is -24%, adjusting for observable risk factors.
2. IV estimator suggest catheterization associated with 10 percentage point reduction in mortality.
   
   E[Y|Z=1]-E[Y|Z=0]                 -0.7%
   -------------------------    =   --------    = -10.4%
   E[X|Z=1]-E[X|Z=0]                   6.7%

Slide 22

Other Examples of Distance IVs

• Brooks et al—Effect of dialysis center profit status on survival.
• McConnell KJ et al—Treatment of head injuries at level I vs level II trauma centers.
• Must be used studying a treatment that is dispensed at particular locations.
  • Not applicable to many prescription medications.
• Treatment must depend on distance.

Slide 23

Calendar Time as an IV

The slide presents a line graph showing "IV Status" by measuring the "Percentage of BB use after HF hospital" from 1993 to 2000. The graph shows a rather slow increase in use before mid 1997, "IV status: Before," and then a much faster increase in use after mid 1997, "IV status: After."

• Beta blocker after HF hospitalization and 1-year mortality.
• Note: Johnston et al. Stat Med 2008.

• Bias: Secular trends in other things related to the outcome.
• Best used when there is a dramatic shift in practice in a short time period: e.g, changes in guidelines, or safety warnings.

Slide 24

IVs can also be created

• 'Randomized encouragement' designs (Ten Have et al).
• Designed delays (McClure M., Dormuth C; work in British Columbia).

Slide 25

One-off Instruments

• Day of the week of hospital admission as an instrument for waiting time for surgery (Ho et al.)
• Surgeons operate only on weekdays and therefore patients admitted on the weekend may have to wait longer for surgical treatment.
• Bias: patients admitted on the weekend were different from those admitted on the weekday.
• Bias: IV could be independently related to the outcome if other aspects of hospital care that could affect the outcome were different over the weekend.

Slide 26

Characteristics of Good Application of IVs

• IV should be have theoretical motivation.
• IV should be strongly associated with treatment.
• IV should be largely unrelated to patient characteristics.
• Some consideration should be given to generalizing the estimate.
• Used in the setting of a large sample.

Slide 27

Role of Instrumental Variable

• IV assumptions are different from those underlying conventional approaches.
• Makes IV excellent for secondary analysis.
• Wang et al, NEJM 2005.
• Problem arises if methods given different results.
• IV method deserve primary status if IV is strong & valid, sample size is large, and unmeasured confounding expected to be great.

Slide 28

Coming soon to the AHRQ Web site...

• Practical guide to IV Methods for Comparative Effectiveness Research, by Brookhart, Rassen, and Schneeweiss.