Sensitivity Analysis for Residual Confounding (Text Version)

Slide Presentation from the AHRQ 2008 Annual Conference

By Sebastian Schneeweiss, MD, Sc.D.

On September September 9, 2008, Sebastian Schneeweiss made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (1.6 MB).

Slide 1

Sensitivity Analysis for Residual Confounding

Sebastian Schneeweiss, MD, Sc.D.
Division of Pharmacoepidemiology and Pharmacoeconomics
Department of Medicine, Harvard Medical School

Slide 2

Outline

1. Residual Confounding and what we can do about it.
2. Simple sensitivity analysis: Array Approach.
3. Study-specific analysis: Rule Out Approach.
4. Using additional information: External Adjustment.

Slide 3

Unmeasured (residual) Confounding

The model diagram presents a subject and a continuum of drug exposure through outcome.

Slide 4

Unmeasured Confounding in Claims Data

Database studies are criticized for their inability to measure clinical and life-style parameters that are potential confounders in many pharmacoepi studies:

• Over-the-counter (OTC) drug use.
• Body mass index (BMI).
• Clinical parameters: Lab values, blood pressure, X-ray.
• Physical functioning, ADL (activities of daily living).
• Cognitive status.

Slide 5

Strategies to Minimize Residual Confounding

• Choice of comparison group:
  • Alternative drug use that have the same perceived effectiveness and safety.
  • Multiple comparison groups.
• Crossover designs (CCO, CTCO).
• Instrumental Variable estimation.
• High dimensional proxy adjustment.

Slide 6

Strategies to Discuss Residual Confounding

• Qualitative discussions of potential biases.
• Sensitivity analysis (SA):
  • SA is often seen as the 'last line of defense.'
    1. SA to explore the strength of an association as a function of the strength of the unmeasured confounder.
    2. Answers the question "How strong must a confounder be to fully explain the observed association."
  • Several examples in Occupational Epi. but also for claims data.

Note: Greenland S, et al. Int Arch Occup Env Health 1994
Wang PS, et al. J Am Geriatr Soc 2001

Slide 7

Dealing with Confounding

Confounding:

• Measured Confounders:
  • Design:
    • Restriction.
    • Matching.
  • Analysis:
    • Standardization.
    • Stratification.
    • Regression.
    • Propensity scores.
    • Marginal Structural Models.
• Unmeasured Confounders:
  • Unmeasured, but measurable in substudy:
    • 2-Stage sample.
    • Ext. adjustment.
    • Imputation.
  • Unmeasurable:
    • Design:
      • Crossover.
      • Active comparator (restriction).
    • Analysis:
      • Instrumental variable.
      • Proxy analysis.
      • Sensitivity analysis.

Slide 8

A Simple Sensitivity Analysis

• The apparent relative risk (RR) is a function of the adjusted RR times 'the imbalance of the unobserved confounder'
  • An image of the equation is portrayed.
• After solving for RR we can plug in values for the prevalence and strength of the confounder:
  • An image of the equation is portrayed.

Slide 9

A Made-up Example

Association between tumor necrosis factor a (TNF-a) blocking agents and non-Hodgkin's (NH) lymphoma in rheumatoid arthritis (RA) patients:

• Let's assume an observed RR of 2.0.
• Let's assume 50% of RA patients have a more progressive immunologic disease.
• ...and that more progressive disease is more likely to lead to NH lymphoma.
• Let's now vary the imbalance of the hypothetical unobserved confounder.

Slide 10

Bias by Residual Confounding

The three dimensional chart displays a plane. The legend reads Fixed: absolute risk reduction (ARR) equals 2.0; Pco equals 0.5.

Slide 11

The slide shows a series of data in the background with two super imposed three dimensional charts displaying planes. An image displaying the url, drugepi.org, is prominently featured in orange.

Slide 12

Pros and Cons of "Array Approach"

• Very easy to perform using Excel.
• Very informative to explore confounding with little prior knowledge.

Problems:

• It usually does not really provide an answer to a specific research question.
• 4 parameters can vary -> in a 3-D plot 2 parameter have to be kept constant.
• The optical impression can be manipulated by choosing different ranges for the axes.

Slide 13

Same Example, Different Parameter Ranges

The three dimensional chart displays a plane. The legend reads Fixed: ARR equals 2.0; Pco equals 0.5.

Slide 14

Conclusion of "Array Approach"

• Great tool but you need to be honest to yourself.
• For all but one tool that I present today:
  • Assuming conditional independence of C_U and C_M given the exposure status.
  • If violated than residual bias may be overestimated.

Note: The slide presents the diagram from Slide 3.

Slide 15

More Advanced Techniques

Wouldn't it be more interesting to know:

• How strong and imbalanced does a confounder have to be in order to fully explain the observed findings?

Slide 16

• Example: Psaty, et al. CCB use and acute MI. JAGS 1999;47:749
• The issue:
  • Are there any unmeasured factors that may lead to a preferred prescribing of a calcium channel blocker (CCB) to people at higher risk for acute myocardial infarction (AMI)?
• An image of an article and graph are portrayed in the slide. The graph contains relative odds of myocardial infarction on the x-axis and relative odds of CCB use on the y-axis.

Slide 17

• Rule Out Residual Confounding; how strong does an unmeasured confounder have to be to fully explain the observed findings?
• The slide shows an image of a data series in the background and a graph. An image displaying the URL, drugepi.org, is prominently featured in orange.

Slide 18

Caution!

• Psaty, et al. concluded that it is unlikely that an unmeasured confounder of that magnitude exists.
• However, the randomized trial ALLHAT showed no association between CCB use and AMI.
• Alternative explanations:
  • Joint residual confounding may be larger than anticipated from individual unmeasured confounders.
  • Not an issue of residual confounding but other biases (e.g. control selection)?

Slide 19

Pros and Cons of "Rule Out Approach"

• Very easy to perform using Excel®.
• Meaningful and easy to communicate interpretation.
• Study-specific interpretation.

Problems:

• Still assuming conditional independence of C_U and C_M.
• "Rule Out" lacks any quantitative assessment of potential confounders that are unmeasured.

Slide 20

External Adjustment

• One step beyond sensitivity analyses.
• Using additional information not available in the main study.
• Often survey information.

Slide 21

Strategies to Adjust Residual Con-founding Using External Information

• Survey information in a representative sample can be used to quantify the imbalance of risk factors that are not measured in claims among exposure groups.
• The association of such risk factors with the outcome can be assess from the medical literature (Randomized controlled trials [RCTs], observational studies).

Note: Velentgas, et al. PDS, 2007
Schneeweiss, et al. Epidemiology, 2004

Slide 22

In our example:

The model diagram presents a subject and a continuum of Rofecoxib exposure through Acute MI.

• OREC: From Survey data in a subsample.
• RRCO: From medical literature.

Slide 23

More Contrasts

The table contrasts the following:

• Cox-2 (872) vs. non-selective NSAIDs (1,302).
• Cox-2 (872) vs. non-users (6,611).
• Cos-2 (872) vs. naproxen (238).
• Rofecoxib (244) vs. naproxen (238).

The potential confounders include obesity, aspirin use, smoking, educational attainment, and income status.

Slide 24

Sensitivity of Bias as a Function of a Misspecified RRCD: Obesity (BMI > = 30 vs. BMI < 30)

The graph displays an x-axis range of 1 through 4.5 and a y-axis range of -20 through 20.

• Cox-2 vs. non-selective NSAIDs: Begins at 0,0 and ends at 4.5,0.
• Cox-2 vs. non-users: begins at 0,0 and ends at 4.5,15.
• Cox-2 vs. naproxen: begins at 0,0 and ends at 0,8.
• Rofecoxib vs. naproxen: Not graphed.

Slide 25

Sensitivity Towards a Misspecified RRCO From the Literature: OTC Aspirin Use (y/n)

The graph displays an x-axis range of 0.1 through 1, and a y-axis range of -20 through 20.

• Cox-2 vs. non-selective NSAIDs: Begins at 0,1 and ends at .94,0.
• Cox-2 vs. non-users: begins at 0,0 and ends at .36,0.
• Cox-2 vs. naproxen: begins at 0,-1 and ends at 1,0.
• Rofecoxib vs. naproxen: 0,-4 and ends at .96,0.

Slide 26

• External Adjustment: Given external information for selected factors on OREC from survey data and RRCD from the literature, how much confounding is caused by not controlling for these factors?
• The slide shows an image of a data series in the background and a graph. An image displaying the URL, drugepi.org, is prominently featured in orange.

Slide 27

Limitations

• Example is limited to 5 potential confounders: No lab values, physical activity, blood pressure, etc.
• What about the 'unknown unknowns?'
• To assess the bias we assume an exposure-disease association of 1 (null hypothesis):
  • The more the truth is away from the null the more bias in our bias estimate.
  • However the less relevant unmeasured confounders become.
• Validity depends on representativenes of sampling with regard to the unmeasured confounders.
• We could not consider the joint distribution of confounders.
• Limited to a binary world.

Slide 28

Solving the Main Limitations

• Need a method:
  • That addresses the joint distribution of several unmeasured confounders.
  • That can handle binary, ordinal or normally distributed unmeasured confounders.
• Lin, et al. (Biometrics 1998):
  • Can handle a single unmeasured covariate of any distribution.
  • But can handle only 1 covariate.
• Sturmer, Schneeweiss, et al. (Am J Epidemiol 2004):
  • Propensity score calibration.
  • Can handle multiple unmeasured covariates of any distribution.

Slide 29

Summary

• Sensitivity analyses for residual confounding are underutilized, although they are technically easy to perform.
• Excel® program for download (drugepi.org).
• The real challenge is the interpretation of your findings.
• This is all summarized in Schneeweiss PDS 2007.