How Can One Examine a Trial for Heterogeneity of Treatment Effect? (Text Version)

---

On September 28, 2010, Carlos Weiss made this presentation at the 2010 Annual Conference.

---

Slide 1

Accounting for Clinical Heterogeneity in Comparative Effectiveness Research

How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)? The Example of the BARI trial for CABG vs PTCA

September 28, 2010
Carlos Weiss, MD, MHS

Slide 2

AHRQ DEcIDE Project: Methods to Study the Heterogeneity of Treatment Effects in Comparative Effectiveness Research

PI: Ravi Varadhan, PhD
Co-I: Jodi Segal, MD, MPH; Cynthia Boyd, MD, MPH; Al Wu, MD, MPH

Consultant: David Kent, MD, MPH
Technical Experts: Curt Furberg, MD, PhD; Bruce Psaty, MD, PhD
Task Order Officer: Parivash Nourjah, PhD

Slide 3

Image: The cover of the New England Journal of Medicine from July 25, 1996 with the title "Comparison of Coronary Bypass Surgery with Angioplasty in Patients with Multivessel Disease."

Slide 4

BARI Clinical Question

Population targeted: "Multivessel disease" with severe angina or ischemia.
Intervention: PTCA (a form of PCI).
Comparator: CABG.
Outcome: 5-yr Mortality.

Slide 5

Questions to Audience—Set 1

What are sources of HTE?

How would pre-specification of analyses affect interpretation of results?

Slide 6

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:

• Angina severity

Slide 7

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:

• Angina severity
• Left ventricular function
• Number of diseased vessels
• Complex lesions

Slide 8

BARI Clinical Question: Sources of HTE in CABG vs PTCA

Slide 9

BARI Clinical Question: Sources of HTE in PTCA v CABG

• Patients:
  • Baseline risk
  • Competing risks
  • Risk of treatment harms
  • Treatment responsiveness

Ideas drawn from Kravitz, Duan & Braslow, 2004, Milbank Quarterly.

Slide 10

BARI Clinical Question: Sources of HTE in PTCA v CABG

• Patients:
  • Baseline risk
  • Competing risks
  • Risk of treatment harms
  • Treatment responsiveness
• Treatment
• Providers
• Environments

Slide 11

Image: Graph depicts how the sources of HTE—Patients, Treatment, Providers, and Environments—can interact with each other.

Slide 12

BARI Results

5-yr Mortality: Overall, no clinically significant nor statistically significant difference.

Slide 13

Image: A chart depicts the 4 subgroup analyses for the BARI trial at 5 years. There is also a 5th subgroup: treated diabetes that is absent or present.

Part way through the trial the safety and data monitoring board requested an analysis of diabetic patients on the basis of a subgroup analysis in a study of aggressive PTCA vs PTCA only if clearly needed. That analysis suggested an increased risk of death in patients with a history of diabetes AND no prior MI. Here, the line says history of diabetes but the variable is actually treated diabetes, defined as diabetes involving the use of insulin or oral hypoglycemic agents at entry into the study.

For the treated DM subgroup, the CABG arm did approximately 15% better than the PTCA arm with respect to 5-year survival.

Wider confidence intervals of 99 percent and 99.5 percent for the a priori and treated diabetes subgroups, respectively, were used to correct for multiple comparisons made here.

Slide 14

Image: A graph depicting survival over time is shown for four subgroups according to treatment arm and presence or absence of treated diabetes, with survival presented on the y-axis and time on the x-axis. The survival curve for patients with treated diabetes who received PTCA is lower than all the other curves, and approaches 60% survival at 6 years. The log-rank test for difference in survival between treatment arms, among patients with treated diabetes, has a p-value of 0.003, suggesting that CABG is superior to PTCA in this group.

Slide 15

Questions to Audience—Set 2

When should one be worried that a subgroup result is an error (Type I or Type II) ?

What can be done to lower error probabilities?

Slide 16

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?
2. Design and measurement quality?
3. Modeling pre-specified?

Slide 17

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?
2. Design and measurement quality?
3. Modeling pre-specified?
4. If No to 1, 2 or 3: Validation study available?

Slide 18

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?
2. Design and measurement quality?
3. Modeling pre-specified?
4. If No to 1, 2 or 3: Validation study available?
5. a. If frequentist, test of interaction performed?
6. b. If Bayesian, pre-specified priors and variance acceptable?

Slide 19

This slide is blank.

Slide 20

Extra Slides

Slide 21

What is Heterogeneity of Treatment Effect?

Non-random variability in the direction or magnitude of a treatment effect.

Slide 22

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

HTE is present on the absolute and/or relative risk scales of treatment effect. Treatment effect cannot be homogeneous in both scales (Figure 1), unless baseline risk is constant.

Figure 1. Heterogeneity of Treatment Effect Is Present in Absolute And/Or Relative Scales. Figure 1 Legend: Absolute baseline risk of the primary outcome is on the x-axis and is the source of HTE in this example. Risk if treated is on the y-axis and the dotted line indicates no treatment effect. Treatment effects according to quintiles of baseline risk are presented for 2 studies (closed, open circles). Treatment effect can be calculated according to absolute risk (solid line) or a relative risk (dashed line) or both. Corresponding effect models are that the absolute risk reduction equals 'a' and relative risk equals 'b'. Treatment effect cannot be homogeneous on both scales. Heterogeneity may be apparent in one effect model and not in the other effect model. The study represented by closed circles would not show HTE on a relative risk scale, but would show HTE related to baseline risk on an absolute risk scale; the study represented by open circles would not appear to have HTE on an absolute risk scale, but would on a relative risk scale.

Suppose that the treatment effect is constant on the absolute risk scale, i.e. Prob(Yi(2)=1) - Prob(Yi(1)=1) = a, for all individuals i, then the individual treatment effect on the relative risk scale is equal to 1 + (a/ Prob(Yi(1)=1)) and varies with individual's baseline risk. Conversely, suppose that the individual treatment effect is constant on the relative risk scale, i.e. Prob(Yi(2)=1)/ Prob(Yi(1)=1) = b, for all individuals i, then the individual treatment effect on the absolute risk scale is equal to (1-b) * Prob(Yi(1)=1), which also varies with individual's baseline risk.

Slide 23

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

Go to detailed overview of the graphs on Slide 22.

Slide 24

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

Go to detailed overview of the graphs on Slide 22.