Heterogeneity and Comparative Effectiveness: Implications for Studies, Reviews, and Policy (Text Version)

---

On September 28, 2010, David Atkins made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (424 KB).  

---

Slide 1

Heterogeneity and Comparative Effectiveness: Implications for Studies, Reviews, and Policy

David Atkins, MD, MPH
Health Services Research and Development
Quality Enhancement Research Initiative
Dept. of Veterans Affairs

Images: The Department of Veterans Affairs and Department of Defense logos. These two logos appear on all subsequent slides.

Slide 2

Session Overview: How to Examine for Heterogeneity and Its Implications

• Within an individual study.
• Within a systematic review or meta-analysis.
• In Policy Decisions.
• Conclusions.

Slide 3

Heterogeneity of treatment effect

• Non-random variation in direction or magnitude of individual treatment effects within a population.
• May be due to factors specific to patient, provider, environment, or treatment.
• Also referred to as clinical heterogeneity.

Slide 4

"Clinical trials are good experiments but poor surveys." —N. Longford

Image: A graph depicts what happens when individuals are sampled from the population:

Sample 1 generates a correct estimate of the average treatment effect but under-represents variation (HTE).

Sample 2 is not only under-represents variation but misrepresents the average.

Only sample 3 gives good population estimates of both the average effect and HTE.

Slide 5

Heterogeneity vs. Applicability

• HTE—How great is variation in the treatment effect for individuals within the study population relative to the "average" effect in that population?
• Applicability—Do the results for the study population apply to a specific patient, group or setting of interest (outside of the study)?

Slide 6

"Clinical trials are good experiments but poor surveys." —N. Longford

Image: A graph depicts what happens when individuals are sampled from the population:

Sample 1 generates a correct estimate of the average treatment effect but under-represents variation (HTE).

Sample 2 is not only under-represents variation but misrepresents the average.

Only sample 3 gives good population estimates of both the average effect and HTE.

Slide 7

Relevance in Policy Debates

• Concern that "average" effects don't apply to individual patients.
• "Cookbook medicine."
• Decisions based on CER that ignore heterogeneity might deny clinicians ability to individualize treatment decisions.

Slide 8

Is alendronate (Fosamax) better than calcium and vitamin D to prevent fracture?

• What factors might cause heterogeneity in the net benefits of alendronate vs. calcium/vitamin D?
• Audience response

Slide 9

Sources of Heterogeneity in Net Benefits of Treatment

• Responsiveness to treatment:
  • e.g., Calcium intake, vitamin D status.
• Vulnerability to harms:
  • ? Risks from bisphosphonates
• Baseline risk in absence of treatment:
  • Age, health status, bone density
• Values attached to outcomes:
  • Convenience of regimens

Slide 10

Scenario

• 63 year old woman with diabetes and ischemic heart disease, evaluated for cardiac procedure to address increasing exertional chest pain despite optimal medical therapy. She has two-vessel disease with preserved LV function. She is pondering whether to have PCA with stent or bypass.

Slide 11

Question

• Is bypass surgery or percutaneous coronary intervention (PCI) superior for treating coronary artery disease?
• Does the answer depend on patient or intervention factors (i.e. is there heterogeneity of the treatment effects )?