Using Value of Information to Prioritize Future Research: A Case Study (Text Version)

On September 29, 2010, Gillian Sanders made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (215 KB).

---

Slide 1

Using Value of Information to Prioritize Future Research: A Case Study

Gillian D. Sanders Ph.D.
Duke Evidence-Based Practice Center
Duke University

On bottom left of every page is the organization Duke Clinical Research Institute.

Slide 2

Duke EPC Collaborators

• Evan Myers M.D., M.P.H
• Laura Havrilesky M.D.
• David Matchar M.D.
• Benjamin Powers M.D.
• Matthew Crowley M.D.
• Ravi Dhurjati Ph.D.
• Gregory Samsa Ph.D.
• Amanda McBroom Ph.D.
• Michael Musty B.A.
• Rebecca Gray Ph.D.

Slide 3

Overview of Project

• Investigating role of value of information analysis (VOI) in helping prioritize research gaps identified as part of EPC reviews
  • Systematic review of priority setting methods
  • Survey of existing priority setting methods
  • Two case studies of evidence reports with models
    • Comparative effectiveness of ACEI/ARBs in patients with ischemic heart disease
    • Comparative effectiveness of management strategies for uterine fibroids

Slide 4

Background

• Uterine fibroids are the most common, non-cancerous tumors in women of childbearing age and the second most common reason women of childbearing age undergo surgery.
• Despite the prevalence and possible complications of uterine fibroids, the currently available evidence-based literature on their natural history and optimal management is insufficient.
• DECIDE project to develop research priorities for comparative effectiveness research for fibroids:
  • Grew out of evidence gaps identified in systematic reviews conducted by EPCs
  • Used modified delphi/nominal group methodology
• Current project investigating use of modeling in the uterine fibroids evidence gap prioritization process.

Slide 5

Methods

• Updating model developed for original Duke Evidence Report on Fibroids
• VOI for subset of priority questions identified through DeCIDE project
• Comparison of VOI results to results of DeCIDE project
• Presentation and discussion of VOI results with subset of stakeholders who participated in DeCIDE project
  • Value of value of information? (VOVOI!)
  • Optimal timing relative to qualitative process

Slide 6

Questions

• Is VOI feasible for a condition like fibroids where the evidence gaps are so large?
• Can VOI complement other priority-setting methods?
• If so, what is best timing of VOI?
• Model focuses on subset of research gaps identified in DeCIDE project
  • Further model development needed
  • Results interesting, but preliminary
  • Focus is on whether this process could be helpful, not these specific results

Slide 7

Model Schematic

Image: A model schematic for the treatment of fibroids shows the following process: Symptomatic fibroids → Treatment → Short-term complications → Improved symptoms → Long-term complications, Improved symptoms, or Pregnancy → No complications or Pregnancy complications.

Slide 8

Model

• Interventions:
  • Myomectomy (surgical removal of fibroid)
  • Uterine artery embolization (UAE-angiographic occlusion of uterine blood supply)
  • MRI-guided focused ultrasound (FUS-ultrasonic energy applied directly to fibroid)
• All 3 preserve uterus:
  • Varying amount, quality of data on recurrence
    • FUS—shortest recovery time, highest recurrence:
  • Limited data on pregnancy outcomes with UAE, FUS (both discouraged in women seeking pregnancy)

Slide 9

Model

• Patient population:
  • Fibroids more common in African-American women
    • At younger ages, more severe disease
  • Age and racial distribution taken from 3000+ patient registry of women undergoing UAE:
    • Mean age 40, 48% African-American
  • Assumed 25% planned pregnancy
    • Based on pre-procedure data from registry
    • Approximately 5% reported pregnancy in follow-up
    • Age distribution modeled separately based on desire for pregnancy

Slide 10

Model

• Age- and race-specific probabilities of fertility, miscarriage, preterm delivery:
  • Older women less likely to get pregnant, higher risk of miscarriage or preterm birth.
  • African-American women higher risk for preterm birth
• This version of model assumes no difference in pregnancy outcomes between treatments:
  • Key area of uncertainty
  • Left out for purposes of simplifying analysis/presentation for stakeholders

Slide 11

Model

• Specific treatments/outcomes after recurrent symptoms not modeled:
  • Simplification
  • Subjects stayed in "retreatment" state until end of simulation
  • Utility of "retreatment" varied widely
• Model run for 3 years or until patient reached age 45
  • Spontaneous pregnancy highly unlikely after 45
  • Evidence for interaction between declining ovarian function and treatment efficacy:
    • Necessary next step, but out of scope

Slide 12

Model Output

• Cumulative probability of events:
  • Assumes 25% of patients attempting pregnancy

Outcome	UAE		Myomectomy		FUS
	Mean	SD	Mean	SD	Mean	SD
Recurrence	0.095	0.924	0.082	0.275	0.225	0.412
Pregnant	0.087	0.282	0.097	0.296	0.085	0.228
Live birth	0.055	0.228	0.063	0.243	0.055	0.222
Preterm birth	0.012	0.110	0.014	0.111	0.012	0.104
(% livebirth)	22.4%		21.8%		22.0%

• 2500 women actively trying to get pregnant within 6 months of procedure required to 800-900 pregnancies, 500-600 live births within 3 years.
• Sample size requirements to determine differences in reproductive outcomes between procedures daunting.

Slide 13

Acceptability Curve

Image: The Acceptability Curve is shown as a line graph, comparing probability of cost-effectiveness against willingness to pay for myomectomy, UAE, and focused ultrasound. The myomectomy line remains at 0.1 probability from $0k to over $90K. The UAE line begins at just above 0.2 probability and rises slightly to 0.3 by over $90K. The focused ultrasound line begins at 0.65 probability and decreases to 0.5 by over $90K.

• Graph shows the proportion of simulations (n=10,000) that a given strategy was preferred at different thresholds of cost-effectiveness.
• FUS has shorter length of stay, faster return to work, lower short-term complications than other procedures → lower costs.

Slide 14

Individual Parameters

Parameter	EVPPI
Utility after retreatment	$1053
Time to return to work	$1047
Time before trying to get pregnant	$1046
Relative recurrence rates	$1043
Time between recurrence and retreatment	$1038
Cost of complicated cases	$1037
Length of stay (all)	$1030
Cost of uncomplicated cases	$9.50

• Highest VOI related to recurrence-both probability and impact on QOL.
• Peri-procedural factors with high uncertainty or high potential contribution to differences (cost of complications, time to return to work) also high VOI.

Slide 15

Different Patient Populations

• Relatively small differences when run separately for white and African-American patients:
  • Main known differences between groups are younger age, higher risk of adverse pregnancy outcomes in black women.
  • Differences likely greater if age-specific treatment effects or treatment effects on reproductive outcomes included.

Images: Two line graphs depicting the partial expected value of perfect information (EVPI) vs WTP Curve for Whites and African-Americans are shown. For Whites, EVPI begins at $1010 at $50K threshold for cost-effectiveness and rises evenly to $1130 at $65K, $1260 at $80K, and $1360 at $95K. For African-Americans, EVPI begins at $1050 at $50K threshold for cost-effectiveness and rises evenly to $1175 at $65K, $1300 at $80K, and $1450 at $95K.

Slide 16

Different Patient Populations

• In this version of model, pregnancy parameters are fixed.
• Pregnancy is a competing risk with recurrence, and assume QOL if pregnant same as improved symptoms:
  • Fewer recurrences, higher overall QOL
• Again, potential importance of age-dependent recurrence risk, differential effects of treatment on pregnancy.

Images: Two line graphs depicting the partial EVPI vs WTP Curve for women desiring pregnancy and not desiring pregnancy are shown. For women desiring pregnancy, EVPI begins at $565 at $50K threshold for cost-effectiveness and rises evenly to $590 at $65K, $610 at $80K, and $630 at $95K. For women not desiring pregnancy, EVPI begins at $1160 at $50K threshold for cost-effectiveness and rises evenly to $1310 at $65K, $1460 at $80K, and $1610 at $95K.

Slide 17

(Very) Cautious Interpretations of VOI Results

• Highest priority topics by VOI related to durability of treatment and effect on QOL:
  • Aligns with #1 priority from DeCIDE qualitative process
  • Unlikely to change with additional model refinement
• Importance of impact of treatments on pregnancy outcomes population-dependent
  • Pregnancy outcomes important, but small proportion of overall patient population
  • Sample size issues in addressing this research gap

Slide 18

Feedback from Stakeholders

• All found results provocative, interesting, helpful in thinking about research priorities.
• Some found VOI results most helpful after qualitative exercise:
  • Qualitative process narrowed options
    Easier to understand VOI when only a few parameters varied
• Others preferred before or parallel to qualitative process
  • Would have helped focus discussion
    Potential way to resolve persistent differences between stakeholders

Slide 19

General Lessons/Next Steps

• Even with pre-existing model as basis, more work than anticipated:
  • Would add significantly to resources needed if done as part of CER.
    Some of this may be attributable to relatively poor state of evidence and relative complexity of modeling outcomes (symptoms And pregnancy) for this clinical area.
• Stakeholders generally very positive about use of quantitative analysis
  • One major challenge is need to educate on both VOI approach in general And content-specific application.
    Could consider shared resource on methods.

Slide 20

General Lessons/Next Steps

• Optimal timing still not clear:
  • Need to develop better method for testing
• Fibroids model/VOI promising:
  • Adding uncertainty about relative treatment effects on reproductive outcomes
  • Adding age-specific effects on ovarian function and interaction with treatment effects on symptoms
  • More specific modeling of retreatment options
  • Move decision point earlier in process to evaluate nonsurgical treatment (medication, watchful waiting)