Engaging Stakeholders, Improving Research: Stakeholder Engagement in the DEcIDE Program

Select to access the PowerPoint® presentation (100 KB).

Slide 1

Engaging Stakeholders, Improving Research: Stakeholder Engagement in the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Program

L. Ebony Boulware, MD MPH
Associate Professor of Medicine
Johns Hopkins University

Slide 2

Objectives

• Identify potential challenges to Stakeholder engagement in comparative effectiveness research (CER) protocol development.
• Discuss engagement strategies that involve Stakeholders in a meaningful way to enhance the quality and relevance of study outcomes.

Slide 3

DEcIDE Network

• Developing Evidence to Inform Decisions about Effectiveness.
• Collection of research centers that the Agency for Healthcare Research and Quality (AHRQ) created in 2005.
• Centers gather new knowledge and information on specific treatments.
• Conducts studies on the outcomes, effectiveness, safety, and usefulness of medical treatments and services.

Slide 4

From the Perspective of the Investigator: DEcIDE Network

• Research priorities established through previous Agency work:
  • Engagement of Stakeholders and technical experts.
  • Institute of Medicine CER priorities.
• Research opportunities competitively offered among centers:
  • Proposal submission.
  • Peer review.
• Contracted research:
  • Aims, approach often pre-specified during contract negotiation.

Slide 5

From the Perspective of the Investigator: DEcIDE Network

• Stakeholder engagement still required during research.
• Plan for engaging Stakeholders must be specified within research applications.

Slide 6

Emphasis of Stakeholder Engagement Changes During DEcIDE Projects

• Identifying and prioritizing topics of CER.
• Framing the questions.
• Selecting the comparators, outcomes.
• Creating conceptual framework.
• Data collection.
• Analyze and interpret results.
• Translation.
• Dissemination.

Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA 2012 Apr 18;307(15):1587-8. PubMed PMID: 22511684

Slide 7

What Key Challenges Does DEcIDE Research Present?

• Orient Stakeholders to established research:
  • Established questions, comparators.
  • Preliminarily established research methodology.
• Help Stakeholders understand their value.
• Identify ways Stakeholder activities can enhance existing research plans:
  • Enhance relevance.
  • Input on planned approaches with end user in mind.

Slide 8

DEcIDE Patient Outcomes in End Stage Renal Disease (ESRD) Study

• 3-year ARRA contract to compare effectiveness of three treatment strategies for patients with ESRD (hypertension treatments, dialysis timing, iron therapy).
• First Stakeholder meeting held within first 4 months of funding (October 2010):
  • Identifying and contacting Stakeholders.
  • Working with partner to strategize and plan (Center for Medical Technology Practice, CMTP).

*Priorities established through IOM recommendations.

Slide 9

Identifying Stakeholders (1)

Slide 10

Identifying Stakeholders (2)

Slide 11

Orienting Stakeholders to DEcIDE ESRD Patient Outcomes Project

• Orient Stakeholders to project:
  • Email brief executive summary.
  • Brief telephone call.
• Prepare for meeting:
  • Employed medical writer to distill 50 page research application into 10 page lay summary.
  • Planned meeting agenda with help of CMTP.

Slide 12

Preparing First Stakeholder Meeting

• In-person event in Baltimore, MD.
• Mailed Stakeholders materials before meeting:
  • Summary of project.
  • Detailed research protocols.
  • Expectations of stakeholder roles.
  • Meeting agenda and structure (pre-assigned breakout group leaders).
  • Questions to be addressed during meeting.
• Began to prepare investigative team.

Slide 13

Conducting Stakeholder Meeting

• Professional moderator.
• Discussed stakeholder roles, importance.
• Project summaries from research team.
• Breakout group discussions (organized by Stakeholder type):
  • Key questions to frame research approach.
  • Consider potential implications of positive or negative findings for decisionmakers.
• Informal lunch with group discussion of breakout group findings.
• Review discussion points and promised follow up.

Slide 14

Special Considerations

• Room setup (Stakeholders in center, investigators out of the way).
• Prior investigator preparation (i.e., more listening, less talking, less technical jargon).
• Interactive breakout groups with investigators as scribes, gathering information but little input.

Slide 15

Helping Stakeholders See Value of Input

• Audio recorded and transcribed entire meeting.
• Created detailed summaries of key points discussed.
• Distributed summary of meeting to Stakeholders.
• Prepared for in-person follow up.

Slide 16

Helping Stakeholders See Value of Input: Preparing for In-person Follow Up

• Research team digested transcribed Stakeholder comments.
• Responded point-by-point to comments, modifying research protocol where appropriate:
  • Enhancing outcomes to improve relevance to different decision-makers (e.g., quality of life).
  • Adjusting emphasis of approach (e.g., considering harms of therapies in addition to effectiveness).

Slide 17

Example Stakeholder Comments
Project 1: Antihypertensive Medications

• Theme 1: Examine relationship of blood pressure and volume control (fluid management).
• Response:
  • Several analyses planned to examine relationships between volume management and blood pressure control.
  • Studies to look at simultaneous changes in blood pressure, volume and blood pressure medications on outcomes.

Slide 18

Example Stakeholder Comments
Project 2: Timing of Dialysis Initiation

• Theme 1: Not enough information about pre-dialysis illness and events that led toward patients starting dialysis (e.g. sicker patients more likely to start earlier).
• Response: Plan to use alternate datasets to assess pre-dialysis medical history:
  • Pre-dialysis ESRD Medicare Claims ("Backcasted Data").
  • Proposed use of other data from other cohorts with CKD (AASK/MDRD) to explore rates of progression toward ESRD.
  • Exploring alternative datasets (e.g. Cleveland Clinic).

Slide 19

Example Stakeholder Comments
Project 3: Iron Management

• Theme 1: Impossible to disentangle simultaneous use of erythropoiesis stimulating agents and iron. Both should be considered as primary exposures of interest.
• Response:
  • Will explore simultaneous use of the two medications longitudinally.
  • Employ complex modeling strategies (marginal structural).

Slide 20

Year 2: In-Person Follow Up

• Second in-person meeting.
• Reviewed Stakeholder feedback and our responses in document provided to them and during presentation.
• Sought additional feedback.
• Provided research updates (preliminary findings).
• Obtained feedback on preliminary findings and.
• Asked Stakeholders for input on framing and dissemination strategies.

Slide 21

Year 3: Extending Impact of Stakeholder Engagement

• Planning third in-person meeting:
  • Pre-meeting engagement to get ideas about extension of current research, other relevant research studies in ESRD.
• Review final preliminary contract research findings, obtain feedback.
• Focus on framing and dissemination of final projects.

Slide 22

Final Planned Engagement

• Plan to send Stakeholders summary of input over life of project.
• Provide Stakeholders with some final products:
  • Obtain feedback on clarity of products.
  • Additional avenues for dissemination.
• Thank Stakeholders for generous time investment.
• Encourage future engagement.

Slide 23

Resource Investment

• Administrative (emails, letters, meeting materials coordination of meetings, obtaining COI statements) (45 hours per meeting).
• Medical writer (20 hours).
• Investigator preparation (20-40 hours depends on size of project).
• Investigator presence at meetings (8 hours for 6 investigators).
• Securing venue for meetings (travel, refreshments).
• Audio recording and transcribing meeting (40 hours).

Slide 24

Successful Strategies

• Early engagement of Stakeholders.
• Well-planned in-person meetings.
• Setting clear expectations of the Stakeholders.
• Demonstrating use of input from Stakeholders with report back.
• Continued engagement throughout project.

Slide 25

Challenges We Faced

• Presenting a lot of scientific information to group of busy Stakeholders:
  • Technical aspects difficult to convey.
  • Sheer amount of information.
• Ambitious contract research schedule:
  • Challenging to engage Stakeholders throughout year.
  • Would have liked to have resources to engage more frequently.
  • Much of the research happening in background while preparation for future meetings ongoing.

Slide 26

Summary

• Engagement of Stakeholders in contract research may require special consideration:
  • Help Stakeholders see their relevance.
  • Translate complex protocols for a broad range of audiences.
• Demonstrating utilization of Stakeholder input important.
• Advance planning critical to maximize output of Stakeholder groups:
  • Partner with experts in Stakeholder engagement.
  • Invest adequate resources and time to maximize yield.