Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality www.ahrq.gov
AHRQ Home—Live Site | Archive Home | Site Map
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Methodological Issues in Systematic Reviews: Formulating Questions (Text Version)

Slide Presentation from the AHRQ 2009 Annual Conference

On September 16, 2009, Thomas A Trikalinos made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (3.3 MB).

Slide 1

Methodological Issues in Systematic Reviews: Formulating Questions

Thomas A Trikalinos, MD, PhD
Co-Director, Tufts Evidence-based Practice Center

Author has no actual or potential conflicts of interest in relation to this activity

 

Slide 2

Asking the right question

Image: An image of two men is show. One man is in a guillotine.

 

Slide 3

We will cover

  • PICO(TS) approach to generate answerable research questions
  • Analytic framework to identify potential questions
  • Other considerations in formulating questions

 

Slide 4

Systematic Review/Meta-Analysis

? Question - Systematic Review/Meta-Analysis - ! Answer

 

Slide 5

Systematic review protocol

  • Well focused study question [ PICO(TS) ]
  • Identification of studies (studies design, source, search strategy)
  • Eligibility criteria (study, patient, and disease characteristics, treatments, outcomes)
  • Data extraction (definition of outcomes, quality assessment)
  • Data summary and analysis (outcomes used, intention to treat)

 

Slide 6

PICO(TS) approach to formulating answerable research questions

  • Population (participants)
  • Intervention
  • Comparator
  • Outcome
  • Time (duration)
  • Study design (experimental / observational, N)

 

Slide 7

Poorly formulated question:

What drugs should be used to treat patients with hypertension?

 

Slide 8

Example: The Well - Formulated Question

The Cochrane Collaboration "How to Conduct a Cochrane Systematic Review" 1996

Image: A flowchat is shown that is labeled "Does drug therapy decrease long-term morbidity and mortality in older persons with mild to moderate hypertension?"

 

Slide 9

Diagnostic test research question - test performance

What is the diagnostic accuracy of sestamibi scan to detect ACI in patients in the ED

Outcomes
  • Sensitivity
  • Specificity
Intervention
  • Technetium-99m sestamibi scan
  • Comparator
Condition of interest
  • AMI
  • Unstable angina
Setting population
  • Adults with symptoms suggestive of acute cardiac ischemia and non-diagnostic ECG seen in the ED

 

Slide 10

Diagnostic test research question - clinical impact -

What is the clinical impact of sestamibi scan to detect ACI in patients in the ED?

Outcomes
  • False negative discharges from the ED
    False positive admits
    Deaths
Intervention
  • Technetium-99m sestamibi scan
  • Comparator
Condition of interest
  • AMI
  • Unstable angina
Setting population
  • Adults with symptoms suggestive of acute cardiac ischemia and non-diagnostic ECG seen in the ED

 

Slide 11

We will cover

  • PICO(TS) approach to generate answerable research questions
  • Analytic framework to identify potential questions
  • Other considerations in formulating questions

 

Slide 12

Approach to (de)constructing research question

  • Question of interest is often over-arching and not directly addressable
    • Ex. How to manage hypercholesterolemia?
  • Need to break it down into answerable subunits
  • Multiple questions frequently needed
  • Final answers (recommendations) typically involve additional information, balancing the benefits and harms, feasibility, costs, etc

 

Slide 13

Simplified Evidence Model Linking Biological Effects (intermediate) and Health Outcomes

Image: An model of Linking Biological Effects (intermediate) and Health Outcomes is shown.

 

Slide 14

USPSTF generic analytic framework for screening topics

Image: The framework for screening topics is shown.

 

Slide 15

Analytic framework for omega-3 FA intake and CVD

Image: An image of Analytic framework for omega-3 FA intake and CVD is show.

 

Slide 16

We will cover

  • PICO(TS) approach to generate answerable research questions
  • Analytic framework to identify potential questions
  • Other considerations in formulating questions

 

Slide 17

Problems defining interventions and comparators

Pulmonary rehabilitation vs conventional care

Exercise training

AND OPTIONALLY

Education

Breathing exercises

Inspiratory muscle training

Psychosocial behavioral interventions

Vs.

Conventional care

AHRQ Evidence Report/Technology Assessment
Pulmonary rehabilitation for COPD, 2007; www.ahrq.gov

 

Slide 18

OK, but there are things that are self-evident, no?

 

Slide 19

Would You Say You "Had Sex" If ..?

Image: An image of an article titled "Would You Say You "Had Sex" If ..?" is shown.

JAMA 1999;281:275-7

 

Slide 20

Percentage Indicating "Had Sex" (95% Confidence Interval)

Image: An image of a table is shown The table breaks down the behaviors of women, men, and overal of indivuals perception of having sex.

JAMA 1999;281:275-7

 

Slide 21

Problems measuring genetic variables: (TA)n in ER alpha

Ann Intern Med 2006;145:255-64

 

Slide 22

Standardizing across labs

Ann Intern Med 2006;145:255-64

 

Slide 23

Problems in coming up with questions

  • Sometimes you don't know what should be the question because there is not yet agreement among investigators about the meaning
  • Pre-emptive analgesia: different investigators have different ideas about what it is
  • Solution: Rather than pick an arbitrary definition, try to group what has been studied into several similar categories and address them, this will provide evidence of the spectrum of mechanisms
  • Question: What is the effect of timing of analgesics on post-op requirements of pain medication and patient outcomes?

 

Slide 24

Problems with diagnosis

  • Varying definitions (lack of an adequate reference standard)
  • Different levels of rigor
  • Tradeoffs between rigor and real world practice (sinus puncture to diagnose sinusitis vs. actual clinical practice: efficacy vs. effectiveness)

 

Slide 25

Image: Skull showing sinus trouble

 

Slide 26

Possible definitions of acute bacterial sinusitis and their effect

Duration of symptoms

< 2 weeks

< 3 weeks

< 4 weeks

Diagnostic criteria

      

Sinus puncture and culture

Low likelihood for false positive
       

Radiography

May include many patients with viral infections
       

Clinical findings

May include many patients with viral infections

 

Slide 27

Many diseases have diverse manifestations

 

Slide 28

Some components of the clinical and paraclinical spectrum of poliomyelitis

Jenicek M. Epidemiology: The logic of modern medicine. Epimed, Montreal, 1995.

 

Slide 29

B. Hypothetical infectious disease example

Jenicek M. Epidemiology: The logic of modern medicine. Epimed, Montreal, 1995.

 

Slide 30

Management of Clinically Inapparent Adrenal Mass (Adrenal Incidentaloma): The Evidence Report

NIH State-of-the-Science Conference.February 4, 2002

 

Slide 31

Incidentaloma is NOT a disease

  • The prevalence of this phenomenon and the pathologies and their likelihood will depend on:
  • Patients with known cancers or not
  • Presenting signs and symptoms
  • Patient's age and sex
  • Biochemical activity findings
  • Initial diagnostic test used (CT, US, MRI)
  • Size of the mass (lower limit)

 

Slide 32

What are the pathologies reported among adrenal incidentaloma?

  • Adenoma
  • Adrenocortical carcinoma
  • Angiomyelolipoma
  • Cyst
  • Epithelial carcinoma
  • Hyperplasia
  • Hematoma
  • Lymphoma
  • Metastases
  • Myelolipoma
  • Neurinoma
  • Pheochromocytoma
  • Regenerative hepatic nodule
  • Renal angiomyolipoma
  • Retroperitoneal fibrosis
  • . . . more

 

Slide 33

Some possible definitions of incidentaloma

 

Slide 34

Issues in formulating a question

  • Narrow versus broad (for individuals/ subgroups or entire population)
  • Clinically meaningful and useful (based on sound biological and epidemiological principles)
  • Very broadly defined questions may be criticized for mixing apples and oranges
  • Very narrowly focused questions have limited generalizability and sometimes may lead to biased interpretations / conclusions

 

Slide 35

Benefits of getting the questions right at the start

  • Improves clarity of the problem
  • Facilitates subsequent steps of the systematic review
  • Helps select right kind of studies for review
  • Anticipates issues, resource needs
  • Improves efficiency, reduces needs to revisit questions / re-review papers, minimizes wasted efforts

 

Slide 36

Summary

  • PICO(TS) method is a very useful technique to formulate answerable research questions
  • Analytic framework can be useful to visualize and think about related issues
  • Defining criteria
  • Biologically meaningful
  • Question and criteria guide the entire systematic review process
Current as of December 2009
Internet Citation: Methodological Issues in Systematic Reviews: Formulating Questions (Text Version): Slide Presentation from the AHRQ 2009 Annual Conference. December 2009. Agency for Healthcare Research and Quality, Rockville, MD. https://archive.ahrq.gov/news/events/conference/2009/trikalinos/index.html

 

The information on this page is archived and provided for reference purposes only.

 
AHRQ Advancing Excellence in Health Care