How to Get Insightful and Useful Audience Input for User Friendly Tools (Text Version)

On September 14, 2009, Maritza E. Matheus made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (662 KB).

---

Slide 1

How to Get Insightful and Useful Audience Input for User Friendly Tools

Maritza E. Matheus, Principal
AHRQ Annual Conference
Monday, September 14, 2009 

Slide 2

Audience Exercise

• Need: paper and pen
• On middle of page, write "packing suitcase for business trip" and circle it 

Slide 3

What are we discussing today?

• When is it useful to obtain audience input?
• Why conduct formal testing?
• Approaches
• Testing mechanics
• Do's and Don'ts
• Examples of eliciting information
• Useful tactical tips
• Audience exercise 

Slide 4

When is it Useful to Obtain Audience Input?

• Developing a consumer product
  • Planning a product
  • Pre-testing of a prototype
  • Post-testing of a model
• Testing an existing product
• Testing messages
• Developing an advertising campaign 

Slide 5

Qualitative Research

• To gather an in-depth understanding of human behavior; answers the "why"
• Provides insights, feedback and information not gleaned from quantitative approaches
• Key: use gleaned information into actionable items and strategies 

Slide 6

Approaches

• Focus group
• Dyad/Triad
• One on One/In Depth Interviews
• Telephone
• Online
• Blogs 

Slide 7

Approaches

Approach	Participants	Duration	Pros	Cons
Focus Group	8-10	1.5-2 hours	Rich in insights, group dynamics	$
Dyads/Triads	2 or 3	1 -1.5 hours	Probe deep	Conduct numerous
One on One Interviews / In Depth Interviews	1 person	1 hour or less	Face to face, phone Delve deep	No group dynamics Conduct numerous
Telephone	1 person	1 hour or less	Good for difficult to schedule participants	Miss face to face cues, interaction
Online	Numerous	1 hour or less	Good for difficult to schedule participants	Miss face to face cues, interaction, $

 

Slide 8

Testing Mechanics

• Clear Objectives
• Planning!
• Appropriate Approach or Combination of Approaches
• Budget
• Qualitative Researcher or In-house
• Venue - field facility or other
• Recruiting (Demographics; Recruiting Guide or Screener)
• Moderating (Topic Areas; Discussion Guide)
• Reporting 

Slide 9

Do's

• Do allow enough time for testing, including approvals
• Do be inclusive in selecting audience (users, non-users, clinicians, family, educators, etc)
• Use tools: flip chart, markers, actual products
• Do honor time limit set with participants
• Do provide food
• Do provide incentives - be creative
• Do audiotape and video tape
• Do set time limit on topic areas
• Do pre and post testing 

Slide 10

Don'ts

• Don't allow friends or family in same groups
• Don't allow for "seasoned" participants
• Don't accept all of the recruited participants (do over recruit)
• Don't ask participants to share their job title or profession unless necessary
• Don't ask "WHY" - use other ways to elicit information 

Slide 11

Examples of Eliciting Information

• Flip Chart
  • Easel, markers
• Design Team
• Visualization
• Word Bubbles
• Picture Decks
• Mind Map 

Slide 12

Word Bubbles

• Use: when learning about sensitive, conflicting or politically-charged attitudes
• Tools: handout with stick figures with bubbles

Two Stick Figures each one with cloud bubble coming from their heads and each figure has a rectangular shape coming from their mouths. The participant is given described a situation in which he/she and another person are face to face. The participant writes in the respective bubble what they are thinking, what they actually say; what they think the person is thinking and what the person actually says.

Slide 13

Picture Decks

• Use: need to learn about emotional associations about an experience or an issue
• Tools: deck of images

Photos of a spider in a web and a women's softball team in a huddle. 

Slide 14

Mind Mapping

• Use: when learning more details or more indepth understanding of behavior, situation, or relationship
• Tools: paper and markers

Flow chart with the words "Quit Day" in the middle and circled. From it, there are 3 branches. The first branch has "Triggers" in a circle, the second branch has "Bad Moods" in a square and from that, "Eating" in a square and from that "Weight Gain" in a square. These represent thoughts a participant may be having who is thinking about quitting smoking and as "quit day" approaches. 

Slide 15

Mind Map

• Example of Completed Mind Map

Flow chart with Disney World is in the middle of the page and circled and it has several branches coming from it.

• Disney World
  • Mickey Mouse, Commercialization, Childhood, Family
  • Florida, Flying, Long lines, Activities, Summer Clothes, Exercise - toning
  • Costly, Save $
  • Family vacation, Fun, Memorable, Kids cranky, No adult time 

Slide 16

Tactical Tips

• Over-recruit Latinos and African-Americans
• Pay for transportation for indigent or elderly
• Mail publications in advance for participants to review before group testing
• Best not to mix some participants, such as physician specialties or physicians with other clinicians
• Consider on-site interviews when appropriate
• Ask colleagues and observers for their insights 

Slide 17

Audience Exercise

• Need: paper and pen
• On middle of page, write "packing suitcase for business trip" and circle it
• Example
• Take 2 minutes to write thoughts that come to mind when packing a suitcase
• When done, circle the most significant branch 

Slide 18

Thank You!

• Questions?
• Contact:
  • Maritza E. Matheus
  • Matheus Marketing, LLC
  • Oakton, VA 22124
  • Tel 703 758-7595
  • Email matheusmarket@aol.com