Understanding Development Methods From Other Industries to Improve the Design of Consumer Health IT

Select to access the PowerPoint® presentation (3.9 MB).

Slide 1

Understanding Development Methods From Other Industries to Improve the Design of Consumer Health IT

Enid Montague, PhD

Agency for Healthcare Research and Quality
Task Order No. 10
Contract No. HHSA2902009000231

Slide 2

Project Oversight

• AHRQ Task Order Officer: Teresa Zayas-Cabán.
• AHRQ Alternate Task Order Officer: Kevin Chaney.
• Project Director: Enid Montague.

Slide 3

Project Team

• Jennifer Crafts (Westat).
• David Damico (Westat).
• Patricia Flatley Brennan (University of Wisconsin—Madison).
• Enid Montague (Northwestern University).
• Ritu Agarwal, Center for Health Information and Decision Systems (CHIDS) (U of MD).
• P.K. Kannan, Center for Health Information and Decision Systems (CHIDS) (U of MD).
• Kenyon Crowley, Center for Health Information and Decision Systems (CHIDS) (U of MD).

Slide 4

Background and Impetus for Project

• Slow adoption of consumer health IT.
• New user-centered design approaches for development of consumer products are emerging in a variety of industries.
• Success of product development approaches, methods, and philosophies in other industries could generalize to personal health care.

Slide 5

Project Objectives

• Identify new and emerging practices for developing consumer products that may generalize to designing consumer health IT applications.
  1. Conduct environmental scan and grey literature review to Identify relevant tools and design methods.
  2. Interview key informants about design processes.

Slide 6

Objective One

• Conduct environmental scan and grey literature review to Identify relevant tools and design methods.

Image: An orange-tinted image shows a sheet of paper filled with binary code.

Slide 7

Environmental Scan and Grey Literature Review: Method Overview

Image: At the center of the figure is a circle captioned "Implications for the Design of Consumer Health IT Applications." Above and on either side of this circle are five squares, containing the following text: "The Domain of Design: Design, Philosophy, Process, Methods," "Design Methods (18) for Consumer Products and their Classification, " "Successful Digital Consumer Products (24)," "Design Methods used for Successful Digital Consumer Products," and "Categories of Consumer Health IT Applications." Arrows point from each of these squares toward the circle. Another arrow points from the circle to a rectangle below it that contains the following text: "Recommended Design Methods for Categories of Consumer Health IT Applications."

Slide 8

Environmental Scan and Grey Lit Review: Findings

• Considerable variety in methods used across products; common themes are captured.
• Choice of method(s) likely influenced by the nature of the consumer health IT application:
  • Development of a hybrid product (integrated physical device and digital component as a single product from the manufacturer, e.g. Fitbit) vs. development of a purely digital product (e.g., WebMD).
• Limitations of study:
  • Design method is just one of many factors in product success.
  • Focus of review was design methods using an engineering approach.

Slide 9

Environmental Scan and Grey Lit Review: Overview of Findings

• Observed many methods used by successful products; applying, disseminating and incentivizing use of good design may be key point.
• Generally for all consumer products:
  • Used methods that incorporate high levels of user involvement and iteration.
  • Used one or more of the most frequently used design methods.
  • Design teams contained medium breadth with regard to size and skill-sets.
  • Engaged human factors experts.
  • Strived to "Keep it simple"—parsimonious feature set.
  • Paid careful attention to user characteristics.

Image: Three arrows pointing in a clockwise direction are joined together to form a circle.

Slide 10

Environmental Scan and Grey Lit Review: Overview of Findings (cont'd)

Additional design considerations for hybrid consumer products:

• Used multiple design methods.
• Used prototypes and consumer feedback based on their use.

Image: A bar graph shows the average number of design methods used: Hybrid, 4.4; Digital, 2.7.

Slide 11

Objective Two

• Interview key informants about design processes.

Image: Two people shown in silhouette are seated at a table for an interview.

Slide 12

Key Informant Interviews: Approach and Challenges

• Approach:

  Image: A large red arrow points from left to right. Five squares are superimposed on this arrow; from left to right, they depict the following steps:

  • Developed interview guide.
  • Identified successful consumer products.
  • Identified potential key informants.
  • Conducted interviews.
  • Identified emerging themes from interviews.

• Challenges:

  • Company restrictions on employees' participation in research.
  • Availability of all original key informants.
  • Rapid turnover within consumer design field.

Slide 13

Key Informant Interviews: Selection Criteria

Recognized market success, as determined by:

• Consumer Reports publication.
• Rated best and most successful products of the year.
• Received awards from product design.
• Number of users or unique visitors.

Key informants who could speak knowledgeably about the product or design process:

• Founders.
• Designers involved with product development.
• Senior management.

Product features relevant for consumer health IT:

• Storing, archiving, and retrieving information.
• Monitoring health-related information.
• Searching for information.
• Supporting task completion such as logging activities, making comparisons, and making decisions.

Slide 14

Key Informants' Products and Companies

Slide 15

Key Informants Interviewed

• Company size range: 4 - 50,000.
• Roles and titles:
  • Design, User Experience, or Human Factors:
    • User Interaction Designer.
    • Senior Designer.
    • Senior User Interaction Designer.
    • Human Factors Engineer.
    • User Research Lead.
  • Management:
    • Senior VP of Product and Program Management.
    • CEO and Founder.
    • Director, Innovation and Advanced Technology.
    • Co-founder.

Slide 16

Methods

• Used interview guide to conduct interviews.
• Recorded and transcribed interviews.
• Analyzed transcripts using NVivo software and a thematic analysis method:
  • Coded interviews based on the key elements in the interview guide (idea generation, identifying end users, concept development, testing, usefulness, strategies influencing commercial success).
  • Analyzed transcripts for overall themes.

Slide 17

Key Informant Interviews: Overview of Primary Themes

Idea Generation:

• Generating ideas for successful consumer products involves a mixture of intuition and in-depth understanding from the market and users.
• This process used in successful products is one that includes both creativity and user and market research.

Identify End Users:

• Identifying end users and user needs involves understanding user experiences in the real world, and observing their successes and frustrations with existing products.
• It also involves conducting research with users and translating these research outcomes into measures of product success and user profiles.

Concept Development:

• Understanding user needs driven by definition of usefulness, market segmentation approaches, and prototyping.
• Prototyping is widely used in the design process. Multiple prototyping techniques are used with an emphasis on low fidelity methods.
• New design methods include releasing imperfect products into the market and providing updates, conducting in-depth observations of human behavior to inform design, and employing rapid prototyping and evaluation techniques.

Slide 18

Key Informant Interviews: Overview of Primary Themes (cont'd)

Role of Testing:

• Testing should take place throughout the design process and facilitate iterative design of prototypes.

Defining Usefulness and Ease of Use:

• Informants believe ease of use to be important in their design activities, but definitions and measures of the construct vary.
• Usefulness is considered important, but is more loosely defined.

Strategies Influencing Commercial Success:

• Many techniques are used to influence success, ranging from the product's design, to marketing strategies, to customer support services.
• Without usability, products are not likely to achieve success.
• However, it takes more than a usable product to achieve product success.

Slide 19

Key Informant Interviews: Secondary Themes

• Secondary themes:
  • Designing for entertainment and experience versus usability and functionality.
  • Leveraging people relationships in product design or business strategies.

Image: A photograph shows a small boy next to an older adolescent on a sofa, both holding controls for a video game.

Slide 20

Areas for Future Research

• Building evidence base of effective consumer health IT application design methods may be addressed by:
  • Comparing alternative levels of user involvement and iteration for different types of consumer health IT applications.
  • Isolating design processes currently in use for consumer health IT applications.
  • Developing case studies retrospectively documenting design processes of market leaders of consumer health IT applications.
  • Identifying contingencies that may affect the efficacy of different design methods for different user populations.
  • Investigating user response to products that are modular in nature.

Slide 21

For More Information

Images: The covers of the two Improving Consumer Health IT Application Development: Lessons From Other Industries reports are shown.

Slide 22

Acknowledgements

• Technical Expert Panel members:
  • Tony Andre (Interface Analysis Associates).
  • Deborah A. Boehm-Davis (George Mason University).
  • Sara Czaja (University of Miami, CREATE).
  • Judith Gregory (IIT Institute of Design).
  • Seth Howard (Epic).
  • Tony Hu (American Innovative, LLC).
  • Rita Kukafka (Columbia University).

Slide 23

THANK YOU!

For additional information please feel free to contact me or visit healthit.ahrq.gov.

www.enidmontague.com