Evaluation of Health IT Tools and Resources Available at the AHRQ NRC for Health IT Web Site: Final Report

1.1 Report Overview

Chapter 1 of this report provides an introduction to the overall project, research questions, introductory information about the National Resource Center for Health IT (NRC), and the Health IT Literacy Guide. Then the current state of health literacy and health IT literacy is presented, including definitions and theoretical constructs. In Chapter 2, methods are described for each of the three phases of research. In Chapter 3, research findings are presented to address the research questions 1, 2, 3, and 4 as stated in Section 1.3. In Chapter 4, recommendations for changes to the Guide are presented based on the research findings. Table 1 outlines the report structure.

Table 1. Report overview

Return to Contents

1.2 Background

The NRC was initially launched in 2004 to deliver technical assistance to AHRQ Health IT Portfolio grantees. Since then, it has become a public resource for sharing research findings, best practices, lessons learned, and funding opportunities with health IT researchers, implementers, and decisionmakers. The NRC Web site contains over 10,000 freely available documents, presentations, articles, and tools. As one of the largest repositories of information related to health IT, it has become a primary "go-to" resource for AHRQ's health IT stakeholders.

1.2.1 Project Background

Literacy, especially in the health context, is an important and evolving area of research ( Berkman et al., 2011; U.S. Department of Health and Human Services, 2010; McCormack et al., 2010). Low literacy can reduce the capacity of an individual to obtain, communicate, process, and understand basic health information and services (Koh et al., 2012). Literacy, defined narrowly, is the ability to read, write, and understand language. The term is also understood more broadly to mean the ability to "identify, understand, interpret, create, communicate, and compute, using printed and written materials associated with varying contexts" (UNESCO, 2003). A related term, health literacy, is the focus of the 2004 IOM Report, Health Literacy, a Prescription to End Confusion, which defines health literacy as having the capacity to "obtain, process, and understand basic health information and services needed to make appropriate health decisions". Health literacy is the term used more commonly in this report.

Engaging patients and their family members in care activities using health IT requires reliable methods to ensure technology accessibility (Koh et al., 2013) and health IT resources that serve individuals regardless of their level of health literacy. Health IT is viewed as an essential component for addressing health literacy (U.S. Department of Health and Human Services, 2010). System designers and Web content suppliers can develop IT resources that address health literacy and accessibility gaps if they apply relevant training and resources when health IT is designed and implemented, such as the Health IT Literacy Guide. However, the NRC Web site activity for the Health IT Literacy Guide was minimal (34 hits during March 2011, as compared with 1,486 view of the Health IT Tools and Resources home page). This raises questions about how these important resources are sought out and accessed by developers and purchasers of health IT and how to increase the Guide's impact.

The purpose of this evaluation was threefold: (1) to evaluate the Health IT Literacy Guide and determine how useful and easy it is to use for its intended audiences; (2) to provide specific recommendations for revising the Health IT Literacy Guide, including how the changes should be implemented and maintained; and (3) to offer general guidance on how to develop similar tools or resources for use in future AHRQ projects.

This evaluation will provide a better understanding of the strengths and weaknesses of the Health IT Literacy Guide and how to improve the Guide based on an environmental scan of relevant literature and Web sites, expert interviews, and focus groups with developers and purchasers of health IT. By implementing evaluation results, the health IT community will be better informed about effective ways to address health literacy, and AHRQ will have recommendations and guidance for offering more current and effective tools and resources to support health literacy through health IT.

1.3 Research Questions

To determine whether the Heath IT Literacy Guide is effective both in terms of the quality of its content and its accessibility by its intended audience—Web and software developers and purchasers of health IT products—in providing effective health IT interventions that are accessible to patients with limited health literacy, the following research questions were used in evaluating the Guide:

1. To what extent does the Health IT Literacy Guide aid developers in designing health IT applications that are accessible to adults with different levels of health literacy?
2. To what extent does the Health IT Literacy Guide aid purchasers in selecting health IT applications that are accessible to adults with different levels of health literacy?
3. In what ways can the Health IT Literacy Guide be improved or updated to be more timely, relevant, and useful to developers in designing health IT applications that are accessible to adults with different levels of health literacy?
4. In what ways can the Health IT Literacy Guide be improved or updated to be more timely, relevant, and useful to purchasers in selecting health IT applications that are accessible to adults with different levels of health literacy?

Return to Contents

1.4 Terms Used in this Report

Common terms and concepts used in this report are provided in Table 2.

Table 2. Common terms used in this report

Return to Contents

1.5 Background on the Guide

The Health IT Literacy Guide was one of several tools developed for the AHRQ Health IT Portfolio by the National Opinion Research Center (NORC) under an AHRQ contract. Its purpose was to provide developers and purchasers of health IT with information about limited literacy populations and the principles of accessible health IT design (Appendix A). The preface of the Guide states that health IT developers are the intended audience because they often have little knowledge of populations with limited health literacy, or technical standards and aspects of accessible health IT design. The Guide is also intended to be used by purchasers of health IT for evaluating health IT products and for directing and evaluating contracted development work. Table 3 lists the sections of the Guide along with each section's objective.

Table 3. Sections of the Health IT Literacy Guide

Section I of the Guide includes (1) definitions of literacy terms, such as general literacy, health literacy, and eHealth literacy as shown in more detail in Table 4; (2) examples of a few health IT applications successfully used by populations with limited literacy; and (3) discussion of the importance to developers and purchasers of addressing the needs of limited literacy users. The introduction identifies limited health literacy and limited accessibility as important barriers faced by consumers seeking to use health IT as an aid in understanding health information, performing self-care, decision making, connecting with providers, communicating with others (such as patients), and storing and accessing personal health records.

Table 4. Definitions from the Health IT Literacy Guide

Section II of the Guide provides a brief description of some of the advantages offered by health IT for limited literacy users, including four types of health IT used by consumers (i.e., Internet Web sites, touchscreen kiosks, personal wireless devices, and home monitoring devices). The table included on page 3 of the Guide describes the four types of health IT, and is reproduced in Table 5, below.

Table 5. Types of health IT used by consumers (from the Health IT Literacy Guide, page 3).

Section III of the Guide describes the importance of universal design, accessibility guidelines for all health IT, and guidelines for specific types of health IT. The definition for "accessibility" as defined in the Guide is included in Table 4. The authors of the Guide note that "some call this usability." Section III of the Guide describes basic universal design principles, plain and clear language, content relevance and format, and content iterative testing and revision. It also makes specific recommendations for the four types of health IT identified in Section II.

Section IV of the Guide provides two pages of resources: articles and Web sites that elaborate on the topics covered in the Guide.

The appendix for the Guide presents the principles from Section III in the form of a checklist with yes/no checkboxes next to the text descriptions drawn from the principles for general health IT and for specific health IT. Figure 1 presents a screenshot of a portion of this checklist.

Figure 1. A portion of the checklist found in the Health IT Literacy Guide

According to the preface of the Guide, the process used to develop the Guide included "a review of the IT and health IT literature; examination of products' and organizations' Web sites; and discussions with developers and purchasers of health IT as well as researchers involved in the evaluation of health IT for limited-literacy populations." In the preface, the authors also noted that more research was needed to address accessibility standards for emerging technologies, and that the initial checklist they compiled would need updates over time.

Return to Contents

1.6 State of Health Literacy and Health IT Literacy

The concept that health consumers need to be health literate—defined as the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions—has been evolving since its introduction in the 1970s (Sørensen et al., 2012). It is now widely accepted that low health literacy is associated with many negative health outcomes, including low levels of health knowledge, increased incidence of chronic illness, and less use of preventive services (Berkman et al., 2011).

Addressing health literacy is a crucial part of any attempt to provide guidance to developers and purchasers of consumer health IT products and applications. However, this is a challenge because health literacy is a concept that is in a state of constant flux. For example, there is no single accepted definition of health literacy and even less consensus on the conceptual models underpinning it.

1.6.1 Definitions

There are at least 25 different definitions of health literacy in the scientific literature; three of the most widely used are presented in Table 6.

Table 6. Varied definitions of health literacy

1.6.2 Theoretical Constructs

As with definitions, there are many different constructs that underpin health literacy. Table 7 describes several of the conceptual frameworks.

Table 7. Health literacy conceptual frameworks

Return to Contents
Proceed to Next Section