Session 0: Overcoming Barriers to Implementation

Moderator: Jean Slutsky, P.A., M.S.P.H., AHCPR 
Panel members: Catherine Borbas, Ph.D., M.P.H., Healthcare Education and Research Foundation (HERF), Minneapolis; and David E. Kanouse, Ph.D, RAND Corporation

Implementing Changes in Health Care Organizations: Lessons Learned and Future Challenges— Catherine Borbas, Ph.D., M.P.H.

The Healthcare Education and Research Foundation (HERF) assists medical service providers and purchasers in integrating quality improvement methods with procedures for accountability. HERF's model to change behavior includes collecting research information, providing clinicians with an awareness of this information, gaining clinician acceptance of the need for modification, and providing a framework for adoption of change. Two primary assumptions have driven this process: (1) with clinically credible information, clinicians will modify their behavior to improve care, and (2) if physicians are involved in the developmental process, they will move quickly to alter deficient systems.

HERF evaluated the effectiveness of approaches to modify hospital, health plan, and clinician behavior as part of an implementation strategy study conducted to improve quality of care for acute myocardial infarction (AMI). Thirty-seven sites participated, and standardized data were collected on the use of four target drugs: thrombolytic agents, aspirin, beta blockers, and lidocaine. Evidence-based guidelines (from the American College of Cardiology) support the effectiveness of the first three and recommend prophylactic avoidance of the latter. HERF surveyed current knowledge and practice patterns, identified opinion leaders (who tend to be early adopters of improved methods), and compared aggregate performance data with results of the knowledge survey.

The intervention study found improved rates of aspirin and beta blocker use. However, despite a good knowledge base on appropriate use of these drugs, inconsistency remained between knowledge and practice. For thrombolytics, clinicians were reluctant to take an action that might be associated with complications such as stroke or with negative experience or psychosocial barriers. Multiple, informal interventions are often necessary before required changes are implemented. Organizations may be more amenable to systematic change than individuals but must also overcome barriers such as resource constraints, competitive factors, and resistance to protocol changes. 

To achieve best results, key hospital audiences and influencers in all clinical stakeholder groups need to be identified, coordination undertaken in iterative implementation efforts with all stakeholders including health plans, and resources made available to assist clinicians in applying guideline recommendations.

Overcoming Barriers to Implementation—David E. Kanouse, Ph.D.

The Women's Health and Hysterectomy Project is a study on guideline implementation for hysterectomy. It evaluated strategies for implementing practice recommendations, testing social influence and state-of-the-art educational intervention, and evaluating appropriateness of care before and after intervention. Hysterectomy was chosen because of the availability of recent assessments and practice recommendations, because appropriateness could be easily measured against defined criteria, and because the criteria and recommendations could be readily updated.

The study employed RAND appropriateness methodology, which can evaluate practice changes using chart review, determine patient profiles, and provide good feedback to practitioners. The study goal was to transform appropriateness ratings into clinically useful recommendations that clinicians would "buy into" and effectively use. The targeted practitioner group consisted of capitated physician practices.

Lessons learned from the study fell into broad categories:

1. In the translation of appropriateness ratings into recommendations, indicators were structured along a sequential pattern, like the actual step-by-step practices of clinicians. Appropriateness ratings showed that alternative diagnostic tests or treatments should be considered prior to procedures such as hysterectomy.
2. The recruitment strategy failed for community hospitals but succeeded with capitated physician groups.
3. The intervention strategy concentrated on selecting local, influential physicians on the basis of their credibility, respect from their colleagues, and involvement in quality improvement activities. The study provided these clinicians with training and output from national panel recommendations, tailored recommendations to their own practice settings, engaged them in participatory development, and provided critical support for followup activities.
4. A local focus group was convened to review the educational aspects of conveying recent medical advances and cutting edge technologies. This local review and modification of panel recommendations created a sense of ownership and buy-in.
5. Barriers encountered included differing perspectives on the necessity of change, the need for local participation, the need to link recommendations with the clinical decisionmaking process (such as helping to develop checklists for clinicians), and the necessity of integrating appropriateness ratings with real-life patient management scenarios.
6. As to the tools used, the study demonstrated that (a) guidelines for implementation can be developed if it is recognized that much work and many unanticipated steps may be required, (b) appropriateness ratings can be a good starting point, and (c) selection for study of a procedure with weak scientific evidence (like hysterectomy), while a complication, also more accurately reflects the real world, where most procedures are cloaked in uncertainty.

The study concluded that to overcome barriers to implementation, practitioners must view practice guidelines as clinically useful, have ownership of responsibility, and receive the opportunity to modify draft reports before buying in to final recommendations. This is facilitated by providing credible guidelines linked to an existing structure and supported by substantive scientific evidence. Recommendations must also be relevant and targeted to program management using epidemiologic or descriptive data. A lesson of the study was that many unnecessary hysterectomies are being performed; with appropriate emphasis on the most prevalent and important scenarios, the potential for positive change is enhanced. 

Return to Contents

Session P: Implementing Guidelines in Specific Populations and Settings

Moderator: Carolyn M. Clancy, M.D., AHCPR 
Panel members: E. Andrew Balas, M.D., Ph.D., University of Missouri; Rita A. Frantz, Ph.D., R.N., F.A.A.N., University of Iowa, and George C. Xakellis, Jr., M.D., M.B.A., Ford Motor Company; and Eric W. Boberg, Ph.D., University of Wisconsin

Evidence-based Implementation of Clinical Practice Guidelines—E. Andrew Balas, M.D., Ph.D.

The main quality concerns in health care are inconsistency in practice patterns, noncompliance with widely accepted standards, lack of substantiating evidence on benefits of a procedure or practice, and excess cost that threatens business survival. In addition, substantive medical research results often are not reflected in practice until many years after publication in the literature.

To support the goal of transforming medical management through guideline recommendations, 6 years ago AHCPR funded a registry of medical interventions, the Columbia Registry of Information and Process Management Trials. The number of such trials is increasing by 50 percent annually, with trial reports published on computerized information interventions, noncomputerized information interventions, and utilization management interventions.

In approximately 100 studies that tested computerized interventions, the most studied intervention, and the most successful, was the physician reminder message (or prompt) that arrives at the time of decisionmaking. Other physician interventions studied included computerized treatment planners, feedback systems, and diagnostic systems. Aside from physician prompting, effective information services in health care include patient reminders, information feedback, patient education (which can be computerized), and treatment design. 

Two common types of information intervention are "profiling" and the "clinical direct report." Profiling, or peer comparison feedback, is imposed on 50 percent of U.S. physicians but has been shown to have a negligible contribution to improved quality of health care and to be costly as well. The "clinical direct report," however, is an effective way to channel published information to practitioners who need it. In simple tabular format, it compares the targeted medical center with other centers in its State and with all centers in the United States in regard to utilization of a particular procedure. Dissemination of such reports was shown in one study to improve usage of a particular procedure by 14 percent.

Implementing Pressure Ulcer Guidelines in Long-Term Care—Rita A. Frantz, Ph.D., R.N., F.A.A.N., and George C. Xakellis, Jr., M.D., M.B.A.

A study was performed at the University of Iowa to evaluate the cost-effectiveness of implementing a guideline for pressure ulcer management. The research was based on the Weinstein and Russell models for analyzing cost effectiveness. "Health benefits" were defined and measured as the number of years of added healthy life. Costs of prevention also were analyzed, as well as costs of treatment necessitated because prevention failed.

The setting for the study was a 77-bed, long-term care facility in the Midwest. The study subjects were residents of the facility who were free of pressure ulcers. Before the AHCPR guideline-based prevention protocols were carried out at the facility, no systematic prevention measures had been in place. The protocol, implemented over a 6-month period, consisted of administrative support, services of a skin care consultant, development of institutional policies and procedures, and intensive, mandatory staff education. Staff performance was monitored and feedback provided; performance evaluations were linked to compliance with the skin care protocol. Data were collected for 6 months before and after implementation of the protocol.

Demographic data were collected as well as risk factors and prevention and treatment activities. Prevention activities at the facility consisted of repositioning and use of pressure-reducing devices. Ulcer treatments included use of wound care supplies, laboratory tests, x-ray procedures, and systemic antibiotics. Costs were calculated in relation to (1) direct care (cost of nursing time and supplies and equipment), (2) indirect care (staff education and personnel and clinical management), and (3) nursing administration overhead.

The pressure ulcer protocols worked both clinically and financially. With the protocol, the ulcers in the facility were almost completely eliminated. Implementation of better practice did not cost any more; rather, resources were reallocated. Before the protocol was instituted, no expenditures had been made for prevention. After the protocol, $11,877 had been spent on preventive measures, most related to repositioning the patients. The higher the risk assessment for a patient, the more expended on interventions. Treatment costs before the protocol implementation amounted to $12,774, and after implementation to $914. The amount spent in the study on prevention roughly equaled what had earlier been spent on treatment, but with an increase in the number of ulcer-free days. Cost-effectiveness analysis showed that the cost per ulcer-free day was $1.60. 

The study showed that: pressure ulcer prevention programs can be implemented with little additional cost; highly technical equipment is not necessary in long-term care to achieve desired outcomes; a successful intervention program increases the intensity of interventions as patient risk increases; and prevention programs require staff training and ongoing supervision to be successful.

Comprehensive Health Enhancement Support System (CHESS): In-Home, Computer-Based Patient Education and Support—Eric W. Boberg, Ph.D.

The Comprehensive Health Enhancement Support System (CHESS) was conceived as a computer intervention when persons faced with a health care crisis need to acquire complex information confidentially and quickly. The goals of CHESS are to improve quality of life, reduce cost of care, improve adherence to treatment, and increase the consumer role in health care. An in-home or community-centered computer-based program was thought to be an excellent tool for these purposes. The system developers searched the literature for theoretical models of crisis and assembled a group of tools for use in a health care crisis. Expert systems, information databases, and computer-mediated communications (support groups) were developed. Needs assessment was conducted among the intended users of each CHESS module. Some of the modules currently in the system address HIV, breast cancer, heart disease, and caregivers of Alzheimer's patients. Emphasis was placed on interfacing with persons with little previous exposure to computers. The reading level of the project is at about the eighth grade.

Services provided comprise information, communication, and analysis. Information services include databases, questions and answers, full-text articles, consumer guides, a referral directory, personal stories, and a dictionary of words and phrases. Communications services include closed on-line support groups with moderators and "Ask an Expert." Analysis services for the HIV module, for example, include risk of transmission, symptom assessment, stress assessments, decision aids, action plans, and personal profiles. This information can later be given to the health care provider.

CHESS has been subjected to evaluation research and randomized controlled trials to assess the effectiveness and outcomes of the system. Research results have consistently shown better quality of life for CHESS users.

The typical user accesses the program about once a day over a 4-month period, often at night. Women, elderly, minorities, and less-educated persons all use the system heavily. CHESS plans to include topics for those health problems that result in 90 percent of cost expenditures, a total of 40 to 50 topics. Developing a new CHESS module can take from 6 to 18 months and costs from $75,000 to $300,000 depending on complexity of the subject and how much research is already available. A patient needs assessment starts each project. 

CHESS is being converted for dissemination over the Internet, and all new modules will be developed for the Internet. Providing the system over the Internet raises issues about cost for Internet service, dependence on telephone lines, linkage to other Web sites, sophistication (naive or expert) of users, and research data collection (easier on the current system).

Return to Contents

Data Needs for Quality Measurement and Improvement

David Blumenthal, M.D., Massachusetts General Hospital

Useful questions for quality measurement and management in the next 5 to 10 years, with significant implications for policy and research, includes: How will definitions of quality change over time? How will approaches to quality measurement and management change? How will changes in measurement and management affect infrastructure needs? What are the policy implications, especially for equity and cost? What are the implications for our research agenda?

Quality varies with roles in the health care system and with the process of health care production. The availability and affordability of medical information to patients, health care providers, and provider organizations through new information technology will permit patients to self-manage illness and to audit a doctor's performance. Also, nonprofessional health care personnel will be able to manage illness outside of health care settings. As the process of preserving health and managing chronic and subacute illness becomes more decentralized and less professionalized, important aspects of quality will be how well organizations and individuals confront self-management issues.

The definition of quality of a health care organization will change in the interim. Structural measures will include information system capabilities, the training and availability of certified nonmedical health care professionals, and new forms of physician certification. Process measures will include self-management, the generation and use of patient electronic feedback, and patient assessments of quality of care. Outcome measures will address reduced cycle time for development and validation of disease-specific self-reports.

These changes in measurement and management have implications for both infrastructure and policy. Information Systems (IS) and Research and Development (R&D) play a central role in sustaining and improving quality and in the diversification of personnel needs and training requirements. The associated short-term policy implications are continued consolidation in the health care industry, competitive advantages to large for-profit and nonprofit organizations, and disadvantages to the public sector and small non-profits. Longer term policy implications are disintermediation for educated consumers (e.g., decreased role for health plans in ensuring quality, direct purchaser-provider relationships) and inequities created by the ability to manage information and interact with information systems.

These postulated trends raise a number of questions, such as: What are the reality, extent, and importance of self-management? Specifically, which conditions lend themselves to self-management and how will the role of physicians change? How important will the psychological value of face-to-face provider-patient interaction be to the viability of telemedicine and Internet-based medicine? Will inequities increase as a result of the availability of information? If so, how will they be measured? What are the privacy concerns regarding the use of the Internet for exchange of personal health care information? In response to these questions, the presenter stated the following.

1. To improve the quality of information on the Internet, some degree of regulatory intervention will be necessary. There should be an industry rating for Internet medical information systems.
2. Patient adherence to treatment during self-management will depend on the availability of easy-to-use software that allows patients to enter information and receive personalized reports on how well controlled their illnesses are, with recommendations.
3. If Internet "push" technology that targets information to a user based on selected characteristics is used, then the interface with the patient must be transparent.
4. Capital and personnel requirements for these innovations will be in the private sector. The role of the Federal Government will be to make private sector discoveries more widely available to different groups, to basic research, and to areas of research ignored by the public sector.

Return to Contents

Health Information Privacy

Lawrence 0. Gostin, J.D., L.L.D. (Hon.), Georgetown University Law Center

The health information infrastructure is the framework that underlies the electronic collection, storage, use, and transmission of information supporting the essential functions of the public health care system. The infrastructure will be quite diverse: patient data may be captured in an electronic longitudinal record; by electronic card technologies, and by comprehensive health databases. This electronic collection of health data can be supported with reasonable assurance of protection of privacy. Future health care information infrastructures can affect individual and group privacy, human goods accrued from the collection of these data, current legal safeguards, and future proposals.

Some privacy risks are associated with the electronic automation of health care records. Security can be breached and different records can be matched to create a more comprehensive and identifiable record. Likewise, the proliferation of systemic users with authorized access to health information makes it impossible to adequately police and secure records to prevent fraudulent use. Lastly, there are instances of unauthorized but lawful access (e.g., the purchase and sale of health information). Nevertheless, electronic automation of health information provides benefits such as consumer choice among health plans, improved monitoring and effectiveness of services, better assessment of system performance, improved administrative efficiency, and safeguards for the public's health through better surveillance.

Despite legal safeguards, considerable inconsistencies exist in privacy protection throughout the United States. The privacy of government records is protected through the Privacy Act and the Freedom of Information Act, but both have substantial loopholes. Most States have statutes to help protect information collection by health departments but not by the private sector and restrictive statutes that only protect certain diseases. These variations among rules for different diseases thwart efficient collection and transmission of data.

There are fundamental problems with the way privacy is viewed. The confidentiality of the doctor-patient relationship implies that information remains solely with the doctor. Yet in the current age, information is also collected by laboratories, managed care organizations, public health departments, and other organizations This poses two theoretical problems. First, ethical and legal safeguards are based primarily on the holder of the record, but electronic records have no holder. Second, because the law is based on the doctor-patient relationship, only a breach of that relationship can evoke legal penalties. To protect the privacy of automated health care records, penalties must be attached to anyone who discloses information, not just the holder of the record. Some proposals for law reform on public health information privacy follow.

1. Data Protection Review. Establish an independent body to review data collection, use, privacy and security standards.
2. Data Collection Justification. Impose a burden on any collector of information to demonstrate its value for public health purposes.
3. Information for Subjects. Allow subjects to know the purpose, time frame, and access restrictions for the information collected on them—they are entitled to that information.
4. Fair Information Practices. No secret data systems exist—give patients access to information about themselves so they can correct and amend records and expunge unneeded information.
5. Secondary Uses of the Data. Require the consent of the individual to use information for a purpose beyond those originally intended.

Policy discussions and claims to the public should not be based on absolute privacy. There should be reasonable assurances that when personal information is collected, health care providers, health care plans, and Government and public health departments will treat the information with respect, store it in an orderly and secure manner, and disclose it only for important health-related purposes and in accordance with publicly accountable principles of fairness.

Return to Contents

Final Session: Evidence-based Practice—An International Perspective

Moderator: Steven H. Woolf, M.D., M.P.H. 
Panel members: Jeremy M. Grimshaw, M.B., C.H.C.B., M.R.C.G.P., Department of General Practice, University of Aberdeen; J. A. Muir Gray, M.D., Anglia & Oxford NHS Executive, UK; and Carmi Margolis, M.D., M.A., Ben Gurion University, Israel

Evidence-based Practice: An International Perspective—Steven H. Woolf, M.D., M.P.H. 

Work in evidence-based practice outside the United States includes: the Cochrane Collaboration in the UK; the technology assessment (TA) work being done in Canada, Thailand, and the Phillippines; the work in Scandinavia in utilizing evidence in the practice setting; and the production of practice guidelines in France and Holland. Collaboration between the United States and these European nations could bring new contributions to the field of evidence-based practice.

International Perspectives on Implementation Research—Jeremy M. Grimshaw, M.B., C.H.C.B., M.R.C.G.P. 

The traditional model of disseminating research findings to physicians by publishing in journals has been unsuccessful. A new approach, using systematic reviews and clinical practice guidelines, seems more promising. Collaboration efforts have begun and promise to decrease duplication of effort across groups and nations. Systematic reviews identify crucial studies and synthesize the findings to ease their application in practice. An increasing recognition of the failure of the traditional dissemination model has led to international efforts to improve the influence of evidence on clinical practice.

Developing Evidence-based Health Care in the United Kingdom—J. A. Muir Gray, M.D. 

The effort to produce evidence-based policy in the U.K. grew out of the concern that research studies, once published, quickly become outdated. To develop better evidence-based policy, systematic reviews must be used to filter out useless information and retain relevant facts, combining data when possible. The World Wide Web offers a viable solution to the problems of traditional dissemination by allowing physicians to quickly locate necessary information. For example, the Consumer Health Information Project (CHIP), a collaborative effort to put up the best and most accurate health information on the Web, represents one of the many new efforts to manage health knowledge. Ultimately, organizations must adapt their culture to encourage use and comfort with evidence-based practice, and individuals must develop skills in searching, appraising, and storing evidence.

Logic Methodology of Clinical Practice Guidelines—Carmi Margolis, M.D., M.A.

The algorithm, the logic structure of clinical practice guidelines, is a step-by-step approach for solving a problem. Decision points in guidelines are grounded in rationale based on scientific evidence. However, some guideline decisions do not rely solely on scientific evidence, and require expert opinion to fill in the gaps. Nonetheless, algorithms have been shown to yield highly accurate rates of diagnoses by technicians. They can serve as tools for comparing differences between competing guidelines or they can assess a guideline's reproducibility.

Return to Contents