Clinical Evidence: Creation, Assessment, and Implementation

Douglas B. Kamerow, M.D., M.P.H., Director,
Office of the Forum for Quality and Effectiveness in Health Care, AHCPR

Evidence has become a major force in medical and health care delivery today and is addressed by numerous publications on the subject. The AHCPR clinical practice guidelines have made an important contribution in the last few years. Evidence-based medicine is defined as the conscientious, explicit, and judicious use of current best data in making decisions about the care of individual patients.

Evidence-based practice originated in the 1950s with the advent of randomized controlled trials and their subsequent use in the systematic reviews that document evidence. In the 1970s, the Canadian Task Force on Periodic Health Exams devoted much effort to assessing the evidence on clinical preventive services. At the same time in England, the Oxford Database of Perinatal Trials was developed. In the 1980s, the United States followed the Canadian example by establishing the U.S. Preventive Services Task Force, which produced a well-regarded report. Medical organizations and the insurance industry became involved as well. Technology assessments were produced, and the technique of meta-analysis came into use. AHCPR began its outcomes research in the 1990s to address variations in care delivery and launched its clinical practice guidelines series. Technology assessment is now a worldwide collaborative activity. The Internet has become an important resource for dissemination of health information.

The need for evidence-based practice arose for several reasons, such as the following.

• The enormous volume of research and published material demands organization.
• Variations in service delivery require description and investigation.
• Pressure on health care resources, as more of the gross national product is spent on health care, has led to a demand for greater value for dollars spent.
• Competition of commercial forces for delivery of health care has led to a demand for value while preserving or improving the quality of care being delivered and allowing for patient preferences.

Evidence-based practice involves both individuals and systems. In the past, knowledge was transferred from expert to student, either at the bedside or through reading the writings of experts. The quality of the provider and the care depended on the individual abilities of the provider. In the newer model, multiple sources of information that account for patient outcomes and preferences are integrated with individual clinical expertise to form evidence-based practice. An important element to add is a system that facilitates this process and makes it mandatory.

AHCPR's 19 published guidelines have made a contribution both to the methodological progress of evidence-based medicine and to management of the health problems they address. AHCPR was not able to fully support implementation of its guidelines, however, because of limited resources. Its role in supporting evidence-based care was redefined to include the following functions:

• Keep relevant information in the public domain.
• Serve as an impartial, neutral broker.
• Encourage multidisciplinary input to all projects.
• Advocate for patient perspectives and needs.
• Protect special populations.

AHCPR' s newly mandated program encompasses three parts. The first part is a series of Evidence-based Practice Centers that will produce evidence reports and technology assessments on important topics. The reports will be more focused than the clinical practice guidelines and will be published and widely distributed. AHCPR's leadership role will be shared with professional societies and managed care organizations, and others, particularly in topic nomination, report review, and implementation. Topics to be addressed in evidence reports must be common conditions, significant for Medicare/Medicaid, costly, and uncertain in effectiveness, and the resulting evidence should be expected to improve the quality of health care. The evidence reports are intended for use in practice guidelines, hospital quality improvement programs, State policies for coverage of technology, and other purposes.

The second aspect of the new program is to evaluate and compare guidelines through the online National Guideline Clearinghouse™ in a public-private partnership. This clearinghouse is expected to be established in the near future. A final component of the program is "product research and evaluation," wherein the range of activities related to evidence, particularly development of evidence, implementation strategies, and quality of practice using implemented evidence, is assessed.

The advantages of the new program include improving the science base of guidelines through the , disseminating evidence nationally, decreasing duplication of effort, increasing uniformity and quality of products, and strengthening public/private partnerships.

Change in medical practice can be achieved through efforts in many areas. Systematic reviews and knowledge of practice patterns can help physicians see where they might modify their practices. The systems of care in which physicians work must be redesigned to include resources and tools that allow incorporation of evidence into practice. Change is further reinforced through incentives and feedback to physicians. Information systems can be used to update physicians' knowledge and skills and to deliver information to them at decisive times in patient care. Fear of liability hampers adherence to evidence-based practice, however, if, for example, patients request more procedures than are recommended by the evidence. Patients and doctors must work together to balance risks and benefits of treatments. Better systems are needed to support evidence-based practice, and competing incentives must be identified and discussed. Finally, "take-home messages" must be clearly extracted from complex and lengthy practice guidelines so that they can be moved into practice. 

Return to Contents 

Keynote Address

Evidence-based Medicine: What Have We Learned?

David M. Eddy, M.D., Ph.D., Kaiser Permanente, California

One of the perceived flaws in traditional medical literature is that it often lacks the necessary outcome information and supporting evidence to help physicians make sound preference decisions. Many physicians are unaware of available data within their own specialties.

A manual by the study team describes four types of guidelines: those based on (1) "global subjective judgment" (evidence not explicit but filtered through the judgment of the guideline developers), (2) evidence (analysis of evidence is explicit, but all other elements are subjective), (3) outcomes (explicit descriptions of probabilities of various outcomes), and (4) explicit preference of those receiving the given treatment. Most guidelines are based on global subjective judgment.

In the past 5 to 10 years, activity and interest in guidelines have changed the face of medicine. Concepts of effectiveness, outcomes management, continuous quality improvement (CQI), and total quality management (TQM) have been applied to medical practice. Organized medicine has embraced evidence-based guidelines, using mathematical modeling and outcomes. Foundations began to support research in the field.

The Agency for Health Care Policy and Research has been the leader of Government programs supporting health services research. The AHCPR guideline program is evidence- and outcomes-based, and it produced one of the best examples of a preference-based guideline. Allied disciplines have reinforced the field through decision analysis, technology assessment, clinical epidemiology, cost-effectiveness analysis, mathematical modeling, outcomes research, performance management, TQM, and CQI. Evidence-based medicine has become the standard for most professional organizations. 

Despite these advances, however, there is a continuing need to overcome additional challenges and ensure proper practice of evidence-based medicine. The use of meta-analysis for synthesizing evidence means that researchers should use either random-effects models or an analytical method that accounts for specific differences in studies. Medical practice should be based on quantitative information on outcomes; that is, the magnitude of all important outcomes is estimated to demonstrate how well a procedure works. Financial costs can then be balanced against outcome figures. Cost-effectiveness analysis is associated with methodological, political, and philosophical problems. The methodological problems include methods, availability of data, and variability in the kind of information such as the cost of a procedure. Despite these problems, cost-effectiveness analysis must be utilized as a practical decisionmaking tool because the alternative—to ignore cost—is unacceptable. Finally, there is the difficulty of finding an evidence base for many established, time-honored treatments. One proposal is to identify questionable old treatments—for which evidence is uncertain—and flag them with the statement that no evidence of effectiveness is currently available. Research could then be designed to demonstrate their effectiveness.

Medical Ethics in the New Medical Marketplace, or How I Learned to Stop Worrying and Love Managed Care

Mark R. Chassin, M.D., M.P.P., M.P.H.,
Mount Sinai School of Medicine

American health care is undergoing tumultuous change, and managed care has garnered most of the publicity. Most noteworthy, however, is the rapid increase in knowledge of medical efficacy and effectiveness and of basic science. Although 80 percent of health care still cannot be related to a strong scientific foundation, the percentage of care based on evidence continues to increase dramatically.

Given the great amount of evidence available, medical practice can no longer be based on opinion as in the past. Physicians must now remain current with the literature, which has increased tremendously over the past two decades. The weight of the knowledge and the speed at which it is increasing have profound implications for every aspect of health care: education and training of physicians, practice of individual physicians, organization financing, and delivery of care. To remain current, doctors will depend on organizations and systems of care to provide them with timely summaries of the evidence.

This growth in knowledge of effectiveness brings with it a moral imperative to improve access to health care for the uninsured. The United States is the only industrialized country that does not provide universal access to health care. Lack of insurance—and consequently, access to health care—has been shown to result in a much greater mortality rate throughout the population, indicating that the market has failed this group.

Managed care has highlighted ethical issues in the doctor-patient relationship. Financial incentives now reward physicians for providing fewer services. Yet no link has been found between financial incentives and improved quality; rather fee-for-service is associated with overuse and capitated payment of physicians with underuse. Such conflicts between financial incentives and quality may jeopardize patients' health, and an intensive effort must be made to measure and improve quality of care.

Similarly, the quest to control cost has led to rationing or withholding of effective care because of its expense. Unwillingness of insurers to pay for health care scientifically proven to be effective results in contentious decisionmaking in the future. With the ongoing development of new treatments and technologies, health care costs will continue to escalate, and will remain a critical component of health care policy.

Many people argue that it is unethical to consider rationing as a cost-containment strategy. An ethical strategy would be guided by principles of quality improvement—quality-driven cost containment. The three main quality problems can be characterized as overuse (care that causes more harm than benefit), underuse (failure to give care that would produce a good outcome), and misuse (appropriate care given poorly). Improving efficiency would improve outcomes and reduce costs, which in turn could enable uninsured individuals gain access to health care. 

Return to Contents

Session A: ERISA: Interplay of State and Federal Responsibilities in Health Care

Moderator: Alice G. Gosfield, J.D., Alice G. Gosfield and Associates, P.C.
Panel members: G. William Scott, J.D., and Marc I. Machiz, J.D., U.S. Department of Labor; Patricia A. Butler, J.D., Dr.P.H., Health Policy Analyst; Joel Michaels, J.D., Michaels, Wishner and Bonner; and Robert Berenson, M.D., Georgetown University School of Medicine

Interplay of State and Federal Responsibilities in Health Care—Alice G. Gosfield, J.D.

The Supreme Court has affirmed that the administration of health benefits by self-funded employers is governed by the Employee Retirement Income Security Act of 1974 (ERISA), and, therefore, claims arising from benefits disputes should be resolved in Federal courts. However, questions remain as to how broadly the preemption clause should be interpreted and whether and under what circumstances ERISA, under Federal governance, preempts patient rights to seek remedies under State tort laws. 

At issue is whether Congress intended the ERISA statute to preempt States' broader regulation of health care quality, which for medical malpractice has traditionally been through the State-based tort system. It is often unclear whether a particular claim should be considered an ERISA benefits dispute or a traditional medical malpractice claim, based on allegations of poor-quality care. The parties involved are employees covered under employer-sponsored health plans, managed care organizations, health care providers regulators, and the legal establishment. Inequities may result from ERISA preemption, such as patient inability in some cases to recover damages even for avoidable negative outcomes.

ERISA Preemption of Medical Malpractice and Related Remedies Against Managed Care Entities—G. William Scott, J.D., and Marc I. Machiz, J.D.

When ERISA was passed in 1975, it was intended to cover pension plans. Health care plans covered by an employer were an afterthought and were included at the last moment. As stated, ERISA's general preemption clause, section 514(a), preempts "any and all State laws insofar as they. . .relate to any employee benefit plan." For the last several years, the U.S. Department of Labor (DOL) has vigorously sought to deter an overly broad interpretation of ERISA's preemption provision, which defeats State tort negligence and malpractice actions brought by patients against managed care entities, particularly health maintenance organizations (HMOs). The DOL has written several amicus curiae briefs in U.S. Courts of Appeal, in District Courts, and in State trial courts.

The ERISA plan is the employer's effort to provide medical care to employees by purchasing coverage in a health plan, such as an HMO, as a benefit of employment. The HMO is not an ERISA plan but is a service provider to the employer's ERISA plan, with three functions: arranging for and providing medical care, acting as plan administrator in deciding covered benefits under the employer's ERISA plan, and acting as insurer. To the extent that State laws regulate the business of insurance, HMOs are subject to such laws because ERISA exempts the business of insurance explicitly from preemption. The types of cases brought by patients have fallen into three categories: doctor malpractice; claims that the HMO is directly liable for its own acts of negligence or malpractice in carrying out medical care; and claims for denial of claims for benefits, when such denial resembles malpractice. In the first case, when the doctors are employees of the HMO (staff model), the employer (HMO) is liable for acts of its employees. When doctors are independent contractors, the doctrine of ostensible agency may apply if a reasonable person would identify the HMO as the provider of medical care (e.g., via display of posters). In such cases, the HMO may be vicariously liable for doctor malpractice, but current legal opinion is split on this issue.

The second example involves acts of negligence by the HMO such as negligent HMO hiring or monitoring of physicians, a negligent specialist referral system imposed on primary care physicians, or a negligent emergency care process. Currently, many State tort claims relate to incentive fee structures that HMOs use to reduce referrals to specialists. 

The third category involves denial of claims of benefits that reasonably resemble malpractice. Such an instance occurred when a pregnant women (whose fetus did not survive) was denied coverage to enter the hospital for fetal monitoring by the health plan Utilization Review Board on the basis that it was not medically necessary despite the primary physician's recommendation to do so. The court held it was a benefits decision and was preempted by ERISA (thus denying her claim any remedy for damages in State court). It is this absence of remedy that is driving the preemption controversy and forms the basis of pending bills in Congress.

ERISA Implications for State Activities in Health Care Quality—Patricia A. Butler, J.D., Dr.P.H.

Federal regulators view much of the preemption issue as one of consumer protection for health plan enrollees, whereas the States' perspective of their authority to regulate both health care providers and insurers derives more from their police power role, reserved for them under the Constitution, to protect public health and safety. States have traditionally set standards for health insurer solvency, marketing, benefits, and grievance resolution. As health coverage is increasingly provided through managed plans, however, States have undertaken greater regulation of managed care activities, such as prohibition of "gag" clauses, requirement for "any willing provider" laws, requirements for quality assurance systems, accessibility and availability of providers to enrollees, and other aspects of health plan quality. ERISA preserves the States' authority to regulate insurance via the "savings" clause, which is the major exception to preemption. Although States can regulate insurers by requiring risk pooling, imposing taxes, or mandating benefits, they cannot directly regulate self-insured employee health plans, which cover 40 percent of insured working Americans. States can, however, wield considerable power, and courts have variously interpreted the definition of business of insurance as it relates to the preemption exception. 

Circuit Courts have differed on whether "any willing provider" laws are insurance regulations, with the more recent rulings saying they are not. Opinion is also split regarding "provider mandates" (e.g., for dentists and optometrists). In rulings opposed by the DOL and baffling to many analysts, a growing number of courts are holding that HMOs are not insurers because they are outside traditional insurance carriers. Future litigation is likely to center on State laws regulating managed care, especially over insurer-provider relationships such as any willing provider laws, restrictions on incentive payments, and review, selection, and termination of physicians. Such laws are vulnerable if challenged and are likely to be ruled subject to preemption. However, laws that resemble benefits mandates are most defensible (regardless of their policy merits) and include definitions of emergency care and hospital length of stay laws. Regardless of the ERISA preemption debate, States have tools other than regulating insurers and HMOs to protect consumers, such as provider licensure to influence the conduct of individual practitioners and institutions, collection and public disclosure of information, and their role as purchasers of health care services.

Interplay of State and Federal Responsibilities in Health Care—Joel Michaels, J.D.

Managed care has actually succeeded in many of its goals, for example in flattening out premium rates and in providing benefits to a broader population. However, despite its overall success, managed care has led to isolated instances of inequity such as in ERISA preemption cases where no damages remedy is available. The problem lies with the unnecessarily narrow judicial interpretations of ERISA's intent (which are now forcing Congress to intervene) and with occasional mismanagement of health plan utilization review systems. It is important to work within the framework of managed care and improve it where necessary. Where errors have led to harm, courts could and should find a way to supply a remedy. 

With regard to malpractice and liability of individuals or health plans, the best tactic for plaintiffs is not to focus on negligent benefit administration, selection of providers, or design of physician incentive plans, but rather to rely on the theory of ostensible agency, where the provider is claimed to be an agent of the plan (evidenced by posters or similar items) and is then vicariously liable for physician conduct, regardless of due care exercised by the plan. Health plans are "immunized" from elements they do control, such as utilization review boards and benefits administration, but are being held liable for elements they may not control. The principle of uniform administration, which was and remains the key strength of ERISA, is thus being eroded.

Enterprise Liability: A Reappraisal—Robert Berenson, M.D.

In common law, development of vicarious liability is based on medieval legal principles of "master" responsibility for servant actions. The rationale for employer liability includes the theories that the employer benefits from the employee and so should pay for employee error, businesses are better able to compensate injured plaintiffs (due to "deep pockets"), and employer liability constitutes a more effective deterrent for negligence than does employee liability and protects a greater number of employees.

In the past, this concept of employer liability was not applied to independent contractors. In 1991, a rationale was proposed based on improving efficiency by spreading disproportionate malpractice risk among specialties and by promoting settlements by hospital defendants. This approach was consistent with Total Quality Management (TQM) principles, because organizations are better equipped to reduce errors than are individuals. The difficulty with this approach was that hospitals had little control over many activities that contribute to errors because practitioners often had multiple hospital affiliations, diagnoses often occurred outside the hospital, and physicians resented being treated as employees. The 1993 Clinton health care reform approach attempted (unsuccessfully) to improve the concept by anointing health plans and not hospitals as the liable enterprise and by emphasizing quality rather than efficiency. There would now, in theory, be countervailing pressure on utilization review criteria and cost containment incentives to physicians. This theory was based, however, on the faulty assumption that health plans would differentiate themselves by the quality of their physicians and hospitals. Thus, economic credentialing (use of low-cost, lesser quality physicians) would be reduced. It was further assumed that, together, health plans and physicians would create quality-based practice guidelines and risk management programs. These assumptions collapsed when physicians entered into nonexclusive relationships with many health plans, and no regulatory framework was developed to encourage closer physician-health plan planning for quality controls. The current reality is that, regardless of previous intentions, enterprise liability is developing not at the health plan level but rather at the capitated or integrated delivery system level.

Return to Contents
Proceed to Next Section