Appendix B: Summaries of PFQ Grantee Activities (continued)

PFQ Grant Summary: Accelerating TRIP in a Practice-Based Research Network

Lead Organization: Physician Micro Systems, Inc. (PMSI)
Partner Team: Practice Partner Research Network (PPRNet), Medical University of South Carolina (MUSC)
Title: Accelerating TRIP in a Practice-Based Research Network
Topic Area: Improved primary care physician adherence to practice guidelines in eight clinical areas
Principal Investigators: Steven M. Ornstein, MD, Associate Professor, Family Medicine, MUSC
AHRQ Project Officer: Margaret Coopey
Total Cumulative Award: $1,294,555
Funding Period: 9/2002-9/2006
Project Status: Received no-cost extension until March 2007 (This information was provided by an AHRQ Grants Management Office report, October 23, 2006. If there was a discrepancy between information provided by the principal investigator (PI) and the report, we presented the end-date provided by the Grants Management report.)

1. Project Description

Goals.  This project sought to improve guideline adherence for 70+ indicators in eight clinical areas (heart disease and stroke, diabetes mellitus, cancer screening, immunizations, respiratory disease/infectious disease, mental health and substance abuse, nutrition and obesity, and drug prescribing for the elderly) by using an electronic medical record (EMR) in 100+ community-based primary care practices across the United States and by expanding PPRNet's multimethod approach to quality improvement.  Over the four-year project period, the project planned to 1) expand the number of practices participating in PPRNet from 40 to 100; 2) increase the number and diversity of clinical practice guidelines tracked in the PPRNet practice reports from 22 to 73; and 3) disseminate the PPRNet-TRIP (Translating Research into Practice) model of quality improvement through performance reports, site visits, and network meetings.  (This last effort was funded by a previous AHRQ TRIP II grant.)

Activities and Progress. PPRNet, a national consortium of primary health care providers and academic researchers from three universities, was formed in 1995 as a joint effort between PMSI, MUSC, and interested primary care practices.  Each PPRNet practice is equipped with Practice Partner Patient Records, the EMR computerized system.  Practices collect data on clinical guidelines outlined by PPRNet.  Data are extracted quarterly from each practice and sent to PMSI electronically or on diskettes, and PPRNet staff generate the quarterly reports.  Prior to receiving the PFQ grant, PPRNet produced quarterly performance reports on 22 clinical indicators for their 40 members.  With PFQ funding, PPRNet expanded activities to include site visits in which MUSC staff and/or consultants from University of Southern California (USC) or University of Virginia (UVA) work with practices to improve guideline adherence, and annual network meetings where PPRNet members meet in person to discuss best practices and share lessons learned. 

In year 1, PPRNet membership increased from 40 primary care practices to 70 practices.  PPRNet held its first annual network meeting in Seattle; 22 of the participating practices attended this meeting. In year 2, PPRNet membership increased to 78 participating primary care practices, 30 of which attended the annual network meeting in Seattle.  In addition, the number of clinical practice guidelines tracked through the EMR increased from the initial 22 to 75, exceeding the project's goal.  Site visits also began in year 2 of the program. In typical site visits, PPRNet staff or consultants visited practices and met with the entire practice team in a large group session for approximately half a day.  Focusing on the practices' quarterly report results, these sessions highlighted successful practice improvements and explored opportunities for future improvements.   The PI and team conducted 68 site visits throughout the second year of the grant.

In year 3, PPRNet membership increased to 101 primary care practices, exceeding this project's recruitment goal. Forty-five primary care practices attended the annual network meeting in Seattle. The project increased the number of clinical guidelines tracked to 84 and added three summary performance indicators.  Site visits continued in years 3 and 4; project staff conducted an additional 79 site visits during the third year of the grant.  All site visits were expected to be completed by July 1, 2006, but information on year 4 performance was not yet available when this summary was written.

2. Partnership Structure/Function

The lead on project activities for this grant is MUSC, where the PI and his staff, who provide overall leadership on this project, are located. The grantee, however, is PMSI, the EMR software company. PMSI's primary role is to administer grant money and to provide technical assistance to the participating practices.  PMSI also provides PPRNet with the names of new clients to use for their recruitment efforts. The partners' roles are summarized in Table 1.

MUSC staff recruit new practices to participate in PPRNet activities, generate quarterly performance reports for practices, conduct site visits, and hold annual meetings for PPRNet members. Consultants from USC and UVA assist MUSC in designing, implementing, and evaluating projects, as well as in conducting site visits at participating practices.

The PPRNet participating practices are responsible for collecting and submitting clinical data on indicators to PPRNet.  Practices participating in PPRNet receive quarterly performance reports, host site visits, and attend annual meetings.

A listserv connects the PI and members of PPRNet.  The PI and PPRNet members share via email information and/or ideas on practice improvements, data access and reporting methods,,EMR changes, etc.  For computer and/or software issues, the PPRNet members contact PMSI representatives directly for assistance. Once a year, PPRNet holds an annual in-person meeting to discuss lessons learned and share best practices.

Table 1. Major Partner Organizations and Roles in the Project

	Organization	Role in Project
Lead Organization (grant recipient)	Physician Micro Systems, Inc.	Administers grant money Develops, maintains, and updates the software program that extracts the data, and coordinates data extraction from participating sites Provides TA for practices that have problems with the software program Provides names of new clients to PPRNet for recruitment into program Maintains electronic discussion list and website for user support Helps host annual network meetings in conjunction with user group meetings
	PPRNet (MUSC, location of PI Steven Ornstein)	Provides overall project leadership Generates reports for participating practices Conducts site visits Leads annual meetings Recruits new practices into PPRNet Designs, implements, and evaluates projects
Key Collaborators	Consultants at USC Keck School of Medicine and UVA College of Medicine	Work with MUSC staff to design, implement, and evaluate projects Conduct site visits
Target Organizations	100+ participating practices from 35+ states; practices range in size from solo nurse practitioners to 10+ clinicians	Collect data on indicators Submit data to PPRNet Participate in PPRNet activities (practice reports, site visits, annual meetings)

3. Project Evaluation and Outcomes/Results

To examine the overall impact of the intervention, PPRNet developed a summary measure incorporating data from each patient within each practice.  Called the Summary Quality Index (SQUID™), this measure calculates the percentage of processes and outcomes that are up to date or under control for a given patient and/or for a given practice.  Across all practices, the summary measure rose from 25.0 percent at the beginning of the intervention (September 2002) to 30.3 percent at the end of year 2 (September 2004), a finding that is clinically and statistically significant.

In addition, the project implemented a summary indicator for diabetes care, termed the Diabetes Summary Quality Index (DM-SQUID™). As of January 1, 2004, the mean DM-SQUID among 72 practices with a total of 22,219 patients was 50.2 percent; as of August 1, 2005, the mean DM-SQUID among 68 practices with a total of 24,429 patients was 58.3 percent.  Among the 66 practices with complete data at both time periods, the mean change in the DM-SQUID was 7.8 percent.  Significant improvements occurred for 12 of the 13 individual measures.  In a mixed linear regression model, practices having a higher proportion of male patients had higher DM-SQUID scores, and practices that attended the two-day 2004 PPRNet network meeting had greater improvements in the DM-SQUID than those that did not; previous experience with PPRNet TRIP research, the hosting of practice site visits, and specialty and practice size were not associated with extent of improvement.

PPRNet conducted a more complete analysis at the end of the program (June 30, 2006).  Preliminary analysis suggests approximately 10 percent improvement in performance indicators.  The evaluation component of the project will also include an in-depth case study of 10 PPRNet practices, a compendium of specific improvement approaches adopted by participating practices, and a final survey of all participating practices regarding the value of the project and its affect on the way they organized and ran their practices.

4. Major Products

• Presentations about the project at the 2003, 2004, and 2005 North American Primary Care Research group meetings; 2004 World Conference of Family Doctors; 2004 AHRQ conference, "Advancing Excellence from Discovery to Delivery"; and two 2005 Medical Records Institute meetings.
• Miller, P.M., S.M. Ornstein, P.J. Nietert, and R.F. Anton, "Self-Report and Biomarker Alcohol Screening by Primary Care Physicians: The Need to Translate Research into Guidelines and Practice."  Alcohol and Alcoholism, vol. 39, no. 4, 2004, pp. 325-28.
• White, M. "Taking it Slow: Implementing an EMR." Washington Family Physician, vol. 32, no. 2, 2005, p. 20.
• Nietert P.J., A.M. Wessell, C. Feifer, and S.M. Ornstein. "The Effect of Terminal Digit Preference on Blood Pressure Measurement and Treatment in Primary Care," American Journal of Hypertension, vol. 19, 2006, pp.147-152. 
• C. Feifer, S.M. Ornstein, R.G. Jenkins, A. Wessell, S.T. Corley, L.S. Nemeth, L. Roylance,
• P.J. Nietert, H. Liszka. "The Logic Behind an Intervention to Improve Adherence to Clinical Practice Guidelines in a Nationwide Network of Primary Care Practices," Evaluation and the Health Professions, vol. 29, no. 1, 2006, pp. 65-88.
• Six additional manuscripts currently being developed.

5. Potential for Sustainability/Expansion after PFQ Grant Ends

PPRNet has received additional grants (focusing on alcohol and cancer) to continue some of its activities.  PPRNet will likely continue to generate reports for practices that continue to participate in its research activities.  Practices that choose not to participate in the research aspect of PPRNet may need to pay to continue to receive the quarterly performance reports.  PPRNet plans to continue to expand its network of primary care practices.  Its goal is to grow by 25-50 practices per year.  At least four additional related activities have developed from this project:

• Dr. Peter Miller and Dr. Raymond Anton, nationally recognized alcohol researchers at MUSC, have worked with project investigators to extend the alcohol research component of the project.  During the summer of 2003, they conducted a survey of PPRNet primary care physicians about their alcohol and biomarker screening practices.  The results from this project have been published.  Drs. Miller, Anton, Ornstein, and Nietert also have been awarded a grant from the National Institute on Alcohol Abuse and Alcoholism to conduct a clinical trial to improve alcohol detection and treatment among hypertensive patients, by applying the PPRNet quality improvement model to a subset of practices participating in the Partnerships project. This project began in September 2004 and will continue for three years.
• A researcher at the Medical College of Georgia, Andria Thomas, PhD, joined the project team as a consultant to study adoption of obesity treatment guidelines in PPRNet practices. She completed a survey of project clinicians about their knowledge of and attitudes toward obesity treatment guidelines, and she conducted interviews with clinicians among practices that have excellent performance in achieving weight loss among obese patients.  She is developing a manuscript summarizing the results of these studies and is collaborating with other project investigators to develop an intervention method that can be tested in PPRNet practices.
• Dr. Matthew White, a project physician from Lakewood, WA, is working with his independent practice association and others in Washington State to share how he has implemented his EMR and reorganized his practice to improve clinical care.  He is making statewide presentations on this subject and has published a brief paper about it.
• Dr. James Wilson, a project physician from Fort Walton Beach, FL, has been contacted by the Institute of Medicine-Board on Health Care Services to present as a case study for performance measurement in a physician practice his work with the project. His presentation will provide background for an Institute of Medicine report, "Redesigning Health Insurance Benefits, Payments, and Performance Improvement Programs."

Return to Appendix B Contents

PFQ Grant Summary: Partnership for Advancing Quality Together

Lead Organization: Research Triangle Institute (RTI)
Partner Team: Five integrated delivery systems: UPMC Health System, Providence Health System (PHS), Intermountain Healthcare (IH), UNC Health Care, and Baylor Health Care System
Title: Partnership for Advancing Quality Together (PAQT)
Topic Area: Health care quality improvement, safety, and preparedness
Principal Investigators: Formerly Lucy Savitz, PhD, at RTI. After she left in September 2006, Shulamit L. Bernard, PhD, director of the Health Care Quality and Outcomes Program, became RTI's principal investigator. Each health system subcontractor has a co-principal investigator as well.
AHRQ Project Officer: Sally Phillips, PhD, RN
Total Cumulative Award: $994,796
Funding Period: 9/2002-9/2005
Project Status: Received two no-cost extensions extending period of performance to September 2007

1. Project Description

Goals.  In 2000, RTI received funding from AHRQ through the Agency's Integrated Delivery System Research Network (IDSRN) initiative.  The IDSRN initiative linked researchers with health care systems to conduct research on cutting-edge issues on an accelerated timetable.  As an IDSRN partner, RTI has collaborated with health care systems to conduct various research initiatives, including projects focused on health care quality improvement (QI), safety, and preparedness.  

When RTI applied for a PFQ grant, collaborators aimed to strengthen their existing IDSRN network and build on their IDSRN partnership work to influence the spread of the evidence base for quality improvement.  Other goals included (1) exploring factors that impede and facilitate inter- and intra-organizational sharing of knowledge; (2) extending the breadth and depth of the evidence base for innovative, sustainable QI and bioterrorism preparedness programs; (3) providing a mechanism to test the transportability of clinical process innovations; and (4) accelerating the rate at which knowledge utilization occurs. In addition, each partnering organization was to participate in at least one patient safety or bioterrorism preparedness project.  RTI later added goals aimed at advancing an understanding of partnership science and sharing such learning at the AHRQ program level. 

Activities and Progress. An eight-month delay in the release of funds from AHRQ delayed work during the project's first year.  During that first year, however, RTI conducted a systematic literature search and applied the findings to (1) the development of a guiding framework for using partnerships to stimulate change and (2) the development of a companion partnership synergy survey.  The survey assesses partnership strength and monitors continuous quality improvement among health care organizations. It addresses topics such as leadership and management, individual empowerment, synergy, and research transfer measures.   

In subsequent years of the project, grant funds enabled RTI's IDSRN partners to meet twice a year at the various partner health systems and to study the diffusion of effective health care interventions in 15 applied research projects pursued by partners under the IDSRN initiative (Table 1). Project examples included medication information transfer across the care continuum, validation of AHRQ's patient safety indicators, development of technology-based training for hospital preparedness, development and implementation of prospective patient injury detection systems, and development of a tool for estimating the financial impact of and opportunities to reduce the cost of waste or poor quality. Of the 15 applied research projects, 10 have concluded and 5 are in progress. The PFQ grant aimed to share knowledge of innovation to leverage the spread of selected IDSRN interventions within and across the health systems in the partnership. 

Table 1.  Partner Participation in IDSRN Initiatives

Project Title	Baylor	IH	PHS	UNC	UPMS
Validating AHRQ Quality Indicators		X	X		X
Assessing the IT Infrastructure in IDSs		X	X	X	X
Validating AHRQ's Patient Safety Indicators		X
Assessing IDS Solutions for Medication Information Transfer		X	X	X
AHRQ-Sponsored Workbook for Regional Preparedness		X			X
Estimating Risk Reduction and Cost-Enhancing Medication Information across Patient Care Settings			X
Facilitating Knowledge Transfer and Utilization via Hospital Patient Safety Indicator Online Query Tool		X
Facilitating Knowledge Transfer and Utilization of a Regional Bioterrorism Preparedness Workbook			X		X
Exploring the Special Needs and Potential Role of Nursing Homes in Surge Capacity for Bioterrorism and Other Public Health Emergencies		X	X	X	X
Cost of Poor Quality or Waste in IDS Settings I	X	X	X	X	X
Cost of Poor Quality or Waste in IDS Settings II		X	X	X
Developing a Targeted Injury Detection System	X	X
Medical Emergency Team Learning Opportunity					X
Implementing a Targeted Injury Detection System to Reduce Inpatient Injuries			X	X
Improving the Quality of Early Cancer Care		X

The in-person meetings of the RTI partnership group brought together senior management and operations staff who could identify their respective organization's needs and help shape further research projects. The meetings provided partners with a forum for presenting and discussing the outcomes of completed IDSRN projects and examining partners' uptake of those projects. RTI served as a conduit for the spread of innovation that led to new IDSRN projects and other diffusion-oriented grants.  

To track the spread of information among its partnership members, RTI compiled correspondence, meeting minutes, and archival records that documented uptake.  RTI asked partners to inform staff when their projects were completed and when there were outcomes to report.  Based on the partner members' health systems experience, RTI and the partner organizations developed a generalized approach to dissemination and implementation for bioterrorism preparedness and QI interventions that is based on the following six steps:

1. Pilot innovation in a credible place by a credible clinical champion with an engaged team that is empowered with resources.
2. Create a toolkit or manual that serves as a conduit with an audit tool for performance monitoring and feedback to involved staff.
3. Encourage review by an adopting organization and/or unit by linking an agent/clinical champion and his or her team.
4. Allow adaptation by an adopting organization/unit over time.
5. Provide for phased implementation by seeding the innovation on a small scale to support minimal adaptation and demonstrated value.
6. Ultimately, spread organization-wide diffusion of intervention as appropriate.

RTI also provided leadership and allocated a portion of its grant funds to support preparation of a supplemental issue of the Joint Commission Journal on Quality and Patient Safety to report on AHRQ learning from the Partnership Program. The supplement is currently scheduled for publication in spring 2007.

2. Partnership Structure/Function

RTI is the "facilitator" of the partnership, which involves several health systems.  Under RTI's innovation and implementation work as an IDSRN contractor with AHRQ, the partnership already existed before the launch of the PFQ program.  The four initial partner health care systems were Intermountain Healthcare (IH), Providence Health System (PHS), University of North Carolina (UNC) Health Care, and University of Pittsburgh Medical Center (UPMC) Health System.  After careful deliberation among RTI's partners, Baylor Health Care System in Texas joined the partnership in 2004 and rapidly became a vital member of the team.  The five partners offer a diversity of patient populations (including populations of priority interest to AHRQ); a strategic cross-section of the health care industry with respect to innovation, experience, and health information technology infrastructure; and health care settings appropriate for applied research. Organizational liaisons at each of the partner health systems are senior executives with sufficient standing to mobilize health system experts and actively engage them in the research process. These leaders have remained relatively constant throughout the grant period.  

The partners all participated in the in-person meetings held biannually at different partner locations. The partners also communicated regularly through conference calls and e-mail.  RTI established a confidential Web site for the partners to support their adoption of, communication about, and dissemination of shared learning. 

Table 2. Major Partner Organizations and Roles in the Project

	Organization	Role in Project
Lead Organization (grant recipient)	RTI	Serves as broker and facilitator in bringing partners together to conduct collaborative research and promote shared learning Provides technical and administrative support in the research process
Key Collaborators	UPMC Health System Providence Health System Intermountain Healthcare UNC Health Care Baylor Health Care System	Participate in biannual meetings and conference calls Assist other collaborators by serving as models for interventions or by translating interventions Work with RTI staff to translate innovative findings into manuscripts

3. Project Evaluation and Outcomes/Results

RTI's project focused on the spread of interventions developed within and across the partner health systems.  RTI researchers also have provided support for broader intellectual development on concepts related to partnerships, including the development of several products and tools (e.g., the partnership framework, the survey tool to monitor partnerships, the six-step implementation strategy, the book chapter on synergies, presentations, and so forth).

The project has produced several important findings and strategies for supporting knowledge transfer: (1) organizational modeling by credible organizations can accelerate knowledge transfer; (2) the primary evidence base (peer-reviewed literature) is limited to the extent that many innovations are not reported, and there is a bias toward reporting only successful efforts even though failed attempts often offer just as much insight; and (3) innovations in health care delivery are often complex interventions with several elements that go unreported and with essential versus adaptable elements of interventions that are not clearly delineated.

The PFQ grant enabled RTI to learn how to manage and sustain a partnership. The partnership has since evolved into a "learning laboratory" with many ideas flowing from the shared learning experience. The ideas have led to proposals for the IDSRN and other AHRQ initiatives.  The partners were exposed to cutting-edge initiatives at the meetings, and their interactions with each other presented new learning opportunities. The partnership also offered the partners credibility within their organizations when they presented new ideas.  

RTI used its partnership strength assessment tool for evaluation, thereby indicating continued, active involvement of partnership organizations.  Given its partnership framework and monitoring tool, RTI has attracted international interest, with health systems in Canada and Sweden participating in some meetings.  

4. Major Products

• Framework and companion survey tool for assessing partnership strength.
• Compendium CD with copies of selected partnership science literature and tools.
• Presentations at AcademyHealth 2004 Annual Research Meeting, "Demand Driven Research: The RTI Integrated Delivery System Research Network," and at the AHRQ Translating Research into Practice meeting, July 2004 (by Dr. Lucy Savitz).
• Supplemental issue of the Joint Commission Journal on Quality and Patient Safety reporting on AHRQ learning from the Partnership Program.

5. Potential for Sustainability/Expansion after PFQ Grant Ends

Given that RTI has received an award through the ACTION program (Accelerating Changes and Transformation in Organizations and Networks), which is AHRQ's new program that builds on the IDSRN, project activities will continue. The ACTION Master Task Order continues the relationship between RTI and its partner health systems, which will function as an applied research network to identify best practices and, for example, develop and test targeted injury detection systems, develop a system to redeploy unused health care resources, and create a prototype national patient tracking/locator model for use in times of disaster. RTI's partner health systems will extend the network's capacity by engaging local partners such as the Utah Department of Health; the Salt Lake Informatics, Decision Enhancement, and Surveillance Center (IDEAS); and the Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill.

The partnership strength model developed by RTI demonstrates that, to see value in a partnership, partners must perceive that they are actively participating in research activities.  To meet the needs of all partners, RTI is continually and actively seeking out research opportunities for them. To this end, RTI has engaged some of the partners in a separate Master Task Order entitled Developing Evidence to Inform Decisions about Effectiveness (DEcIDE), which was awarded to RTI through AHRQ's Effective Healthcare Program. Local partners of the partnering health systems were subcontractors on the first project awarded as part of the Master Task Order.

It is uncertain whether in-person meetings, which are dependent on funding, will continue after the PFQ grant ends.  Yet, regular communication and collaboration with most of the partners will certainly continue as a function of the partners' ongoing involvement in important projects that are in progress at RTI.

Return to Appendix B Contents
Return to Contents
Proceed to Next Section