Program Assessment Rating Tool Evaluations

---

For the FY 2004 President's budget, the Office of Management and Budget (OMB) picked several programs at each agency across the Federal government to review. These performance assessments were developed using the program assessment rating tool (PART).

The PARTs are a series of questions designed to provide a consistent approach to rating programs across the Federal Government. PARTs are diagnostic tools that rely on professional judgment to assess and evaluate programs across a wide range of issues related to performance.

Translating Research Into Practice / Data Collection and Dissemination

---

Program: Translating Research Into Practice

Agency: Department of Health and Human Services
Bureau: Agency for Healthcare Research and Quality

Purpose: 100
Planning: 88
Management: 56
Results/Accountability: 33

___ Results Achieved
___ Results Not Demonstrated
_X_ Measures Adequate
___ New Measures Needed

Key Performance Measures	Year	Target	Actual
Long-term Measure: Rate of hospitalizations for pediatric asthma in persons under age 18 (Modified existing measure)	1999		178,901
	2000		150,876
	2010	105,613
Long-term Measure: Number of immunization- preventable pneumonia hospital admissions of persons aged 65 and older (Modified existing measure)	1999		178,264
	2000		743,487
	2010	520,441
Long-term Measure: Number of immunization-preventable influenza hospital admissions of persons aged 65 and older (Modified existing measure)	1999		17,508
	2000		16,529
	2010	11,570

Rating: Adequate

Program Type: Research and Development.

Program Summary: Translating Research Into Practice (TRIP) grants are provided to public or private entities, universities, and clinics, to address common health conditions, including: infant mortality, cancer screening, cardiovascular disease, diabetes, HIV/AIDS, child and adult immunizations, and mental health and pediatric asthma. Grantees assess the effectiveness of promising new interventions; determine if the new interventions are replicable; compare their benefits, costs, and effects on existing approaches; and focus on the day-to-day provider behavior changes that could improve health outcomes.

The assessment found:

1. There is general consensus that the program purpose of TRIP is clear.
2. The program developed new long-term goals in September 2001 and modified them for the 2004 budget. The 2004 national measures will allow TRIP grantees to assess how the use of new interventions in health care settings could help improve health outcomes/reduce the number of unnecessary hospitalizations for those with pediatric asthma and those receiving adult immunizations. The program recognizes the difficulty of linking expenditures to actual program performance, but plans to better integrate its existing planning and budget databases to easily identify those programs that are not meeting their GPRA goals.
3. The program does not use performance information to manage the program and does not measure efficiencies and/or cost savings.
4. There is a lack of long-term and annual performance goals. As a result, it is difficult to measure the success of the program or the achievements/ contributions it may be making to changes in provider behavior.

In response to these findings, the Administration will:

1. Maintain funding at the 2003 budget level to ensure continued efforts to go beyond collecting data to actually changing provider behavior and thus improving health outcomes.
2. The program is addressing its management deficiencies and will begin better integrating its planning and budget decisionmaking processes.

Program Funding Level (in millions of dollars)

2002 Actual	2003 Estimate	2004 Estimate
14	7	7

Program: Data Collection and Dissemination

Agency: Department of Health and Human Services
Bureau: Agency for Healthcare Research and Quality

Purpose: 83
Planning: 89
Management: 80
Results/Accountability: 67

___ Results Achieved
___ Results Not Demonstrated
_X_ Measures Adequate
___ New Measures Needed

Key Performance Measures	Year	Target	Actual
Long-term Measure: Number of months after the date of completion of the Medical Expenditure Panel Survey data will be available (New measure)	1997		19-27
	2008	12
Long-term Measure: Number of organizations that will use Healthcare Cost and Utilization Project databases, products or tools to improve statewide health care quality for their constituencies (New measure, baseline under development)	2010	5
Annual Measure: The strategy for achieving the long-term goal on Healthcare Cost and Utilization Project databases, products or tools (New measure, baseline and targets under development)

Rating: Moderately Effective

Program Type: Research and Development.

Program Summary: These programs collect data on the cost (Medical Expenditure Panel Survey), use (Healthcare Cost and Utilization Project), and quality of health care in the United States and develop and survey beneficiaries regarding their health care plans (Consumer Assessment of Health Plans).

The assessment found:

1. The Agency for Healthcare Research and Quality (AHRQ) recently developed new long-term and annual performance measures and goals. Program partners have committed to achieving the stated goals and these programs undergo regular evaluations.
2. The programs do not yet have data available to measure their new targets.
3. Program managers acknowledged difficulties tracking budgetary expenditures and their impacts on actual program performance. AHRQ will begin to connect budget and planning systems to identify more easily those activities not meeting their goals.
4. The purpose of these programs is clear—to have a unique impact on the need for and availability of national level health care cost, utilization, and health plan data. These programs do not effectively articulate their public benefits.
5. AHRQ regularly collects timely and credible performance data and uses these data to manage the program. There are some management deficiencies including an inability to determine the full annual costs of these programs.

In response to these findings, the Administration will:

1. Collect performance data on the new measures.
2. Propose an increase of $5 million above the 2003 Budget to support AHRQ's efforts to ensure continued collection and availability of national health care cost, use, and quality data.
3. AHRQ has begun to address management deficiencies by adopting performance-based contracts that require superior performance toward achieving established goals.

Program Funding Level (in millions of dollars)

2002 Actual	2003 Estimate	2004 Estimate
55	60	65

Return to Medical Expenditure Panel Survey

Current as of February 2003