Appendix A: Estimated Costs for Evaluation Design

Design and Evaluation of Three Administration on Aging (AoA) Programs: Chronic Disease Self-Management Program Evaluation Design—Final Evaluation Design Report (continued)

Table A1: Cost Estimates for the Experimental Evaluation with 50 Sites

Task	Estimated Cost	Comments / Assumptions
Environmental Scan and Focus Groups	$125,000	Tasks will include a reassessment of feasibility of the design based on fresh CDSMP program data and new policy environment, identifying Medicare outcomes that would be proxies to Medicaid outcomes, assessment of the extent of States' outreach to dual eligible enrollees, identifying a number of states which have robust CDSMP outreach to dual eligible beneficiaries, among these states identifying/recruiting a few states interested in sharing Medicaid data, review of other evidence based initiatives that introduce contamination to the site selection process, and the availability of appropriate instruments with good psychometric properties. Conducting focus groups will facilitate decision making on design-related issues such as estimating impact on Medicaid outcomes using Medicare data on dual eligibles and feasibility of implementing the RCT. It will also provide valuable input to the evaluation design.
Develop a Uniform Application System	$450,000	Work with AoA, the sites, and possibly with the TA contractor to design a uniform registration and baseline data collection system at each evaluation site. This system will be supported by a Participant Tracking System (see below). This task will first involve reviewing and understanding of current practices at each evaluation site. This task will involve meetings and coordination with a number of stakeholders.
Evaluation Design and TEP	$75,000	TEP will review and comment on the evaluation design and evaluation report.
IRB/OMB Clearance	$35,000	Preparation of OMB package for mail and phone-based CDSMP participant survey and web-based host site survey.
Measurement of Implementation Factors	$250,000	Developing and conducting a web survey of evaluation sites, including trainers, to measure implementation factors that may influence outcomes. Write a brief report of results. This will also help interpret outcome evaluation results.
Project Management	$200,000	These are administrative costs associated with project management, client relations, subcontractor relations and communications, etc.
Participant Tracking System	$500,000	These are costs associated with development of participant tracking system; site training including TA; continuous monitoring/reporting of site data collection and assessing contamination; and maintenance of the system. This estimate assumes costs to develop and maintain a participant tracking system for 50 sites during the evaluation period. To assess contamination, everybody who takes CDSMP workshops will also be recorded regardless of whether they participate in the evaluation. This list will then be checked against the list of control group members.
Data Collection	$765,000	This estimate reflects the cost of planning and preparation (e.g. obtaining sampling frame, survey piloting and design, site selection, programming survey into CATI) for the data collection, the data collection, compensation of follow up responders for their time ($15 per follow up), longitudinal file development (Medicare data and survey data) and all post-data collection survey activities (QA, developing non-response weights, etc.). There three waves: Baseline (paper-based), 6-month follow up (CATI+mail), and 12-month follow up (CATI+mail). Target number of baseline responses to survey is 3,000 individuals (1,500 treatment, 1,500 control). Baseline paper-based responses will be coded using double data entry. Cost of purchasing Medicare and Medicaid data is not included.
Analysis and Reporting	$200,000	Includes an interim and a final analysis/report.
*TOTAL EVALUATION CONTRACT* *Experimental Design, 50 Evaluation Sites*	*$2,600,000*	This evaluation is estimated to cost about $2,200,000 if 20 evaluation sites are used. Targeting larger baseline samples of 4,000 and 5,000 is estimated to cost additional $120,000 and $240,000, respectively.

Total evaluation contractor cost excludes the cost of recommended AoA grants to host sites for participation in the evaluation. As explained in the report, these grants are for 1) developing and implementing a uniform CDSMP application system and 2) sustaining and enhancing CDSMP participant recruitment. This additional cost is estimated to be $3,120,000 for 50 sites. Please see below for a breakdown of these grants.

AoA Grant	Estimated Cost to AoA	Comments/Assumptions
AoA Grants to Develop Uniform Application System, Data Collection System, and to Support a Liaison with the Evaluator on Data Matters	$480,000	25% time of a staff with $60,000 annual salary for two years per state. Cost estimate assumes inclusion of host sites from 16 states.
Grants to Expand and Enhance Site Recruitment and Marketing Strategies	$2,640,000	In order to maintain and double current participation beyond expiration of ARRA funding, we strongly recommend grants of $165,000/state. These funds will also cover staff that will input into and use the Participant Tracking System. Cost estimate assumes inclusion of host sites from 16 states in the evaluation.
Total AoA Grant Cost	*$3,120,000*

Table A2: Cost Estimates for the Quasi-Experimental Evaluation with 50 Sites

Task	Estimated Cost	Comments/Assumptions
Environmental Scan and Focus Groups	$125,000	Tasks will include a reassessment of the feasibility of design based on fresh CDSMP program data and new policy environment, identifying Medicare outcomes that would be proxies to Medicaid outcomes, assessment of the extent of States' outreach to Medicaid enrollees, identifying a number of states which has robust CDSMP outreach to Medicaid beneficiaries, among these states identifying/recruiting a few states interested in sharing Medicaid data, review of other evidence based initiatives that introduce contamination to the site selection process, and the availability of appropriate instruments with good psychometric properties. Conducting focus groups will facilitate decision making on design-related issues such as Medicaid-related issues and feasibility of implementing the RCT. It will also provide valuable input to the evaluation design.
Develop a Uniform Application System	$450,000 for 50 sites	Work with AoA, the sites, and possibly with the TA contractor to design a uniform registration and baseline data collection system at each evaluation site. This system will be supported by a Participant Tracking System (see below). This task will first involve reviewing and understanding of current practices at each evaluation site. This task will involve meetings and coordination with a number of stakeholders.
Evaluation Design and TEP	$85,000	TEP will review and comment on the evaluation design and evaluation report.
IRB/OMB Clearance	$35,000	Preparation of OMB package for mail and phone-based CDSMP participant survey and web-based host site survey.
Measurement of Implementation Factors	$250,000	Developing and conducting a web survey of evaluation sites including trainers to measure implementation factors that may influence outcomes. Write a brief report of results. This will also help interpret outcome evaluation results.
Project Management	$200,000	These are administrative costs associated with project management, client relations, subcontractor relations and communication, etc.
Participant Tracking System	$440,000	These are costs associated with development of participant tracking system; site training including TA; continuous monitoring/reporting of site data collection and assessing contamination; and maintenance of system. This estimate assumes costs to develop and maintain a participant tracking system for 50 sites during the evaluation period. To assess contamination, everybody who takes CDSMP workshops will also be recorded regardless of whether they participate in the evaluation. This list will then be checked against the list of control group members. This version of the PTS will not have a module for randomization.
Data Collection	$615,000	This estimate reflects the cost of planning and preparation (e.g. obtaining sampling frame, survey piloting and design, site selection) for the data collection, the data collection, compensation of follow up responders for their time ($15 per follow up), longitudinal file development (Medicare data and survey data) and all post-data collection survey activities. There three waves: Baseline (paper-based), 6-month follow up (CATI+mail), and 12-month follow up (CATI+mail). Target number of baseline responses is 2,000 individuals among people taking the CDSMP workshop. Cost of purchasing Medicare and Medicaid data is not included.
Analysis and Reporting	$450,000	Includes an interim and a final analysis/report. Number of separate PSM analyses will need to be at least as many as the number of sites – beneficiaries will be “hard-matched" by site in which they reside. Furthermore, if it appears that there will be reasonable number of dual eligibles in the study sample, then beneficiaries will need to be hard-matched by their dual status as well. This doubles the number of separate PSM analyses.
*TOTAL EVALUATION CONTRACT* *Quasi-Experimental Design (Propensity Score Matching), 50 Evaluation Sites*	*$2,600,000*	This evaluation is estimated to cost about $2,100,000 if 20 evaluation sites are used.

Total evaluation contractor cost excludes the cost of recommended AoA grants to host sites for participation in the evaluation. As explained in the report, these grants are for 1) developing and implementing a uniform CDSMP application system and 2) sustaining and enhancing CDSMP participant recruitment. This additional cost is estimated to be $2,432,000 for 50 sites. Please see below for a breakdown of these grants.

AoA Grant	Estimated Cost to AoA	Comments/Assumptions
AoA Grants to Develop Uniform Application System, Data Collection System, and to Support a Liaison with the Evaluator on Data Matters	$480,000	25% time of a staff with $60,000 annual salary for two years per state. Cost estimate assumes inclusion of host sites from 16 states.
AoA Grants to Expand and Enhance Site Recruitment and Marketing Strategies	$1,952,000	In order to maintain and significantly increase current CDSMP participation beyond expiration of ARRA funding, we strongly recommend grants of $122,000/state. These funds will also cover staff that will input into and use the Participant Tracking System. Cost estimate assumes inclusion of host sites from 16 states in the evaluation.

Return to Contents
Proceed to Next Section

Page last reviewed December 2014
Page originally created May 2011

Internet Citation: Appendix A: Estimated Costs for Evaluation Design. Content last reviewed December 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://archive.ahrq.gov/research/findings/final-reports/aoa/aoachronic-apa.html

The information on this page is archived and provided for reference purposes only.