Initial Core Set of Children's Healthcare Quality Measures

Identification of Initial Core Set of Measures for Voluntary Use by Medicaid and Children's Health Insurance Program (CHIP) Programs

Core set recommended by the AHRQ National Advisory Council Subcommittee on Children's Healthcare Quality Measures for Medicaid and CHIP Programs, September 18, 2009

This summary presents a brief overview of the processes used to recommend an initial core set of children's health care quality measures to the Agency for Healthcare Research and Quality (AHRQ) National Advisory Council on Healthcare Research and Quality, the measures recommended, and next steps to be taken by AHRQ and the Centers for Medicare & Medicaid Services (CMS).

Background | SNAC and the Public Process | Recommendations | Important Considerations | Next Steps | Tables

Background

Title IV of the Children's Health Insurance Program Reauthorization Act (CHIPRA; Public Law 111-3; February 3, 2009) called for the Secretary of the U.S. Department of Health and Human Services (HHS) to identify an initial core set of children's health care quality measures to be posted for general comment by January 1, 2010 (Section 401(a)). In April 2009, the AHRQ and CMS signed a Memorandum of Understanding giving AHRQ leadership responsibilities for identifying the initial core set, working in very close partnership with CMS which has the authority for implementation of all CHIPRA provisions.

As part of their effort to use a transparent and evidence-based process for identifying an initial core set of pediatric quality measures for voluntary use by Medicaid and CHIP Programs to be posted for general comment by January 1, 2010: CMS and AHRQ asked the AHRQ National Advisory Council on Healthcare Research and Quality (NAC) to establish a time-limited Subcommittee on Children's Healthcare Quality Measures for Use by Medicaid and CHIP programs (SNAC). The SNAC was asked to hold public meetings and was charged with developing criteria for identifying core measures, applying those criteria to measures in use or otherwise nominated for consideration, and making recommendations to the NAC. The NAC in turn provides advice to the Director, AHRQ, and to the Secretary, HHS. For more information on the public SNAC meetings and overall process for identification of the initial core measure set, other CHIPRA quality activities, a link to CMS CHIPRA activities, and a link to Public Law 111-3, visit https://www.ahrq.gov/chipra.

Top of Page

SNAC and the Public Process

The SNAC met for the first time in July 2009 to discuss and further refine a SNAC-developed modified Delphi process to score measures known to be in use by Medicaid and CHIP programs on validity, feasibility, and importance. The SNAC also agreed to try to recommend a core measure set no greater than 25 measures. The SNAC report to the NAC in July 2009 is available at .

By mid-October 2009, the SNAC had 121 measures to consider during a second modified Delphi process. These 121 measures included measures that:

1. Had passed criteria for validity, feasibility, and importance at the July SNAC meeting.
2. Were deemed to be in need of additional information at the July SNAC meeting.
3. Had been nominated by SNAC members, members of the public, or members of the CHIPRA Federal Quality Workgroup or identified through continuing environmental scans.

Public nominators included members of the Medicaid Medical Directors Learning Network, the Medical Association's Physician Consortium for Performance Improvement (PCPI), the National Partnership for Women and Families, and additional nominations obtained through E-mail to an AHRQ address. CHIPRA Federal Quality Workgroup nominations came from CMS and the Health Resources and Services Administration (HRSA). Papers by CHIPRA contractors also recommended measures for consideration.

Using a second modified Delphi scoring process in early September, SNAC members selected 65 of the 121 measures as meeting criteria for validity, feasibility, and importance. At the September 17-18, 2009 SNAC meeting, several measures that did not strictly pass the second Delphi round (e.g., validity scores of 6 or 6.5 rather than the cutoff of 7) were added to the list of measures to be discussed and voted on during the meeting. An example was adolescent well-care visits.

In addition, during the September 17-18, 2009, SNAC meeting, the SNAC voted to combine some measures and consider them to be single measures for purposes of further SNAC voting. For example, three well-child care visits were combined into one measure for voting purposes (shown as PHP-9 on the SNAC recommendation table); and the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) surveys for children with and without special health care needs were combined into the voting category FEC-1/FEC-5.

Top of Page

Recommendations

Through a series of discussions followed by a series of votes over the day and a half meeting, the SNAC agreed to recommend to the NAC 25 measures for the initial core measure set. (Table 1).

The SNAC-recommended set of measures will next be discussed by the Children's Health Insurance Program Reauthorization Act (CHIPRA) Federal Quality Workgroup and reviewed by Medicaid and CHIP officials. Subsequently, a CMS-AHRQ-recommended initial core set will be forwarded to the Secretary, HHS, for her consideration and then to the Office of Management and Budget (OMB) for its review, before a final recommended set of measures is posted for general comment, by January 1, 2010.

Top of Page

Important Considerations

It is important to note that the SNAC did not recommend that the measures on the list be implemented "as is." Rather, the measure denominators should be respecified so that they are feasible for use across all Medicaid and CHIP programs and by providers, consumers, and intermediaries (e.g., health plans contracting with State Medicaid programs). For example, HEDIS CAHPS® (FEC 1 and 5) (Healthcare Effectiveness Data and Information Set/Consumer Assessment of Health Plans and Providers) as currently specified are currently used primarily by Medicaid managed care health plans that report to the National Committee for Quality Assurance (NCQA). The SNAC recommended that in the future, the CAHPS® instruments be used by all Medicaid and CHIP programs so that family experiences of care across a broader spectrum of covered children can be understood, compared, and when needed, acted upon. Similarly, few of the measures are used to report data that distinguish care quality by race, ethnicity, socioeconomic status, or special health care need status among children. In the future, consistent with the CHIPRA legislation, data should be collected so that meaningful comparisons can be made between children with and without special health care needs and among children of varying racial and ethnic identities.

Top of Page

Next Steps

The SNAC-recommended set of measures will next be discussed by the CHIPRA Federal Quality Workgroup on September 23, 2009, and reviewed by Medicaid and CHIP officials in a CMS listening session on September 29, 2009, and at other venues. Subsequently, a CMS-AHRQ-recommended initial core set of measures will be forwarded to the Secretary, HHS, for her consideration, and then to the White House Office of Management and Budget for its review, before posting a final recommended set for general comment, by January 1, 2010.

Public comments on the process of identifying the initial core measure set for voluntary use by Medicaid and CHIP programs are continuously invited via the E-mail address at .

Top of Page

Table 1. SNAC-recommended initial core set of children's healthcare quality measures, in priority order by number of SNAC votes, September 17-18, 2009

SNAC Voting: 1st place votes receive 3 points; 2nd place votes receive 2 points; 3rd place votes receive 1 point. Sum of votes = sum of points awarded. Rank ordered here by total number of points awarded. Thus, 25 measures were recommended by the SNAC.

*Key: V=Validity F=Feasibility I=Importance IQR=Inter-Quartile Range N=Number of Subcommittee members rating measure. 

Table 2. Measures that Passed Delphi II but Were Not Chosen as Recommended Measures by SNAC Voting, September 18, 2009

 Table 3. Measures That Did Not Meet Thresholds for Delphi II Scoring on Validity, Feasibility, and Importance

Key:
V = Validity
F = Feasibility
I = Importance
IQR = Inter-Quartile Range
N = Number of Subcommittee members rating measure

Top of Page