Table 1. Strengths and weaknesses of strategies for handling missing data

Methodological Considerations in Generating Provider Performance Scores

Reason data are missingData missing in a way that is not related to true performance (e.g., missing at random)Data missing in a way that is related to true performance (e.g., low performers not reporting data)
Option 1: ImputationStronger: may reduce the risk of misclassification due to chanceWeaker: may introduce systematic performance misclassification
Option 2: Report the average scoreStronger: will not result in systematic performance misclassificationWeaker: high likelihood of resulting in systematic performance misclassification; and creates incentive not to report low performance
Option 3: Report only the available dataStronger: will not result in systematic performance misclassificationWeaker: may create incentive not to report low performance
Option 4: Report only when providers are not missing any dataStronger: will not result in systematic performance misclassificationWeaker: may create incentive not to report low performance
Option 5: Report the lowest possible scoreWeaker: high likelihood of resulting in systematic performance misclassification*Stronger: potentially less likely to result in systematic performance misclassification; creates incentive to report all performance data
Option 6: Report the lowest observed scoreWeaker: high likelihood of resulting in systematic performance misclassification*Stronger: potentially less likely to result in systematic performance misclassification; creates incentive to report all performance data

* For example, if providers with fewer patients are more likely to have missing data (regardless of their performance), then these providers will be systematically misclassified as low performers.

Return to Document

Page last reviewed September 2011
Internet Citation: Table 1. Strengths and weaknesses of strategies for handling missing data: Methodological Considerations in Generating Provider Performance Scores. September 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://archive.ahrq.gov/professionals/quality-patient-safety/quality-resources/value/perfscoresmethods/perfsctab1.html