Effect of Provider Volume on the Accuracy of Hospital Report Cards
A Monte Carlo Study
Background—Hospital report cards, in which outcomes after the provision of medical or surgical care are compared across healthcare providers, are being published with increasing frequency. However, the accuracy of such comparisons is controversial, especially when case volumes are small. The objective was to determine the relationship between hospital case volume and the accuracy of hospital report cards.
Methods and Results—Monte Carlo simulations were used to examine the influence of hospital case volume on the accuracy of hospital report cards in a setting in which true hospital performance was known with certainty, and perfect risk-adjustment was feasible. The parameters used to generate the simulated data sets were obtained from empirical analyses of data on patients hospitalized with acute myocardial infarction in Ontario, Canada, in which the overall 30-day mortality rate was 11.1%. We found that provider volume had a strong effect on the accuracy of hospital report cards. However, provider volume had to be >300 before ≥70% of hospitals were correctly classified. Furthermore, hospital volume had to be >1000 before ≥80% of hospitals were correctly classified.
Conclusions—Producers and users of hospital report cards need to be aware that, even when perfect risk adjustment is possible, the accuracy of hospital report cards is, at best, modest for small to medium-sized case loads (ie, 100–300). Hospital report cards displayed high degrees of accuracy only when provider volumes exceeded the typical annual hospital case load for many cardiovascular conditions and procedures.
- Received October 22, 2013.
- Accepted February 11, 2014.
- © 2014 American Heart Association, Inc.