An Administrative Claims Measure Suitable for Profiling Hospital Performance on the Basis of 30-Day All-Cause Readmission Rates Among Patients With Heart FailureCLINICAL PERSPECTIVE
Background— Readmission soon after hospital discharge is an expensive and often preventable event for patients with heart failure. We present a model approved by the National Quality Forum for the purpose of public reporting of hospital-level readmission rates by the Centers for Medicare & Medicaid Services.
Methods and Results— We developed a hierarchical logistic regression model to calculate hospital risk-standardized 30-day all-cause readmission rates for patients hospitalized with heart failure. The model was derived with the use of Medicare claims data for a 2004 cohort and validated with the use of claims and medical record data. The unadjusted readmission rate was 23.6%. The final model included 37 variables, had discrimination ranging from 15% observed 30-day readmission rate in the lowest predictive decile to 37% in the upper decile, and had a c statistic of 0.60. The 25th and 75th percentiles of the risk-standardized readmission rates across 4669 hospitals were 23.1% and 24.0%, with 5th and 95th percentiles of 22.2% and 25.1%, respectively. The odds of all-cause readmission for a hospital 1 standard deviation above average was 1.30 times that of a hospital 1 standard deviation below average. State-level adjusted readmission rates developed with the use of the claims model are similar to rates produced for the same cohort with the use of a medical record model (correlation, 0.97; median difference, 0.06 percentage points).
Conclusions— This claims-based model of hospital risk-standardized readmission rates for heart failure patients produces estimates that may serve as surrogates for those derived from a medical record model.
Received June 26, 2008; accepted July 7, 2008.
Readmissions to the hospital shortly after discharge are expensive and often preventable events for heart failure patients.1–3 Heart failure is the most frequent principal discharge diagnosis for hospitalized Medicare beneficiaries4 and is associated with all-cause readmission rates of >40% by 6 months after discharge.1 The Medicare Payment Advisory Commission, an independent federal body that advises the US Congress, has recently called for hospital-specific public reporting of readmission rates, identifying heart failure as a priority condition.2 Readmissions may result from a variety of care deficits, including premature hospital discharge, inadequate preparation of the patient and family for discharge, complications that manifest after discharge, and poor care transitions. Moreover, studies have identified underutilized interventions in and out of the hospital that can decrease readmission risk after a heart failure hospitalization.5–12 Despite the clinical and policy importance, a validated, risk-standardized statistical model to accurately profile hospital readmission rates is not available in the published literature.13
Editorial see p 9
Clinical Perspective p 37
This article presents a model, approved by the National Quality Forum,14 to estimate hospital-specific readmission rates for Medicare patients hospitalized with heart failure. We developed and validated the model with Medicare administrative claims data and determined whether estimates from the claims model were good surrogates for the results of a medical record model. We sought to ensure that this model had all the key attributes for publicly reported outcomes measures articulated by an American Heart Association Scientific Statement.15 This approach extends the work that produced National Quality Forum–approved models of 30-day mortality rates for acute myocardial infarction and heart failure, now publicly reported on Hospital Compare by the Centers for Medicare & Medicaid Services (CMS).16–18
For administrative model derivation and validation for 2003 and 2004 cohorts, we used 2002–2005 claims data from the Medicare inpatient, outpatient, and carrier (physician) Standard Analytic Files. The Medicare Enrollment Database was used to determine Medicare fee-for-service enrollment status and mortality status. For medical record model validation, we analyzed 1998–2001 medical record data from the National Heart Failure (NHF) Project, a CMS quality improvement project.19 The NHF cohorts consisted of 39 477 records in 1998–1999 and 39 405 records in 2000–2001, with up to 800 discharges randomly selected in each state, Washington DC, and Puerto Rico.
We identified hospitalizations of patients ≥65 years of age with a principal discharge diagnosis of heart failure as potential index heart failure hospitalizations (International Classification of Diseases, 9th Revision, Clinical Modification codes 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, or 428.xx). Because our focus was on readmission, we excluded hospitalizations during which the patient died or was transferred to another acute care facility.
To maximize our ability to risk-adjust and to identify readmissions, we restricted the cohort to patients enrolled in fee-for-service Medicare Parts A and B for 12 months before their heart failure hospitalization and who continued in fee-for-service for ≥30 days after their discharge.
Our unit of observation was a heart failure hospitalization. Additional heart failure hospitalizations that occurred within 30 days of discharge of an index hospitalization were classified as an outcome, ie, a readmission, and thus not defined as an index hospitalization.
The primary outcome was 30-day readmission, defined as the occurrence of at least 1 hospitalization in any US acute care hospital for any cause within 30 days of discharge after an index hospitalization. We identified readmissions from the hospital claims data. Each readmission was attributed to the hospital that discharged the patient.
We developed candidate variables for the model from the claims codes using the aforementioned data sources. To assemble clinically coherent codes into candidate variables, we used the publicly available CMS hierarchical condition categories (CCs) to group codes into 189 CCs.20,21 A CC was indicated as present for a given patient if it was coded in any of the hospital inpatient, outpatient, or physician claims data sources in the prior 12 months, including the index admission. Additional candidate variables included 2 demographic variables and procedure codes relevant to heart failure readmission risk. A physician team identified candidate variables and differentiated CC variables that when coded as secondary diagnosis codes during the index hospitalization could represent either comorbid conditions on admission or complications of care (eg, urinary tract infection). To avoid including potential complications as comorbidities, we did not code them as risk factors if they appeared only as secondary diagnosis codes for the index hospitalization and not on claims in the prior year.
A physician team (H.M.K., J.S.R., B.D.S., S.M.B., E.E.D.) selected risk factors for the final model on the basis of their statistical association with and clinical relevance to readmission, with reference to prior research.13 Additional details are provided in the Statistical Appendix in the online-only Data Supplement.
For the derivation of the administrative claims model, we randomly sampled half of the 2004 hospitalizations that met inclusion criteria. We conducted analyses of model performance by using a generalized linear model with a logit link function. To assess model performance at the patient level, we calculated the area under the receiver operating characteristic curve (AUC), explained variation as measured by the generalized R2 statistic, and calculated the observed readmission rates in the lowest and highest deciles on the basis of predicted readmission probabilities.23 We also compared performance with a null model, a model that adjusted for age and sex, and a model that included all of the candidate variables.
Risk-Standardized Readmission Rate
Given the clustering of admissions within hospitals and that hospitals were our unit of inference, we estimated risk-standardized readmission rates by using hierarchical generalized linear models.24 This modeling strategy accounts for within-hospital correlation of the observed readmission rates and reflects our assumption that after adjustment for patient risk and sampling variability, the remaining variation is due to hospital quality.
We next calculated risk-standardized hospital-specific readmission rates. These rates are obtained as the ratio of the number of “predicted” to “expected” readmissions, multiplied by the national unadjusted rate. The predicted number of readmissions in each hospital is estimated given its own patient mix and with its own hospital-specific intercept. The expected number of readmissions in each hospital is estimated with its own patient mix and the average hospital-specific intercept based on all hospitals in our sample. (Additional information is available in the Statistical Appendix in the online-only Data Supplement.) This is a form of indirect standardization.
We compared the model performance in the derivation sample with its performance in the remaining half of the 2004 claims data and, separately, with the 2003 claims data. The model was recalibrated in each validation set. We calculated indices that quantify overfitting for each validation data set, each time calculating a risk score using the regression estimates from our derivation model.23 A risk score coefficient that is much different from 1 and an intercept different from 0 are indicative of overfitting. We also examined whether model performance varied for important subgroups of patients: older patient age, sex, race/ethnicity, and urban/rural hospital.
Medical Record Model Validation
We developed a separate medical record model of readmission risk on the basis of NHF data (see Statistical Appendix in the online-only Data Supplement). We also linked the patients in the NHF cohort to their Medicare claims data, including claims from 1 year before the index hospitalization, so that we could calculate the risk-standardized state readmission rates in this cohort separately using medical record and claims data models. We conducted this analysis at the state level, for the 50 states plus the District of Columbia and Puerto Rico, because medical record data were only available in sufficient numbers to perform a state-level comparison. To examine the relationship between the risk-standardized rates obtained from medical record and administrative data models, we estimated a linear regression model describing the association between the 2 rates, weighting each state by the number of index hospitalizations, and calculated the intercept and the slope of this equation. A slope close to 1 and an intercept close to 0 would provide evidence that the risk-standardized state readmission rates from the medical record and claims models were similar. We also calculated the difference between the state risk-standardized readmission rates from the 2 models.
Analyses were conducted with the use of SAS version 9.1.3 (SAS Institute Inc, Cary, NC). Models were fitted separately for the NHF cohort and the 2004 cohort. We estimated the hierarchical models using the GLIMMIX procedure in SAS. The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.
The authors had full access to the data and take responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.
After exclusions were applied, the 2004 data included 567 447 heart failure hospitalizations (Table 1). The derivation sample consisted of 283 919 hospitalizations in 4669 hospitals with at least 1 hospitalization, with an unadjusted 30-day readmission rate of 23.6%. The observed readmission rate ranged from 0% to 100% across these hospitals with 25th, 50th, and 75th percentiles of 17.1%, 23.0%, and 28.2%, respectively. The median annual number of Medicare heart failure hospitalizations was 71 (25th, 75th percentile: 27, 166). In 7% of admissions, the patient died within the 30 days after discharge. In 4.4% of admissions, the patient died without being readmitted, and in 2.6% of admissions, the patient was readmitted and died within the 30 days after discharge. Despite an average survival time of 14 days, the readmission rate was 36.9% among the 7% of admissions in which the patient died within 30 days, compared with a readmission rate of 22.7% for admissions in which patients survived the full 30 days.
The claims model included 37 variables (2 demographic, 9 cardiovascular, and 26 comorbidity variables) (Table 2). The mean age of the cohort was 79.9 years, with 57.8% women and 16.3% nonwhite patients. The mean observed readmission rate in the derivation data set ranged from 15% in the lowest decile of predicted readmission rates to 37% in the highest predicted decile, an absolute difference of 22% (Table 3). The AUC was 0.50 for a null model, 0.52 for a model with only age and sex, and 0.60 for a model that included all the candidate variables.
Hospital Risk-Standardized Readmission Rates
Figure 1 displays the distributions of the risk-standardized 30-day readmission rates. The 5th percentile was 22.2%, and the 95th percentile was 25.1%. The 25th and 75th percentiles were 23.1% and 24.0%, respectively. The mean risk-standardized rate was 23.6%. The odds of all-cause readmission for a hospital 1 standard deviation above average was 1.30 times that of a hospital 1 standard deviation below average.
Administrative Model Validation
With the use of the remaining 50% of heart failure index hospitalizations from 2004 and all of the 2003 data, the regression coefficients and SEs were similar to those for the derivation data set. The performance was also similar in the validation data sets (Table 3).
Medical Record Validation
Initial NHF data contained 78 882 hospitalization records. The final NHF validation sample included 64 329 hospitalizations from 4437 hospitals after exclusions were applied for age <65 years (8.3% of initial sample), in-hospital death (5.1%), transfer to another acute care facility (0.4%), incomplete information in the 12 months before admission (2.7%), incomplete 30-day readmission information (4.1%), and heart failure hospitalizations within 30 days of prior index hospitalizations (1.6%). The crude 30-day readmission rate was 23.7%.
In 6.8% of admissions, the patient died within the 30 days after discharge. In 4.1% of admissions, the patient died without being readmitted, and in 2.7% of admissions the patient was readmitted and died within the 30 days after discharge. The readmission rate was 39.6% among the 6.8% of admissions in which the patient died within 30 days compared with 22.5% for admissions in which patients survived 30 days.
The medical record comparison model included 30 variables (Table 4). In the medical record model, the AUC was 0.58, and the observed readmission rate ranged from 16% in the lowest predicted decile to 34% in the highest (Table 5). In the same cohort of 64 329 hospitalizations, the administrative model had an AUC of 0.61 and observed readmission rates ranging from 15% in the lowest predicted decile to 38% in the highest predicted decile.
The estimated state-specific standardized readmission rates derived from each model are displayed in Figure 2. The slope of the weighted regression line between chart- and claims-based state readmission rates was 0.81 (SE=0.0008), and the intercept was 0.04 (SE=0.0002). The correlation coefficient of the standardized readmission rates from the 2 models was 0.97 (SE=0.03). The median difference between the models in the state-specific risk-standardized readmission rates was 0.06 percentage points (25th percentile=−0.5; 75th percentile=0.5; 10th percentile=−0.9; 90th percentile=0.8 percentage points). The odds of all-cause readmission for a hospital 1 standard deviation above average was 1.34 times that of a hospital 1 standard deviation below average.
The claims model shows comparable discrimination by patient age, sex, and race/ethnicity, as well as by hospital urban/rural status (Figure 3). Compared with the overall AUC of 0.60, the AUCs are comparable when the sample is restricted to male patients (0.60), female patients (0.60), white patients (0.62), nonwhite patients (0.60), rural hospitals (0.60), nonrural hospitals (0.60), patients ≥85 years of age (0.58), and patients <85 years of age (0.61).
We present a hospital 30-day risk-standardized all-cause readmission model for heart failure patients that can be used in performance measurement and quality improvement. The model uses administrative claims data from Medicare fee-for-service patients and produces results that are very similar to a model based on medical record data. The model reveals a clinically meaningful range of 30-day readmission rates among the nation’s hospitals.
Readmission after a hospitalization for heart failure is an important target for quality improvement. Heart failure accounts for ≈800 000 hospitalizations of Medicare fee-for-service patients annually. The risk of readmission is remarkably high within even a short time after discharge. Our study shows that 1 in 4 patients returns to the hospital within 30 days. Many studies have demonstrated the effectiveness of in-hospital and postdischarge interventions in reducing the risk of readmission, suggesting that hospitals and their partners have the ability to lower readmission rates.5–7,9–12,25 A 25% reduction in the readmission rate (ie, from 23.6% to 17.7%) could result in ≈50 000 fewer readmissions annually. Incentives for hospitals to reduce readmission rates are currently limited. In fact, hospitals that reduce readmissions could lower their Medicare revenues. Even institutions that participated in studies of successful interventions to reduce readmission often abandoned them soon after the study was over.26 Measuring readmission could promote sustained efforts to reduce readmission rates.
This model is consistent with the American Heart Association standards for models suitable for public reporting of outcomes in that it is transparent, excludes potential complications, uses an analysis appropriate for the organization of the data, and is validated against clinical data.15 It is patient oriented in that it includes readmissions to any acute care hospital, not just the discharging hospital, and minimizes the incentives for gaming readmission etiology as being unrelated to the index heart failure hospitalization. Inpatient and outpatient comorbidity information from the year before the index hospitalization captures the patients’ clinical conditions. The hierarchical model takes into account the structure of the data, with discharges clustered within hospitals, isolates variations due to quality differences, and accommodates hospitals with small volumes by appropriately reflecting their limited data in the estimates.
The agreement between the estimates from the claims model and from the medical record model suggests that despite the known limitations of administrative codes, the proposed model can stand in place of a model with more detailed clinical information for hospital-level profiling. The AUC and the explained variation of the model are modest, but the use of the model is to profile hospital performance on the basis of patient status at admission, not to develop a model with the best ability to predict outcomes for individual patients. In addition, the performance at the patient level is consistent with previously published models developed to predict readmission after a heart failure hospitalization, which also show modest c statistics for models based on administrative27 and medical record28 data. Furthermore, we excluded covariates that we would not want to adjust for in a quality measure, such as potential complications, patient race and socioeconomic status, and discharge disposition (eg, discharge to a skilled nursing facility). These characteristics may be associated with readmission and thus could increase the model performance to predict patient readmissions.13,29 However, they may reflect quality or system factors that should not be included in an adjustment that seeks to control for patient clinical characteristics while illuminating important quality differences.
Discrimination is lower in the heart failure readmission administrative and medical record models than in the heart failure mortality models.17 The risk of readmission may be much more dependent on the quality-of-care and system characteristics than on patient severity and comorbidity characteristics. The readiness for discharge, the proper medications, and the proper transition to the outpatient setting may be even more important for readmission than death. Intervention studies underscore this potential, finding substantial decreases in readmissions.5–7,9–12,25 Furthermore, some heart failure admissions may be discretionary, with higher rates in geographic areas with a greater supply of hospital beds.29
The approach has several limitations. The approach to calculating risk-standardized readmission rates is only validated with Medicare data. However, ≈75% of the patients hospitalized with heart failure are ≥65 years of age.30 In addition, we were unable to test the model with a Medicare managed care population because data are not currently available on those patients. Furthermore, the chart validation was conducted by state-level analysis because sample size was insufficient for hospital-level analysis. In addition, our modeling approach does not account for within-patient correlation of multiple index heart failure hospitalizations per patient because of computational limitations. However, a relatively small share of patients (9.1%) had multiple index heart failure hospitalizations. Although it is important to include all admissions for these patients so that hospital efforts to reduce readmissions among patients with multiple heart failure hospitalizations are fully reflected in the measure, we may have overstated the precision of individual covariates’ association with the risk of readmission. Furthermore, although not every readmission may be preventable, the all-cause readmission outcome minimizes incentives for gaming and best captures outcomes that are important to patients and amenable to quality improvement because interventions have generally shown reductions in non–heart failure as well as heart failure readmissions.
Finally, our approach focuses on 30-day readmission and not death. If a patient died within 30 days after discharge without a readmission, we coded the outcome as no readmission. We recognize that this has the effect of counting such a death as a “no event” readmission outcome. In addition, such patients have a shorter length of follow-up during which they are eligible to experience the readmission outcome. This approach is thus intended to be used in conjunction with the publicly reported heart failure mortality measure to reflect performance on readmission and death. We believe that it is important to retain these admissions in the measure as opposed to excluding them because they provide a more complete picture of quality of care and resource use, including for individuals at the end of life. Despite the shorter average survival time of 14 days, the readmission rate was higher for these admissions, at 36.9%, compared with the readmission rate of 22.7% for admissions in which the patient survived the full 30 days. Another possible approach to handling the competing outcome of death is to use a composite outcome of readmission or death. However, we believe that it is important to show the outcomes separately because the factors that predict readmission and death may differ and because, from a quality improvement perspective, it would not be possible to assess whether hospital performance was driven by readmission rates or mortality rates if a combined outcome were utilized.
In conclusion, this article presents an instrument to produce hospital-specific risk-standardized estimates of 30-day readmission rates after discharge for heart failure. These estimates can bring to light a perspective on the health system’s performance and facilitate tracking of improvement over time.
The authors thank Dima Turkmani, Maureen O'Brien, and Debra Chromik at the Colorado Foundation for Medical Care; Geoffrey Schreiner from the Center for Outcomes Research and Evaluation, Yale–New Haven Hospital; Angela Merrill and Eric Schone from Mathematica Policy Research; and Lein Han and Michael Rapp at CMS for their contributions to this work. CMS reviewed and approved the use of its data for this work and approved submission of the manuscript.
Sources of Funding
Drs Stauffer and Schuur were funded by the Department of Veterans Affairs during the time the work was conducted. The analyses on which this publication is based were performed under contract No. HHSM-500–2005-CO001C, entitled “Utilization and Quality Control Quality Improvement Organization for the State (Commonwealth) of Colorado,” funded by the CMS, an agency of the US Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Dr Krumholz reports that he is a consultant to United Healthcare. Dr Normand reports that she is funded by the Massachusetts Department of Public Health to monitor the quality of care after cardiac surgery or percutaneous coronary intervention. The other authors report no conflicts.
Medicare Payment Advisory Commission (MedPAC). Promoting greater efficiency in Medicare. Available at: http://www.medpac.gov/documents/Jun07_EntireReport.pdf. June 2007. Accessed August 29, 2008.
Centers for Medicare & Medicaid Services. Medicare ranking for all short-stay hospitals by discharges fiscal year 2005 versus 2004. Available at: http://www.cms.hhs.gov/MedicareFeeforSvcPartsAB/Downloads/SSDischarges0405.pdf. Accessed August 29, 2008.
Gonseth J, Guallar-Castillon P, Banegas JR, Rodriguez-Artalejo F. The effectiveness of disease management programmes in reducing hospital re-admission in older patients with heart failure: a systematic review and meta-analysis of published reports. Eur Heart J. 2004; 25: 1570–1595.
National Quality Forum. National Quality Forum endorses consensus standards for quality of hospital care. Available at: http://www.qualityforum.org/news/releases/051508-endorsed-measures.asp. Accessed August 29, 2008.
Krumholz HM, Brindis RG, Brush JE, Cohen DJ, Epstein AJ, Furie K, Howard G, Peterson ED, Rathore SS, Smith SC Jr., Spertus JA, Wang Y, Normand S-LT. Standards for statistical models used for public reporting of health outcomes: an American Heart Association Scientific Statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group; cosponsored by the Council on Epidemiology and Prevention and the Stroke Council endorsed by the American College of Cardiology Foundation. Circulation. 2006; 113: 456–462.
Centers for Medicare & Medicaid Services. Hospital Compare: a quality tool for adults, including people with Medicare. Available at: http://www.hospitalcompare.hhs.gov. Accessed August 29, 2008.
Krumholz HM, Wang Y, Mattera JA, Wang Y-F, Han LF, Ingber MJ, Roman S, Normand SL. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006; 113: 1693–1701.
Krumholz HM, Wang Y, Mattera JA, Wang Y-F, Han LF, Ingber MJ, Roman S, Normand SL. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006; 113: 1683–1692.
QualityNet. Condition Category-ICD-9-CM Crosswalk. Available at http://www.qualitynet.org/dcs/ContentServer?cid=1182785083979&pagename=QnetPublic%2FPage%2FQnetTier3&c=Page. Accessed August 29, 2008.
Deleted in proof.
Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001.
Koelling T, Johnson M, Cody R, Aaronson K. Discharge education improves clinical outcomes in patients with chronic heart failure. Circulation. 2005; 111: 179–185.
AHA. 2007 Statistical Fact Sheet. Populations: older Americans and cardiovascular disease. Available at: http://www.americanheart.org/downloadable/heart/1168618073572OLDER07.REV.pdf. Accessed August 29, 2008.
Readmission soon after hospital discharge is an expensive and often preventable event for patients with heart failure. We present a model approved by the National Quality Forum for the purpose of public reporting of hospital-level readmission rates by the Centers for Medicare & Medicaid Services. We developed a hierarchical logistic regression model to calculate hospital risk-standardized 30-day all-cause readmission rates for patients hospitalized with heart failure. The model was derived with the use of Medicare claims data for a 2004 cohort and validated with the use of claims and medical record data. The unadjusted readmission rate was 23.6%. The final model included 37 variables, had discrimination ranging from 15% observed 30-day readmission rate in the lowest predictive decile to 37% in the upper decile, and had a c statistic of 0.60. The 25th and 75th percentiles of the risk-standardized readmission rates across 4669 hospitals were 23.1% and 24.0%, respectively. State-level adjusted readmission rates developed with the use of the claims model are similar to rates produced for the same cohort with the use of a medical record model (correlation, 0.97; median difference, 0.06 percentage points). This claims-based model of hospital risk-standardized readmission rates for heart failure patients produces estimates that can serve as surrogates for those derived from a medical record model.
The online-only Data Supplement is available with this article at http://circoutcomes.ahajournals.org/cgi/content/full/1/1/29/DC1.