A Prediction Model to Identify Patients at High Risk for 30-Day Readmission After Percutaneous Coronary Intervention
Background—The Affordable Care Act creates financial incentives for hospitals to minimize readmissions shortly after discharge for several conditions, with percutaneous coronary intervention (PCI) to be a target in 2015. We aimed to develop and validate prediction models to assist clinicians and hospitals in identifying patients at highest risk for 30-day readmission after PCI.
Methods and Results—We identified all readmissions within 30 days of discharge after PCI in nonfederal hospitals in Massachusetts between October 1, 2005, and September 30, 2008. Within a two-thirds random sample (Developmental cohort), we developed 2 parsimonious multivariable models to predict all-cause 30-day readmission, the first incorporating only variables known before cardiac catheterization (pre-PCI model), and the second incorporating variables known at discharge (Discharge model). Models were validated within the remaining one-third sample (Validation cohort), and model discrimination and calibration were assessed. Of 36 060 PCI patients surviving to discharge, 3760 (10.4%) patients were readmitted within 30 days. Significant pre-PCI predictors of readmission included age, female sex, Medicare or State insurance, congestive heart failure, and chronic kidney disease. Post-PCI predictors of readmission included lack of β-blocker prescription at discharge, post-PCI vascular or bleeding complications, and extended length of stay. Discrimination of the pre-PCI model (C-statistic=0.68) was modestly improved by the addition of post-PCI variables in the Discharge model (C-statistic=0.69; integrated discrimination improvement, 0.009; P<0.001).
Conclusions—These prediction models can be used to identify patients at high risk for readmission after PCI and to target high-risk patients for interventions to prevent readmission.
On October 1, 2012, the Center for Medicare and Medicaid Services (CMS) implemented 2 provisions of the Affordable Care Act that focus new attention on preventable readmissions. In particular, Medicare reimbursements will be adjusted based on 30-day readmission rates for acute myocardial infarction (MI), pneumonia, and heart failure.1 The Affordable Care Act included these provisions to provide hospitals incentives to improve the transition of care from the inpatient to outpatient settings and to reduce the $26 billion cost of hospital readmissions to Medicare.2 A substantial number of Medicare patients (14.6%) are readmitted to the hospital after percutaneous coronary intervention (PCI), and readmission rates between hospitals vary substantially.3,4
WHAT IS KNOWN
Hospital readmission after percutaneous coronary intervention adds substantially to medical expenditures.
Rates of readmission after percutaneous coronary intervention vary between hospitals and are associated with particular patient characteristics.
WHAT THE STUDY ADDS
This article derives a prediction model and point-based risk scoring system for predicting an individual patient’s risk of 30-day readmission after percutaneous coronary intervention that clinicians can implement at the bedside.
Procedural and patient factors discovered during hospitalization add minimally to prehospital patient characteristics in predicting readmission risk; risk for 30-day readmission can be predicted relatively well before the procedure.
The identification of patients at high risk for readmission after PCI is a logical first step in the process of developing and implementing interventions to reduce readmission rates and improve hospital quality of care. Nevertheless, despite the intense interest in PCI readmissions and potential plans for adjusting hospital compensation based on readmission rates, there is limited understanding of how to prognosticate and mitigate risk for readmissions after PCI. Previous studies have identified risk factors associated with 30-day readmission after PCI.3,5–7 However, these studies have been limited by a number of factors, including being limited to specific age groups3 or being derived from single centers.6 In addition, none has developed and validated a prediction model and associated risk score that could be implemented by clinicians and hospitals to quantify risk of readmission and, thus, help identify high readmission risk patients before discharge.
In that context, we sought to develop distinct pre- and postprocedural prediction models for 30-day all-cause readmission in a large, representative population of patients undergoing PCI and derive a risk score useful to clinicians.
The Massachusetts Department of Public Health collects data on all PCI admissions performed in adults ≥18 years of age at all nonfederal Massachusetts hospitals. The data are collected by trained hospital personnel using the National Cardiovascular Data Registry (NCDR) Cath-PCI data collection instrument and are submitted electronically to the Massachusetts Data Analysis Center at Harvard Medical School. Definitions for the data elements of the Cath-PCI registry are available at http://www.ncdr.com/WebNCDR/Elements.aspx. Selected covariates and outcomes are audited, adjudicated, and verified. Specifically, all cases for which the patient died in the hospital during the PCI admission, and all cases with variables known to be strongly predictive of adverse events, including cardiogenic shock, emergency or salvage procedure, ST-elevation MI, left main coronary stenosis of >50%, and patients who are designated as compassionate use or exceptional risk cases, are adjudicated by a committee of 16 physicians and 2 data managers. To obtain data on patients subsequent to discharge, including information on readmissions, we then linked data from the Massachusetts Data Analysis Center to hospital-discharge billing data collected by the Massachusetts Division of Health Care Finance and Policy. Matching of these data sets was performed on the basis of the following hierarchical sets of criteria: (1) hospital medical record number, discharge date, and hospital; (2) hospital medical record number, admission date, and hospital; and (3) admission date, discharge date, hospital, date of birth, zip code, and sex. We initially considered all PCI admissions in nonfederal hospitals in Massachusetts between October 1, 2005, and September 30, 2008. We considered admissions during which the patient received multiple PCIs as one PCI admission. Then we excluded PCIs in patients who could not be matched between the 2 data sets (n=2726), were not residents of Massachusetts (n=2998), had incomplete data (n=57), or did not survive the index hospitalization (n=589); 36 060 unique PCI admissions remained for inclusion in this study. This study was approved by the Research and Data Access Review Committee, which is the Institutional Review Board at the Massachusetts Department of Public Health.
Outcomes and Covariates
Readmissions within 30 days of discharge were identified by linking data from hospital-discharge billing data collected by the Massachusetts Division of Health Care Finance and Policy. Readmissions at separate facilities occurring within 1 calendar day of discharge were considered to be transfers and linked to form single episodes of care. Although planned readmissions for nontarget vessel revascularization (staged procedures) were not clearly identified within our data set, we adopted the identical algorithm used in the endorsed CMS performance measure. Specifically, we identified staged PCI admissions as hospitalizations during which (1) PCI was performed, and (2) the principal diagnosis was not an acute cardiovascular condition. We defined the following diagnoses as acute cardiovascular conditions: congestive heart failure (402.01, 402.11, 402.91, 404.01, ICD-9-CM 404.03, 404.11, 404.13, 404.91, 404.93, 428.xx), acute MI (410.xx except 410.x2), unstable angina (411.xx), arrhythmia (427.xx, except 427.5), or cardiac arrest (427.5). Readmissions during which PCI was performed and for which the primary discharge diagnosis did not include any of the listed codes during the readmission hospitalization were thus classified as staged (n=709 out of 796 readmissions with PCI; 89.0%). Rehospitalizations for staged PCIs were not considered to be readmissions for this analysis. In addition, we produced additional models in which we did not account for potential staged readmissions, as well as those in which all rehospitalizations associated with repeat revascularization were not considered to be readmissions. Because model parameters, coefficients, and predictive performance were unchanged in these sensitivity analyses, only the primary analyses are presented.
We considered a wide array of candidate variables for incorporation into the prediction model based on clinical relevance. These included sociodemographic factors (age on admission, sex, race, smoking status, and insurance type, including Medicare, health maintenance organizations, and state insurance, which includes Medicaid, Massachusetts free care, Commonwealth Care Alliance, workers compensation, and other forms of state government insurance), presentation status variables at index admission (source of admission, elective versus urgent versus emergent presentation, shock on admission, presentation in heart failure), and clinical characteristics (Canadian Cardiovascular Society/New York Heart Association class, previous MI, peripheral vascular disease, chronic lung disease, previous PCI, previous coronary artery bypass graft, previous congestive heart failure, hypertension, diabetes mellitus [with and without insulin use], and glomerular filtration rate [GFR]). For the Discharge model, we additionally considered a number of angiographic, PCI, and post-PCI variables, including length of stay, access site and nonaccess site bleeding, and vascular complications, discharge disposition (home or another facility), stent type (drug-eluting stent or bare metal stent), attempt at closure with a vascular closure device, use of a direct thrombin inhibitor, high-risk lesion, bifurcation lesion, number of diseased vessels, periprocedural MI, any other complications (including postprocedural cardiogenic shock, congestive heart failure, stroke, tamponade, thrombocytopenia, contrast reaction, renal failure, or subsequent emergency PCI), and whether medications from the following categories were prescribed at discharge: β-blockers, statins, angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, and platelet inhibitors. Missingness of data was <0.1% for all variables apart from GFR and race. For GFR (3.5% missing), we elected to create a category of “not measured” for this variable. For race (5.8% missing), missing data were combined with the “other” race category. Sensitivity analyses were performed in which missing race was included as a distinct category, and results were substantively unchanged (data not shown).
We developed 2 separate multivariable models, 1 to predict risk of readmission at the moment of initial presentation (pre-PCI model) and the other to predict risk of readmission at the moment of discharge (Discharge model). For the pre-PCI model, only sociodemographic, presentation status, and clinical characteristics were eligible for inclusion. All variables assessed before and after PCI up until discharge were considered for the Discharge model. As such, a pre-PCI model has the potential to affect actual decisions about whether or how to pursue PCI, improve admission planning, and enhance patient understanding of the risks for the procedure. The Discharge model, by contrast, could influence decisions about how to prevent readmission at discharge, for example, by ensuring the availability of additional clinical resources for a high-risk patient after leaving the acute care setting.
For both the pre-PCI and Discharge models, we developed these models within a two-third random sample (Developmental cohort), leaving the remaining one-third for validation (Validation cohort). To facilitate clinical use and implementation of the model in routine clinical care, a reduced model was estimated using logistic regression according to the method of Harrell.9 In this approach, the total adjusted variability (R2) of an initial model considering all candidate variables is first estimated. Then sequential backward elimination is performed, with estimation of total model adjusted R2 at each step. When the adjusted R2 falls <95% of the R2 of the initial model, the selection procedure is terminated, with remaining variables retained in the model. We used generalized estimating equations to account for known clustering of readmission risk within hospitals.4 The models were then applied to the Validation cohort. Model discrimination was assessed via measurement of the C-statistic, and plots of observed versus predicted rates of readmission within the Validation cohort were generated to assess calibration. The improvement in discrimination of the Discharge model compared with the pre-PCI model was assessed by calculation of the integrated discrimination improvement.10 The integrated discrimination improvement provides a quantitative summary that measures the average absolute improvement in individual risk predictions from the inclusion of additional variables to a prediction model.
Finally, for the pre-PCI model, we created a simplified risk score to assist with bedside implementation of the prediction model. Point totals for variables were assigned based on model coefficients using methods described previously.11
All statistical analyses were performed by the Massachusetts Data Analysis Center at Harvard Medical School using the SAS Version 9.2.
Clinical characteristics of patients in the study, stratified by 30-day readmission, are shown in Table 1. Overall, 10.4% of patients were readmitted within 30 days. In bivariate comparisons, readmitted patients were more likely to be elderly (68.1 versus 64.3 years old; P<0.001), female (38.3% versus 29.6%; P<0.001), or black (4.0% versus 2.4%; P<0.001). Patients with Medicare or state-financed insurance were more likely than patients with private insurance or health maintenance organization insurance to be readmitted (P<0.001).
Among clinical characteristics, patients with a history of heart failure, previous MI, diabetes mellitus, cerebrovascular disease, peripheral vascular disease, chronic lung disease, and previous PCI were all more likely to be readmitted than patients without those diagnoses (P<0.001 for all). Patients presenting with acute coronary syndromes or those undergoing urgent or emergent PCI were also more likely to be readmitted (<0.001 for both). Angiographic variables associated with early readmission included multivessel coronary disease, target lesion in a bypass graft (P<0.001), and high-risk lesions (P<0.001).
Model Development and Validation
After multivariable adjustment with backward elimination, 10 variables were retained in the pre-PCI model: age, sex, insurance type, GFR category, current congestive heart failure as well as history of congestive heart failure, chronic lung disease, peripheral vascular disease, cardiogenic shock at presentation, admission status, and history of previous coronary artery bypass graft surgery (Figure 1A). In the Discharge model, additional retained variables included β-blocker prescribed at discharge, post-PCI vascular or bleeding complications, discharge location, race, diabetes mellitus status and modality of treatment, the use of a drug-eluting stent during the index procedure, and extended length of stay (Figure 1B). β-weights and standard error for models are included in the online-only Data Supplement. To assess for effect modification between renal function and contrast volume used, we reran the Discharge model with an interaction term between GFR and contrast volume predicting 30-day readmission. Because the interaction term was not statistically significant, only the primary results are presented here.
The distribution of predicted rates of 30-day readmission at discharge varied widely among patients, with the 95% observed range from 3.3% to 26.0%. Model discrimination was moderate in the validation data set for the pre-PCI model (C-statistic=0.68) and only marginally improved in the Discharge model (C-statistic=0.69; integrated discrimination improvement=0.009; P<0.001). The comparison of observed 30-day readmission rates in the validation sample compared with the predicted risk of 30-day readmission did not show evidence of overfitting in both the pre-PCI and Discharge models (Figure 2).
Creation of a Risk Score
We created a simplified risk score to predict 30-day readmission risk to facilitate clinical use of the model11 (Table 2). Because the pre-PCI model was more parsimonious, had similar discrimination to the Discharge model, and could be used early in the course of a patient’s admission, we chose to create this risk score based on the pre-PCI model alone. Using variables known at the time of the procedure, this model can discriminate among patients at low risk (<9%), intermediate risk (10% to 21%), and high risk (>24%) of 30-day readmission after PCI. We applied this model to the Validation cohort, which showed similar discrimination (C-statistic=0.67). Using the point score, 415/6225 (6.7%) of the low-risk patients, 615/3859 (15.9%) of the intermediate-risk patients, and 218/604 (26.5%) of the high-risk patients were actually readmitted. A Web-based calculator that enables the clinical use of this model is available at http://www.massdac.org/riskcalc_readm30day.
To address the pressing need to identify patients at high risk for readmission after PCI, we developed and validated 2 prediction models in a comprehensive sample of PCIs performed in Massachusetts. Being able to identify high-risk patients should be useful to clinicians and hospitals in risk stratifying patients on the basis of readmission risk and potentially as a foundation for interventions to reduce readmission rates in high-risk patients. To our knowledge, our risk score is the first for 30-day readmission after PCI and may be useful to clinicians, hospital administrators, or investigators designing interventions to reduce readmission after PCI.
Our rationale for developing 2 distinct risk models stems from the different potential applications and clinical use of these models. In particular, at the beginning of a hospital admission (before PCI), a clinician may want to identify patients at high risk of readmission to direct interventions to reduce the risk of readmission from the outset, as well as engage patients and increase transparency of the procedural risks. For example, identifying a particular patient at high risk of readmission may prompt more intensive case management attention or in some cases might influence the decision to perform PCI. At the end of hospitalization, identifying a patient at high risk, for example, might lead to enhanced predischarge counseling, involvement of home services, altering discharge disposition to rehabilitation services versus home, or shortening the time interval between discharge and subsequent follow-up. Recently, we have demonstrated that even after risk adjustment, hospitals vary significantly with respect to rates of patients readmitted within 30 days after PCI, such that a large number of hospitals are likely to suffer financial consequences unless efforts to identify high-risk patients and prevent readmissions are successfully implemented.4
Several studies examining predictors of readmission have recently been described.3,5–7 Our study confirms some risk factors for discharge and identifies other, previously unrecognized, risk factors. A retrospective review of 315 241 Medicare patients undergoing PCI in 2005 demonstrated an association with female sex, older age, diabetes mellitus, heart failure, renal failure, previous ischemic heart disease, and acute MI.3 A retrospective review of 15 498 PCI admissions at Saint Mary’s Hospital in Rochester, Minnesota, from January 1998 through June 2008 identified female sex, Medicare insurance, less than high school education, unstable angina, cerebrovascular incident or transient ischemic attack, moderate-to-severe renal disease, chronic obstructive pulmonary disease, peptic ulcer disease, metastatic cancer, and a length of stay >3 days as predictive of readmission within 30 days in multivariable analysis.6 In a study of 40 093 PCI patients in New York State, 12.4% of whom were readmitted, Hannan et al5 identified similar predictors of readmission, including age, depressed ejection fraction, multivessel disease, recent MI, peripheral vascular disease, chronic obstructive pulmonary disease, diabetes mellitus, and worsening renal failure or dialysis. Ricciardi et al7 identified female sex, higher age, body mass index, severe comorbid condition, acute coronary syndrome, higher number of significant lesions, fewer number of lesions treated, and emergency PCI status as all predictive of 30-day cardiac readmission.
Our study is unique from these previous studies in several ways. First, unlike several previous studies,3,6 our study represented a broadly captured population of patients from multiple centers across the complete age range and insurance spectrum. Second, in contrast to previous studies, ours is the first to separately develop and validate a prediction model and create a clinical risk score that facilitates the implementation of the model to identify individual patients who are at high risk of readmission, a critical step in the goal to reduce hospital readmission rates. Newly designed interventions that have the potential to limit preventable readmissions, reduce healthcare costs, and improve care may have the greatest impact on this vulnerable population. Finally, in performing separate analyses of pre-PCI and Discharge variables and showing similar discrimination between these, our study is the first to demonstrate that a clinician can predict risk for 30-day readmission with only variables known before catheterization or PCI, which may facilitate discharge planning early in the course or even before a PCI admission.
We have identified that prescription of a β-blocker at discharge is associated with reduced likelihood of repeat readmission within 30 days after a PCI admission. Considering a large proportion of readmissions after PCI are related to chest pain, β-blockade may directly reduce hospital readmission by reducing myocardial oxygen demand and subsequent ischemia. Alternatively, β-blockade may be associated with another, causal factor that reduces 30-day readmission. For example, patients with more stable blood pressure might have been more likely to receive a β-blocker and less likely to experience readmission. The addition of a β-blocker may also have characterized a high-quality, careful discharge, which in turn reduced the likelihood of readmission.
Our findings should be interpreted in the context of several potential limitations to our analysis. The models did not have exceptional discrimination. However, although the discrimination of our models was not high, the models compared favorably with previously published models to predict readmission risk. A review of 30 models to predict hospital readmission in a variety of medical conditions revealed that only 7 could identify high-risk patients early in a hospitalization, and of those, the C statistic values ranged between 0.56 and 0.72.12 Factors not reflected in patient characteristics, such as the quality of the transitions from inpatient to outpatient care, render C statistics for readmissions low generally. In addition, although our data set included a broad spectrum of patients representing all age ranges and insurance types, our analysis was limited to Massachusetts hospitals and may not be completely generalizable to other hospital settings.
Next, although we attempted to identify and exclude staged procedures from the model, we were not able to validate the accuracy of the algorithm used to define these readmissions. However, we purposefully adopted a definition of staged procedures consistent with that used in the proposed CMS performance measure to ensure that our model was aligned with future models that may be used to assess hospital quality. Hospitals trying to minimize 30-day readmission after PCI and regulators evaluating the performance metric are likely focus on the cases considered readmissions by CMS, even when those readmissions are planned. In addition, it has been observed in routine practice that the vast majority of repeat procedures within the first month after PCI are staged procedures, consistent with our findings, and suggesting that nonstaged revascularizations have a minimal impact on PCI-related readmissions.13 Also, although our models can categorize patients into predicted risk groups for readmission, they do not distinguish between preventable or unpreventable readmissions. We believe that effective interventions to reduce preventable readmission should focus on patients at high risk of readmission. As such, although our model does not predict preventability, it can identify high-risk groups of patients for whom preventing readmission has the potential for highest effectiveness. Future efforts should be devoted to developing methods for specifically identifying those admissions that could have been prevented through improved quality of care. Other important areas of future investigation include modeling risk factors for the competing risks of 30-day readmission and 30-day mortality. We deliberately chose to model 30-day readmission, without consideration for mortality, to be aligned with the CMS-proposed quality measure. The risk of death within 30 days that is not associated with hospitalization could, thus, be a competing risk for patients that was not accounted for. Given the low rate of death not associated with readmission (<0.3% in this study), we do not believe that this would have a strong influence on our study results.
Finally, although we attempted to make the models parsimonious because readmission risk is multifactorial and not dependent on a small number of strong predictors, the models still include a large number of variables (11 and 19) to retain sufficient predictive ability. However, we believe that Web-based tools and other information technologies can support their implementation in routine clinical care.
The underlying use of these novel clinical models is crucial in the context of rising interest and accountability in measuring hospital readmission as a quality metric, linked to reimbursement. Future research is needed to identify interventions that can cost-effectively mitigate the rate of readmission. These models to identify higher-risk patients provide an important step in designing such strategies.
We appreciate the contributions of Dr Daniel Shivapour to this work, and we thank Dr Frank Harrell for helpful discussions.
Sources of Funding
This work was supported in part by the Massachusetts Department of Public Health and a grant from the American Heart Association (12CRP9010016).
Kenneth Rosenfield is a recipient of the following: research grant, Abbott Vascular, <$10 000; Bard Peripheral Vascular, <$10 000; Ownership Interest, Lumen Biomedical, <$10 000; Medical Stimulation Corp, <$10 000; VIVA Physicians Association, ≥$10 000; Consultant/Advisory Board, Abbott Vascular, <$10 000; Angioguard, <$10 000; Boston Scientific, <$10 000; Complete Conference Manager, <$10 000; Harvard Clinical Research Institute, <$10 000. Laura Mauri is a recipient of the following: research grant, Abbott, ≥$10 000; Boston Scientific, ≥$10 000; Cordis, ≥$10 000; Medtronic, ≥$10 000; Eli Lilly, ≥$10 000; Daiichi Sankyo, ≥$10 000; Bristol Myers Squibb, ≥$10 000; sanofi-aventis, ≥$10 000; Consultant/Advisory Board: Cordis, <$10 000; Medtronic, ≥$10 000. John Spertus is a recipient of the following: research grant, American College of Cardiology Foundation, >$10 000; Equity, Health Outcomes Sciences, <$10 000. The other authors report no conflicts.
The online-only Data Supplement is available at http://circoutcomes.ahajournals.org/lookup/suppl/doi:10.1161/CIRCOUTCOMES.111.000093/-/DC1.
This article was handled independently by Philippe Gabriel Steg, MD, as Guest Editor. The editors had no role in the evaluation of the article or in the decision about its acceptance.
- Received January 10, 2013.
- Accepted June 7, 2013.
- © 2013 American Heart Association, Inc.
- 2.↵U.S. Department of Health and Human Services. Administration implements new health reform provision to improve care quality, lower costs. http://www.healthcare.gov/news/factsheets/2011/04/valuebasedpurchasing04292011a.html. Accessed January 9, 2013.
- Yeh RW,
- Rosenfield K,
- Zelevinsky K,
- Mauri L,
- Sakhuja R,
- Shivapour DM,
- Lovett A,
- Weiner BH,
- Jacobs AK,
- Normand SL
- Hannan EL,
- Zhong Y,
- Krumholz H,
- Walford G,
- Holmes DR Jr.,
- Stamato NJ,
- Jacobs AK,
- Venditti FJ,
- Sharma S,
- King SB III.
- Ricciardi MJ,
- Selzer F,
- Marroquin OC,
- Holper EM,
- Venkitachalam L,
- Williams DO,
- Kelsey SF,
- Laskey WK
- 8.National Quality Forum. National Voluntary Consensus Standards for Patient Outcomes: Measure Summary, Measure OT1-008-09. http://www.qualityforum.org/Projects/n-r/Patient_Outcomes/OT1-008-09_Supplement,aspx. Accessed January 9, 2013.
- Harrell FE
- Stolker JM,
- Cohen DJ,
- Kennedy KF,
- Pencina MJ,
- Lindsey JB,
- Mauri L,
- Cutlip DE,
- Kleiman NS