Inter-Relationship of Procedural Mortality Rates in Vascular Surgery in England
Retrospective Analysis of Hospital Episode Statistics From 2005 to 2010
Background—Wide variations in vascular surgical outcomes have been demonstrated in England. The objective of this study was to determine whether risk-adjusted postoperative mortality rates for elective and emergency vascular surgical procedures were inter-related.
Methods and Results—A retrospective observational study using National Health Service administrative data on adult patients undergoing elective or emergency vascular surgery from 2005 to 2010. The 10 procedures covered the broad spectrum of workload for a vascular surgical service. The primary outcome measure was in-hospital mortality, and secondary outcomes were 30-day and 1-year mortality. Data were risk-adjusted using multilevel modeling. Analyses comprised a 2-level basket designed to evaluate variations in outcome and whether the outcome of each procedure could be predicted by the composite outcome of all other procedures. A total of 116 596 vascular surgical procedures were performed across 166 providers. For 9 of 10 procedures, there were hospitals lying outside 95% control limits for ≥1 mortality outcome. The key finding was that ≥1 risk-adjusted mortality outcome for any 1 of the 9 vascular surgical procedures could be predicted by the aggregated within provider performance of the other vascular surgical procedures combined.
Conclusions—Hospital-level risk-adjusted mortality for both elective and emergency vascular procedures in England varies considerably, and providers were globally high or low performers. The data should be made available to patients, relatives, and the purchasers of services to drive improvements in the provision of vascular surgical services.
- health services research
- outcome assessment (health care)
- quality of healthcare
One major current focus of all first-world healthcare systems is to constantly improve the quality of healthcare delivery.1,2 The need for such a focus is highlighted through the media in their reporting of high-profile examples of healthcare services failing to provide an acceptable standard of care.3,4 Two proven strategies to mitigate against these failings are the reporting of clinical outcomes and the implementation of well-designed quality improvement frameworks based around outcomes.1,5
Both strategies rely on the routine measurement of outcomes that represent the most valid end-point of clinical care pathways, with acceptance from clinicians, commissioners, and patients.6 Although many metrics have been proposed, in specialties such as vascular surgery where the patients have multiple medical comorbidities and are elderly and the procedures complex, mortality remains a widely studied and valid clinical end-point.7,8
Several studies have demonstrated wide variations between hospitals in mortality for individual vascular surgical procedures.9–12 Although the underlying reasons for these variations are not fully understood, it is likely that institutional infrastructure and processes of care have a bearing on subsequent outcomes.13,14 Furthermore, it is plausible that the outcomes of different procedures might be inter-related, lacking specificity. This would manifest as hospitals with low mortality rates for certain operations demonstrating lower mortality rates for other procedures.15,16 If this were demonstrated, the results could directly inform the commissioning and purchasing of services from different providers.
The objective of this study was to determine whether risk-adjusted mortality rates for both elective and emergency vascular surgical procedures were inter-related at hospital level through an analysis of perioperative and longer term mortality rates.
The data source for this study was the Hospital Episode Statistics (HES) data warehouse for the period April 1, 2005, to March 31, 2010. This period was chosen as a balance between deriving a data set of adequate size to power the study and ensuring that the observed outcomes reflected contemporary practice.
HES are administrative data, which contain patient-level information on every admission to public hospitals in England. The data set can be considered an inclusive record of National Health Service hospital activity in England because every hospital is required to submit a minimum data set to the Department of Health. More than 12 million new records are added each year. Supplementing in-hospital data, information on outpatient attendance, and Emergency Department attendance have been included since 2003 and 2008, respectively.17 Sequential systematic reviews of coding accuracy in HES covering the period from 1989 to 2010 have found procedural and diagnostic coding to be sufficiently reliable for use in observational research.18,19 A recent systematic review has also demonstrated increasing use of HES data in outcomes research.20 Information in several domains is available, including medical information, social demographics, and geographical data.17 In this study, the HES were linked to registry data from the Office of National Statistics to provide longer term and out-of-hospital mortality data.21,22
Ten procedure groups chosen were those that appropriately reflected contemporary vascular and endovascular surgical practice: elective open infrarenal abdominal aortic aneurysm (AAA) repair, emergency open infrarenal AAA repair, elective infrarenal endovascular aneurysm repair (EVAR), emergency EVAR, elective carotid endarterectomy (CEA), emergency CEA, elective lower extremity arterial bypass (LEAB), emergency LEAB, elective lower extremity amputation, and emergency lower extremity amputation. The Office of Procedures, Census and Surveys version 4 coding system was used to identify the procedural codes relating to the vascular surgical procedures of interest based on analyses in previous studies (Table 1).9–12,23
All elective and emergency admissions in adult patients (>17 years of age) were included in the case identification. Data were extracted on age, sex, and social deprivation indices (based on residential address) and comorbidity.20 Patient-level comorbidity data were extracted and used to generate a validated comorbidity risk score as previously described.24 The hospital where the procedure was performed was defined as the index hospital.
The primary outcome measure was in-hospital all-cause mortality. This was defined as death occurring after admission to an index hospital and before discharge from either the index hospital or any subsequent receiving hospital in the cases where the patient was transferred from the index hospital.
Secondary outcomes were 30-day all-cause mortality, defined as any death within 30 days of the index operation, and 1-year all-cause mortality, defined as any death within 365 days of the index operation.
It is important to note that the definition of in-hospital mortality means that in cases where patients had long hospital stays before death, in some cases death occurred >30 or 365 days after surgery. The importance of this is that the subsequent mortality rates for in-hospital mortality could plausibly be higher than those for 30-day mortality.
Established methodology was used to clean the data set.9,11,25 Entries with obvious coding errors, duplicated data, and missing data fields (such as operation dates) were excluded (representing 1.55% of the entire data set for this study). Other than EVAR, which was identified using a validated and nationally agreed on composite of codes before the introduction of a specific Office of Procedures, Census and Surveys designation, for each admission only the primary procedural code field was considered to exclude cases that developed in-hospital as complications of medical care and to prevent instances where multiple procedures on the same admission could contribute to the outcomes of >1 operation. Where a patient underwent 1 of the selected procedures more than once in the 5-year study period, only the earliest procedure was used in analyses.
Interhospital transfers were accounted for by linking discrete admissions into superspells but assigning the outcome to the hospital in which the surgery occurred.20 This provides a more accurate analysis of in-hospital death, as when transfers occur postoperatively, a patient may die in the hospital in which he or she is convalescing.
For each procedure, in-hospital death, 30-day death, and 1-year death were modeled using logistic regression adjusted for patient age, sex, RCS Charlson score, and social deprivation indices and stratified by HES year in accordance with published methodology.26 Hierarchical modeling was used with patient factors forming first-level predictors and a random hospital effect at the second level. Each patient’s expected mortality was calculated by summing the product of the parameter estimates from the logistic regression models with the relevant covariate value. The parameter estimates used to calculate the expected numbers of deaths at each hospital were obtained only from the data from other hospitals.
After data selection, extraction, cleaning, and risk adjustment, the analyses for this study were stepwise with 2 distinct stages: (1) to confirm the presence of variations in outcome among the procedures in this data set, and (2) to determine whether the outcomes of one procedure could be predicted by the collective outcomes of the other vascular surgical procedures.
Confirming Variability in Mortality: Variations in mortality have been described in previous studies.9–12,23 It was necessary to confirm that similar variations continued to exist between hospitals using the current data before commencing the subsequent analyses. The risk estimates from each individual patient and within each hospital were used to calculate the expected number of deaths for each condition at each hospital using multilevel modeling. For each procedure, the discrepancy between the expected and observed mortality in each hospital was quantified and tested using standardized funnel plots. Funnel plots are graphical tools that allow the deployment of statistically defined control limits around a sample of data and have been validated for use in the monitoring of hospital mortality data.27,28
A Poisson distribution modeled the expected divergence between observed and expected mortality. A statistically significant divergence was reported when it exceeded the 95% confidence interval of the Poisson distribution.29 For each outcome and procedure, hospitals were assigned a standardized performance measure based on the difference between its observed and expected death rates.
Outcome Inter-Relationship Quintile Analysis: A comparison was made between the outcome for each of the 10 procedures in turn against the aggregated outcome for the remaining 9 procedures (eg, elective open AAA repair versus an aggregate of all 9 other procedures) based on published methodology.16 Hospitals were ranked according to their standardized performance measure for the remaining 9 procedures and placed into 5 groups of roughly equal numbers of patients (aggregate mortality quintiles) such that hospitals with the highest death rates relative to expectations were in the highest numbered group (quintile 5) and those with the lowest death rates in quintile 1. Individual hospital data were retained in the analysis, giving estimates of spread within each quintile rather than using a superaggregated point estimate. There was no minimum volume requirement for inclusion in analyses, and, consequently, all cases from all hospitals in the study window were included.
Risk-adjusted procedure-specific mortality was calculated for each of the 10 procedures and for each of the 3 outcomes (in-hospital death, 30-day death, and 365-day death) and plotted against each quintile of aggregate mortality for each outcome. The relationship between procedure-specific and aggregate mortality was tested using logistic regression.
Statistical analysis was performed using SAS version 9.2 (SAS Institute, Cary, NC).
Application to the local Research Ethics Committee confirmed that ethical approval was not required.
There were 116 596 vascular surgical procedures across 166 hospitals during the 5-year study period. Demographic and crude mortality outcomes data for each of the procedures are given in Table 2.
The key findings of the study were the following:
Significant variations in mortality continue to exist between English hospitals across the range of common vascular surgical procedures;
The outcomes of specific vascular surgical procedures were significantly interdependent on the combined outcomes of all other vascular surgical procedures within individual hospitals;
High long-term mortality rates were observed for emergency vascular procedures, with interhospital variability persisting even at 1-year postsurgery.
Variability in Mortality
Excess variability for death rates was observed for both the elective and emergency vascular surgical procedures. Not all procedures demonstrated variability at all time points, but those variations observed were statistically significant. Elective and emergency open AAA repair, LEAB, and lower limb amputation showed highly significant variations in outcome at every time point. Variability was the greatest for elective open AAA repair with 10.4%, 9.0%, and 8.2% of hospitals lying above the control limits for in-hospital, 30-day, and 1-year death. Elective EVAR demonstrated excess variability for in-hospital and 1-year death (Table 3). Examples of funnel plots for elective open AAA repair and emergency lower limb amputation in-hospital mortality are presented in Figures 1 and 2. All funnel plots are presented in Appendix 1 (in-hospital mortality), Appendix 2 (30-day mortality), and Appendix 3 (1-year mortality; see online-only Data Supplement).
Outcome Inter-Relationship Quintile Analysis
For ≥1 of the 3 mortality outcomes, the risk-adjusted mortality rate for 9 of the 10 procedures increased along the progression from the lowest aggregate mortality quintile to the highest aggregate mortality quintile (Tables 4–6). Representative examples of the quintile plots for elective open AAA repair and emergency lower limb amputation are provided in Figures 3–8. This finding was highly significant because it meant that the risk-adjusted mortality for any 1 vascular surgical procedure could be predicted by the performance of each hospital across the remaining procedures. Higher aggregate mortality hospitals had higher-than-expected mortalities for each procedure being examined and vice versa.
The strongest relationships were demonstrated for elective open AAA repair, emergency open AAA repair, elective LEAB, and emergency lower limb amputation, all of which were strongly associated with the aggregated mortality rate for other vascular surgical procedures for all 3 mortality outcomes (all P<0.001).
Long-Term Mortality After Emergency Surgery
Excess variability was evident for 4 out of the 5 emergency vascular surgical procedures at 1-year (emergency open AAA repair, CEA, LEAB, and lower limb amputation), whereas for in-hospital and 30-day mortality, this was evident for only 3 out of 5 procedures (open AAA repair, LEAB, and lower limb amputation; Table 3). Although emergency open AAA repair had the highest 1-year mortality rate (38.4%), it was noteworthy that emergency EVAR, LEAB, and lower limb amputation also demonstrated broadly similar 1-year mortality rates (22.8%, 26.5%, and 25.8%, respectively), which all represented large increases in comparison with their 30-day mortality rates (Table 2).
The key findings of these analyses were that divergent outcomes persist across the range of elective and emergency vascular surgical procedures and that the risk-adjusted outcomes for any 1 procedure could be predicted by the aggregated outcomes of the other procedures studied.
The clear significance of these findings is that they have demonstrated that hospitals can be identified as better or worse performers across a range of procedures within a single specialty rather than demonstrating isolated cases of divergently better or worse performance.
Within any system, there is a need to maintain focus on ensuring patient safety and the provision of a minimum standard of healthcare quality. Previous benchmarking studies in vascular surgery have quantified the clinical outcomes of the procedures described in this work, and these have been used to define minimum standards below which no provider should fall through the development of national guidelines.9 In many cases, national bodies are implementing Quality Improvement Frameworks targeted at specific procedures to ensure that providers exceed the fundamental standards.30,31 These results should enthuse stakeholders that this work remains central to the provision of safer vascular services.
This kind of information is also informative to commissioning groups and hospital managers in the strategic overview and performance management of healthcare services. It is important that results based on mortality outcomes are considered in the context of other performance measures to allow systems improvement rather than to simply stigmatize outlier units.32,33
Aside from the potential for these results to be used in the performance management of individual hospitals, the underlying reason for these relationships remains to be proven. Measures such as hospital case volume that are strongly related to outcome are a corollary of structural characteristics and process factors.13,15 With particular reference to vascular surgery, process factors involving critical care provision have been shown to affect outcomes after AAA repair, whereas structural characteristics such as availability of on-site cardiac catheterization facilities, intensive care facilities, and surgical staffing levels have been shown to be associated with lower mortality rates after LEAB.14,34 It is intuitive that the structural and process factors at hospital level, which might contribute to improved outcomes for 1 type of high-risk operation, would also contribute to improvements for other operations, and this could plausibly underlie the positive inter-relationships found in this and other studies.16
It is noteworthy that although vascular service reconfiguration in England has been centered around the improvement of outcomes for complex arterial surgery (AAA, EVAR, CEA, and LEAB), the findings of this study suggest that outcomes after ablative surgery (lower limb amputation) can also be predicted by the aggregated outcomes of the other high-risk or complex procedures performed. This finding supports the quality improvement framework guidance issued by the Vascular Society of Great Britain and Ireland that seeks to improve mortality after lower limb amputation.31 Recommendations such as the provision of 24×7 vascular expertise to optimize postoperative care mirrors the guidance issued for complex surgery such as AAA repair.30
Contemporary findings from both England and North America are congruent with the findings of the present study. Studies using data from the national Medicare population found strong positive correlations in risk-adjusted inpatient mortality rates for AAA repair, CEA, and LEAB, whereas others have demonstrated positive correlations between the procedure-specific and aggregated mortality of any 1 of 11 high-risk elective vascular and noncardiac general surgical procedures.16,35 Strong positive correlations have also been demonstrated for coronary artery bypass grafting and valve replacement surgery, whereas in the case of elective AAA repair, it has been shown that high-volume, low-mortality endovascular units also had the lowest mortality rates for open AAA repair.12,36
Interestingly, although one would expect the same benefits derived for elective surgery to apply to emergency care, the published evidence is not conclusive. Thus, although marked regional variability in crude mortality outcomes have been demonstrated for emergency laparotomy, an association with volume has not been found for emergency colorectal resection.37,38 Data from the American College of Surgeons National Surgical Quality Improvement Program have demonstrated inconsistency in hospital performance for elective and emergency general surgery with a low correlation coefficient for 30-day mortality.39 Nonetheless, the findings of positive inter-relationships between elective and emergency vascular surgical procedures in this study are supported by other research, which has demonstrated an association between the elective EVAR workload of a unit and mortality after both open and endovascular repair of ruptured AAA. This might reflect differences in the heterogeneity of general surgical patient populations in comparison with vascular patients.40
Persisting hospital-level variability in long-term postoperative mortality after emergency vascular surgery initially seems counterintuitive because underlying patient and disease factors might be expected to be the predominant determinants of longer term survival. In fact, the findings plausibly interlace with the results of other studies, which highlight the importance of medical optimization of this group of patients.41 Preoperative optimization through specialist medical and anesthetic input has been demonstrated to reduce postoperative mortality and complications after major vascular surgery, and it has been shown that such postoperative complications are major determinants of long-term postoperative survival.42–45
Limitations of this study include the possibility of interhospital coding variability inherent to the use of retrospective administrative data, and it is acknowledged that there remains no universal consensus on risk adjustment methodology with the consequence that different models can yield conflicting results.46–48 Although the risk-adjustment strategy used in the present work is based on published methodology, which has been shown to have similar discriminatory power to models derived from clinical databases, studies of retrospective data cannot account for all possible confounding variables.26 Interestingly, however, none of the standardized funnel plots showed evidence of overdispersion, suggesting that the influence of uncontrolled factors in the risk-adjustment process was not significant.49 The use of mortality as a metric for assessing the quality of care has been criticized, although it remains a commonly used measure in health outcomes research and is particularly relevant in vascular surgery.20,33 Because the purpose of this work was to focus future research on quality improvement through identification of underlying processes of care and hospital structural characteristics, analysis was performed at hospital rather than at individual surgeon level. Individual surgeon capability is a factor in determining postoperative outcome, but it has been demonstrated that surgeon-level outcomes can be significantly influenced by hospital structural characteristics.50
Hospital-level risk-adjusted mortality outcomes for both elective and emergency vascular procedures in England vary considerably. Outcomes between procedures were inter-related, consistent with the existence of a hospital-level effect. These data should be made available to patients, relatives, and the purchasers of services to drive improvements in the provision of vascular surgical services.
WHAT IS KNOWN
Positive correlations have been shown between elective in-hospital mortality rates after open abdominal aortic aneurysm repair, carotid endarterectomy, and lower extremity arterial bypass.
Hospitals with low mortality rates for elective endovascular aneurysm repair have the lowest mortality rates for elective open abdominal aortic aneurysm repair.
WHAT THE STUDY ADDS
Divergent short- and long-term postoperative mortality outcomes were evident across the range of index elective and emergency vascular surgical procedures in England.
Risk-adjusted hospital-level outcomes for any 1 procedure could be predicted by the aggregated performance in other procedures.
The findings support the existence of hospital-level infrastructural and process factors that underlie postoperative outcomes.
S. Sinha produced the article. M.M. Thompson and A. Karthikesalingam developed the concept and edited the article. S. Sinha and P.J.E. Holt performed analyses and edited the article. J.D. Poloniecki provided statistical advice and expertise. This report is an independent research supported by the National Institute for Health Research.
Sources of Funding
P.J.E. Holt is a Clinician Scientist financially supported by the National Institute for Health Research (NIHR-CS-011-008). This piece of work has received no additional funding from external sources.
The views expressed in this publication are those of the authors and not necessarily those of the National Health Service, the National Institute for Health Research, or the UK Department of Health.
The Data Supplement is available at http://circoutcomes.ahajournals.org/lookup/suppl/doi:10.1161/CIRCOUTCOMES.113.000579/-/DC1.
- Received September 1, 2013.
- Accepted November 20, 2013.
- © 2014 American Heart Association, Inc.
- Darzi A.
- Kennedy I.
- Francis R.
- Holt PJ,
- Poloniecki JD,
- Khalid U,
- Hinchliffe RJ,
- Loftus IM,
- Thompson MM
- Elixhauser A,
- Steiner C,
- Fraser I
- Urbach DR,
- Baxter NN
- 17.↵Hospital Episode Statistics website. http://www.hesonline.nhs.uk [Accessed 16 May 2011].
- Campbell SE,
- Campbell MK,
- Grimshaw JM,
- Walker AE
- Burns EM,
- Rigby E,
- Mamidanna R,
- Bottle A,
- Aylin P,
- Ziprin P,
- Faiz OD
- Sinha S,
- Peach G,
- Poloniecki JD,
- Thompson MM,
- Holt PJ
- 21.↵Office for National Statistics. http://www.statistics.gov.uk [Accessed 16 May 2011].
- 22.↵The Department for Communities and Local Government. The English Indices of Deprivation 2010. London: Department for Communities and Local Government, 2011.
- Aylin P,
- Bottle A,
- Majeed A
- Tekkis PP,
- McCulloch P,
- Steger AC,
- Benjamin IS,
- Poloniecki JD
- 30.↵The Vascular Society of Great Britain and Ireland. Framework for improving the results of elective AAA repair. 2011.
- 31.↵The Vascular Society of Great Britain and Ireland. Quality improvement framework for major amputation surgery. 2010.
- Rubin HR,
- Pronovost P,
- Diette GB
- Lilford R,
- Pronovost P
- Saunders DI,
- Murray D,
- Pichel AC,
- Varley S,
- Peden CJ
- Mohammed MA,
- Deeks JJ,
- Girling A,
- Rudge G,
- Carmalt M,
- Stevens AJ,
- Lilford RJ
- Spiegelhalter DJ