Effect of Endovascular Aneurysm Repair on the Volume–Outcome Relationship in Aneurysm Repair
Background— We aim to quantify the relationship between the annual caseload (volume) and outcome from elective endovascular (EVR) or open repair of abdominal aortic aneurysms (AAAs) in England between 2005 and 2007.
Methods and Results— Individual patient data were obtained from the Hospital Episode Statistics. Statistical methods included multiple logistic regression models, mortality control charts, and safety plots to determine the nature of any relationship between volume and outcome. The case-mix between hospitals of different sizes was examined using observed and expected values for in-hospital mortality. Outcome measures included in-hospital mortality and hospital length of stay. Between 2005 and 2007, a total of 57 587 patients were admitted to hospitals in England with a diagnosis of AAA, and 11 574 underwent AAA repair. There were 7313 elective AAA repairs, of which 5668 (78%) were open and 1645 (22%) were EVR. In-hospital mortality rates were 5.63% for all elective AAA repairs with rates of 6.18% for open repair and 3.77% for EVR (odds ratio, 0.676; 95% CI, 0.501 to 0.913; P=0.011). High-volume aneurysm services were associated with significantly lower mortality rates overall (0.991; 0.988 to 0.994; P<0.0001), for open repairs (0.994; 0.991 to 0.998; P=0.0008), and EVR (0.989; 0.982 to 0.995; P=0.0007). Large endovascular units had low mortality rates for open repairs.
Conclusion— A strong relationship existed between the volume of surgery performed and outcome from both open and endovascular aneurysm repairs. These data support the concept that abdominal aortic surgery should be performed in specialized units that meet a minimum volume threshold.
Received January 5, 2009; accepted August 3, 2009.
Previous studies have demonstrated significant relationships between annual caseload (volume) and outcome for vascular surgical procedures.1–3 To date, only 1 study has attempted to define the relationship for endovascular aneurysm repair (EVR).4
The healthcare political landscape is evolving rapidly, with many countries now advocating specialization in surgical practice and regionalization of complex services, which would include arterial surgery. The change in delivery of specialized services has been driven by the increasing complexity of cases undertaken and the advent of new technology, in tandem with a focus on reporting health outcomes.5,6 With an emphasis being placed on the public reporting of these outcomes, used to inform patient choice,7,8 elective mortality may be used as a barometer of quality. Unfortunately, elective aneurysm mortality rates remain high in the United Kingdom.9–11
The elective mortality rate for repair of abdominal aortic aneurysm (AAA) has assumed a greater importance since the announcement of national screening programs for AAA, which to be cost-effective should be based around a limited number of screening centers, each covering a population of 1 million.12,13 The delivery of surgical repair of screen-detected aneurysms must be performed at the lowest possible mortality rate if it is to be an acceptable treatment for these asymptomatic patients, and radical models of service delivery, based on outcomes data, have been suggested.14
To achieve these low mortality rates, vascular surgeons are likely to use an increased use of EVR, which has a lower mortality rate than open repair. As experience of these techniques increases, and technology improves, it is likely that an increasing proportion of cases will be within the scope of routine endovascular management. Consequently, the National Institute for Health and Clinical Excellence has recently suggested that EVR should only be performed in specialized centers with appropriate expertise in endovascular aortic surgery. This will imply a move toward centralization of aneurysm services.
This study aims to inform health policy, models of service configuration, and the processes of commissioning. This will be done by determining the nature of any relationship between case volume and the outcome of EVR in conjunction with reviewing the relationship underlying open repair of elective AAA.
WHAT IS KNOWN
For open repair of abdominal aortic aneurysm, higher operative volume confers a survival advantage.
Endovascular aneurysm repair can be delivered at a lower mortality than open repair.
WHAT THE STUDY ADDS
Endovascular aneurysm repair demonstrated an independent volume effect.
Total aneurysm experience (open and endovascular) appeared to be important in determining outcome.
The uptake of endovascular aneurysm repair is insufficient in the English National Health Service, which is contributing to operative deaths.
Dataset and Extraction
Hospital Episodes Statistics (HES) data were acquired for a 2-year period from April 1, 2005, to March 31, 2007. Using in-house programs, data for all the AAA cases and all AAA repairs were extracted and cleaned using previously published methods.2 The data extraction process was tristage with an initial trawl that identified all the episodes that matched the criteria, and then a second stage to exclude from the qualifying spells any episodes that might not have the diagnoses or procedures of interest. A further iteration was run to ensure that each record was reduced to 1 record per spell (admission) to avoid double counting and individual case. In summary the post processing involved the following 3 functions:
Identify all relevant episodes from all spells from the entire HES dataset;
Exclude episodes from spells without codes of interest; and
Reduce the data to 1 record per spell.
The HES are the English National Health Service administrative dataset and contain information on every hospital admission to an National Health Service hospital. The data are based around 2 large coding systems: the International Classification of Diseases, 10th Revision 10 (ICD-10) diagnostic codes; and the Office of Population, Census, and Surveys, version 4 (OPCS-4) procedural codes. In summary, the data extracted pertained to a diagnosis of, or repair of, an AAA along with patient-level demographic factors and comorbidity data.
Identification of EVR Cases
The identification of EVR was not straightforward. This was because over this study period, which mainly used OPCS-4.2, different hospitals coded for EVR in different ways, as no unifying code was available. Toward the end of the period 2 codes, L27 and L28, for EVR were available under OPCS-4.3, but had not been adopted universally by the end of the study.
To identify EVR cases, all procedural codes in the HES that occurred in tandem with a procedural code for an AAA repair were examined. These codes were checked manually for logic, and against advice as to how to code EVR during this transitional period. A few commonly used combinations of codes, either 2, 3, or 4 code runs, were in use, and from these the most appropriate positional codes were selected to identify only infrarenal AAA.
Having satisfied the logic of a diagnosis of AAA (ICD-10 codes I173 or I174), the codes used to identify an EVR superset covering of any segment of the aorta were:
Based on OPCS 4.3: L27.1 to 9 (endovascular insertion of a stent graft), or L28.1 to 9 (endovascular stenting of an aortic aneurysm), or L26.5 (percutaneous insertion of stent into aorta) or L26.6 (fenestrated graft), or L26.7 (abdominal branched graft); and
Based on OPCS 4.2: An operative code for an open aortic aneurysm repair in combination with Y02.2 (insertion of stent into organ) and a positional code Z34.1 to 9.
Codes for a groin cut-down (Y52.8) and an arteriotomy approach to an organ under CT/US guidance (Y53.2) were variably used.
From this superset of all EVR, infrarenal cases were identified by their specific codes. Branched and fenestrated grafts were not considered.
All analyses were performed using SAS version 9.1 (SAS Institute, Inc). Analyses were of a combined group of elective AAA repairs, and open and EVR subgroups. An α level of 0.05 was designated as the value to report statistical significance in all studies. The details of the methods used are described below and they have also been described in previous publications.2,15
Summary data were produced for the each group. The effect of volume on outcome was evaluated in a number of different ways using both crude data and after risk-adjustment. The samples were the same for both crude and adjusted analyses. The primary outcome measure was in-hospital death, with operative complication rates and hospital length of stay investigated as secondary outcomes.
Demographic and comorbidity data were used to identify which diagnostic codes were related to a significantly higher or lower in-hospital mortality. These factors were used in risk-adjustment of the safety plots (below). Risk-adjustment was by fitting a multiple logistic regression model. Demographic variables were used and secondary diagnoses were included in the model if selected by a forward selection procedure. Differences in the effect of each variable on outcome of EVR and open repair were quantified (Tables 1 and 2⇓). The model was a binary logit model, and maximum likelihood estimates were generated to estimate log odds ratios and tested using χ2 tests for each variable in the model. The independent variables were quantified in terms of odds ratios (ORs) and 95% Wald CIs.
Mortality control charts were created for all subgroups and showed the in-hospital mortality rate at an individual hospital against the number of cases performed. There was visual reference to the national mean mortality and to 95% CI. Centers that lay outside the CIs had a less than 5% chance that their mortality rate was consistent with the national mean.
For quintile analysis, cases were divided into volume quintiles, each containing similar numbers of cases, for each clinical group. The quintiles were arranged to include all hospitals of the same volume in the same quintile, rather than splitting hospitals of the same volume to achieve exactly the same number of cases per quintile. Crude mortality rates were examined between the volume quintiles and descriptive data produced for each. These analyses were primarily descriptive and augmented the quantitative risk-adjusted analyses from the statistical models.
Risk-adjusted safety plots were created using twice the national average mortality as the safety threshold (k=2). These plots indicate the outlier status of individual units.15 The aim of this investigation was to assess whether hospitals could provide evidence of safety rather than no evidence of danger as in the mortality control charts. Inpatient mortality rates were compared using the relative risk (RR) of mortality at a particular hospital compared to the death rate elsewhere. The data were arranged into 3 groups: hospitals with a RR between 0 and 1 (0<RR<1; green), hospitals with a relative risk between 1 and 2 (1<RR<2; blue), and hospitals with a RR greater than 2 (2<RR; red).
The probability was calculated that the relative risk of mortality at a given hospital was different to twice that elsewhere. The probability values were displayed on a scale of log10(odds) to distinguish small probability values that differed by orders of magnitude. Odds were used rather than probability values to exploit the fact that log(odds) are equal to 0 for P=0.5, and so evidence of safety and danger were shown in different directions on the y axis.
These log10(odds) values were shown on the y axis against the hospital procedural volume along the x axis to construct “safety plots.” Log10(odds) of 1.3, equivalent to 1-tailed probability value of 0.05, was indicated by solid horizontal lines on the charts. Hospitals that lay outside the 2 lines generated a significant weight of evidence that their mortality rate was inconsistent with the threshold value, being either higher or lower. Hospitals that lay within the “control bands” may still have had a RR of mortality greater than, or greater than twice, the national average, but there was insufficient evidence to be able to identify them as “safe” or “unsafe.” Overall, this technique provided 3 alternative states in to which hospitals fell:
evidence of safety;
insufficient evidence of safety or danger; and
evidence of danger.
The case-mix at individual hospitals was investigated by calculating the expected mortality for an individual hospital based on its case-mix compared to that nationally. The expected number of deaths at a hospital was calculated by fitting the model to the data from all hospitals after excluding the hospital of interest. Estimates from the fitted model were then obtained for the case-mix at the excluded hospital. The observed and expected risk values for an individual hospital were displayed graphically against the number of cases performed at that hospital over 2 years. Finally, the observed:expected risk ratios were calculated.
Between April 1, 2005, and March 31, 2007, there were a total of 57 587 admissions to hospitals in England with a diagnosis of an infrarenal AAA. Summary data are provided for each subgroup (Table 3).
11 574 AAA repairs were performed with 7313 elective repairs, 1888 urgent repairs, and 2373 repairs of ruptured AAA. The in-hospital mortality rates were 5.63%, 20.4%, and 37.8% respectively. 2522 patients were admitted with a ruptured AAA but did not have an operation performed. The mean age of this group was 80.6 years, and 2009 died; an in-hospital mortality rate of 79.7%.
Of the 7313 elective AAA repairs, 5668 (78%) were open repairs and 1645 (22%) were EVR. The mean age of patients undergoing elective AAA repair was 72.6 years, with means of 72.2 years in the open group and 73.9 in the EVR group. The median length of stay was 12.9 days, being 9.42 days in the EVR group and 13.9 days in the open group.
412 (5.63%) in-hospital deaths were observed in the combined group, which comprised 350 (6.18%) for open cohort and 62 (3.77%) in the EVR cohort (OR, 0.595; 95% CI, 0.452 to 0.783; P<0.0001). After adjusting for all other factors in the risk model the overall effect of EVR in reducing in hospital mortality remained significant (OR, 0.676; 95% CI, 0.501 to 0.913; P=0.011).
The mortality control chart for the combined group showed a number of significant outliers, and these were predominantly lower volume hospitals (Figure 1). A similar relationship was seen in both the open and EVR subgroups (Figures 2 and 3⇓).
A combined estimate for the effect of volume on mortality was obtained from a multiple logistic regression model with risk-adjustment for age, sex, and 11 diagnostic risk factors. It was found that an increasing hospital volume was associated with a significant reduction in the odds of mortality for the combined group (OR, 0.992; 95% CI, 0.988 to 0.995; P<0.0001) with this value being a reduction in odds per additional elective case performed. After including a factor adjusting the combined relationship to account for the highly significant reduction in mortality described for EVR, as well as for comorbid conditions, the relationship was maintained (OR, 0.993; 95% CI, 0.989 to 0.997; P=0.010). This equated to a 13% reduction in the odds of mortality for each additional 20 cases performed.
Exploring the relationship of open and EVR cases independently (ie, creating a false situation in which centers provided only open or EVR cases) provided estimates of the effect of open and EVR independent of the other modality of treatment. The estimates were for open repairs (OR, 0.99; 95% CI, 0.989 to 0.999; P=0.0216) and EVR (OR, 0.993; 95% CI, 0.987 to 1.000; P=0.0572). When the relationships were estimated in light of true practice (ie, each hospital providing both treatment modalities) the following estimates were generated: for open repairs (OR, 0.994; 95% CI, 0.991 to 0.998; P=0.0008) and EVR (OR, 0.989; 95% CI, 0.982 to 0.995; P=0.0007). The rationale for these 2 analyses is discussed later. Hospitals performing a high total volume of cases, open and EVR, had mortality rates below the national average for open repairs even when the predominant modality of treatment was EVR.
Increasing age was a statistically significant risk factor for mortality for open repairs (OR, 1.045; 95% CI, 1.029 to 1.062; P<0.0001), but the effect was smaller and not significant for EVR (OR, 1.024; 95% CI, 0.990 to 1.060; P=0.167). Conversely, male gender appeared protective for EVR (OR, 0.495; 95% CI, 0.264 to 0.928; P=0.028), but not for open repair (OR, 0.889; 95% CI, 0.665 to 1.162; P=0.430). Interestingly, chronic ischemic heart disease (P=0.787), atherosclerosis (P=0.175), peripheral vascular disease (P=0.630), or chronic lung disease (P=0.834) were not statistically significant factors in the risk model for EVR, whereas they were all highly significant for open repair.
For open and EVR modalities together and separately, a number of hospitals demonstrated procedural safety for elective AAA repairs, after risk-adjustment, and in all subgroups a small number of hospitals demonstrated evidence of danger (Figures 4 through 6⇓⇓).
Analysis by volume quintiles demonstrated significant relationships between the annual case volume undertaken and mortality for all groups (Table 4). In particular those hospitals performing fewer than 8 EVR per annum had mortality rates significantly higher than all other hospitals performing EVR (6.88% versus 3.02%; OR, 2.370; 95% CI, 1.397 to 4.032; P=0.003). Of 91 hospitals with an endovascular service, 64 performed fewer than 8 repairs per annum. Of 134 hospitals performing open AAA repair, 59 were in the lowest volume quintile performing fewer than 15.5 repairs per annum.
The median length of stay was lower at higher volume hospitals for EVR with a median of 7.25 days in the highest volume quintile against 10.2 days in all other quintiles combined (Figure 7).
Hospitals performing a higher number of cases, either EVR or open, had a higher predicted operative death rate than lower volume hospitals (7% versus 5%), but a lower observed death rate than predicted (Figure 8). In almost every hospital performing more than 100 repairs per annum the observed number of deaths was lower than the expected number, a relationship not observed in lower volume hospitals.
This study demonstrated significant volume-related improvements for in-hospital mortality for AAA repair, even after account was taken for the beneficial effects on mortality rates of EVR. The results implied interdependency between open and endovascular outcomes such that hospitals with a greater total aneurysm experience had improved outcomes irrespective of the treatment modality used. Hospitals with a higher total aneurysm experience were more likely to operate on patients of a higher risk but had a lower overall in-hospital mortality despite this. The mechanisms underlying these results are complex and have been discussed in detail,1 but were likely to represent a combined effect of high volume specialist surgeons, better equipped intensive care units, and locally available support services.
Previous studies have demonstrated a strong volume-outcome relationship for open AAA repair,1,2,16,17 but to date only 1 study has investigated the relationship between EVR volume and outcome.4 The findings of this study corroborate those previously published and add to the limited evidence base regarding the effective delivery of these procedures. The scale of effect, including the finding of a higher magnitude of effect for EVR than open repair, was consistent between this study and the North American study.4 This conclusion is drawn from both the estimate of the volume-outcome relationship for EVR alone (OR, 0.993) and from the estimate of EVR within systems that provided open repair and EVR (OR, 0.989). This is important as the confidence limits of the theoretical EVR-only analysis are at present broad, as there were a smaller number of cases and deaths than for open repairs over this limited time period. The data are highly suggestive of an independent volume effect for EVR that was stronger than that of open repair. This finding was perhaps unexpected, but the agreement between these 2 large population-based studies should not be underestimated and the results should be used to inform health policy. Certainly, when taken in the light of clinical practice where hospitals perform both open repair and EVR the results were highly significant for both modalities of treatment after adjustment for the other modality.
Further agreement was found between these 2 studies in the low uptake of EVR on a national basis, which was likely to be inappropriate given the proven survival advantage delivered through EVR,18 a fact reinforced by these data. The reduction in postoperative mortality observed was very similar to that described in the original trials assessing EVR, although, as would be expected for a population-based study, the observed death rates were somewhat higher than those reported only from selected trial centers.
The results from the present study provided further evidence that aneurysm services should be regionalized to high-volume hospitals. This suggestion would appear to be concordant with those of a recent major review of the organization of health services,6 which stated that clinicians must ensure that the care delivered to patients is of high quality and safe, as well as being cost-effective. The suggestion was that this would involve the strategic reconfiguration of complex surgical services, which would include AAA repair. The major aim for aneurysm services must be to offer all patients EVR where suitable, but with surgery performed within high-volume specialist centers.
The safety plots presented in conjunction with the other analyses of volume and outcome highlighted the fact that the demonstration of safety was in part dependent on case volume. Both of these factors (safety and volume) formed the basis of a recent model for the provision of aneurysm services in England, which demonstrated that the reconfiguration of services might prevent a significant number of operative deaths for AAA repair.14 When this published model was applied to these data, another significant reduction in mortality was shown (P=0.037) which would support an immediate drive to reconfigure services.
These data would also suggest that the uptake of EVR was low in England, approximately 1 in 5 cases performed, when compared to European and Australasian countries. In support of these results were the similar findings of other sources of data such as those provided by BIBA Medical, Vascunet, and the National Vascular Database.9–11 One confounding factor might be the complexity of identifying EVR cases at this stage. Although every attempt was made to determine the true number of EVR performed, the clinical coding systems over the study period did not provide a single unifying code for EVR, and so it was not possible to be certain about the case identification process, which was complex. There are now specific EVR codes in use and their uptake should be enforced in preference to any local coding arrangements. Refinements in systems for the reimbursement of clinical activities should reflect these changes in coding and provide an incentive for their use.
As well as being consistent in terms of the proportion of EVR performed, these data were consistent with registry data in terms of the proportion of patients reporting specific comorbidities.9 Clearly there is scope for improving both clinical coding and the completeness and accuracy of national datasets, but the consistency of these results is at least reassuring.
A national aneurysm-screening program will commence in selected centers in England from 2009 with further expansion expected over the next 5 years. This is expected to reduce aneurysm-related mortality on a population basis through early diagnosis and the prevention of aneurysm rupture. Screening will be associated with concurrent increases in the number of elective AAA procedures performed as diagnosis rates increase.13 It is only appropriate to screen patients when a treatment can be offered that is acceptable to those patients. For AAA, this must comprise the delivery of elective aneurysm repair at the lowest possible operative mortality rate, which under current practice would include limiting the treatment of screen-detected cases to high volume hospitals experienced in EVR.
A focus on health outcomes is central to the assessment of a modern health service, together with a will to reduce unwanted variations in these measured outcomes.5 As low case volume has been identified as one source of these variations in outcomes, it may be that service commissioning groups, or purchasers, will support the formation of designated aneurysm centers and that these hospitals are likely to be the same as those from which aneurysm screening programs will ultimately be delivered.
It may be possible that the very high mortality seen in the first quintile of the EVR group might have reflected a learning curve for newer centers undertaking EVR. If this was the case, then there may be consequences for the way in which, or duration for which, surgeons and radiologists are mentored in the initial stages of an endovascular program. It would seem unlikely that there is a need for two thirds of all centers (64 of 91) performing EVR to be doing so at annual volumes of fewer than 8 cases. Similarly, half of all hospitals performing open repair were doing fewer than 15 cases a year. A more pragmatic view might be to accept that there are already too many aneurysm services in England and that efforts would be best channeled into formalizing clinical network arrangements, rather than setting up new low-volume centers.
One question that arose from these results was as to the true nature of the relationship between volume and outcome. Although multiple logistic regression models were used correctly in this study to provide risk-adjusted point estimates for operative risk, it was possible that the relationship was not in fact linear for EVR. For both the open and combined groups the quintile results did appear to fit a linear model with r2 greater than 0.8, but the statistic was lower for EVR. The implication of this was, not that the relationship was neither robust nor significant, but more that the extent of the reduction in odds of mortality might be overestimated at the extreme of high volume in this linear model. As such the magnitude of effect for each modality has not been reported, in terms of the odds ratio of mortality per case by modality, but only the overall effect estimate. Future studies should focus on this relationship for EVR and aim to define the nature of the relationship through a detailed statistical curve-fitting exercise.
A strong relationship exists between the volume of surgery performed and outcome from both elective open and endovascular aneurysm repair. These data support the formation of a regionalized service for aneurysm repair. This is likely to occur in conjunction with other vascular surgical services. Aneurysm screening services should be formed around these vascular surgical centers. These results should be used to inform the commissioning of services.
Sources of Funding
Dr Poloniecki is supported by the BUPA Foundation. Dr Holt holds a Circulation Foundation Research Fellowship and received an equipment grant from the FH Muirhead charitable trust.
Department of Health. World Class Commissioning, 2008. http://www.dh.gov.uk/en/Managingyourorganisation/Commissioning/Worldclasscommissioning/DH_083204. Accessed December 12, 2008.
Darzi A. High quality care for all: NHS Next Stage Review final report. In: Department of Health, editor: TSO, 30 June 2008; 1–92.
Department of Health. NHS Choices publishes surgery survival rates, 2008. http://www.nhs.uk/NHSEngland/Hospitalmortalityrates/Pages/Data.aspx. Accessed 20th January 2009.
Department of Health. Patient Choice, 2008. http://www.dh.gov.uk/en/Healthcare/PatientChoice/DH_469. Accessed January 20, 2009.
European Society for Vascular Surgery. First vascunet database report, 2007. http://www.esvs.org/files/Other_Committees/ESVS_VASCUNET_REPORT_2007.pdf. Accessed July 17, 2009.
BIBA Medical Ltd. http://www.cxvascular.com/bibamedical. Accessed July 17, 2009.
Vascular Society of Great Britain and Ireland. National Vascular Database. http://www.vascularsociety.org.uk. Accessed July 17, 2009.
Multicentre Aneurysm Screening Study Group. Multicentre aneurysm screening study (MASS): cost effectiveness analysis of screening for abdominal aortic aneurysms based on four year results from randomised controlled trial. BMJ. 2002; 325: 1135–1141.
Holt PJE, Poloniecki JD, Loftus IM, Thompson MM. Demonstrating safety through in-hospital mortality following elective repair of abdominal aortic aneurysm in England. Br J Surg. 2007; 95: 64–71.
Greenhalgh RM, Brown LC, Kwong GP, Powell JT, Thompson SG, EVAR trial participants. Comparison of endovascular aneurysm repair with open repair in patients with abdominal aortic aneurysm (EVAR trial 1), 30-day operative mortality results: randomised controlled trial. Lancet. 2004; 364: 843–848.