Causes and Consequences of Missing Health-Related Quality of Life Assessments in Patients Who Undergo Mechanical Circulatory Support Implantation
Insights From INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support)
Background—Missing health-related quality of life (HRQOL) data in longitudinal studies can reduce precision and power and bias results. Using INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support), we sought to identify factors associated with missing HRQOL data, examine the impact of these factors on estimated HRQOL assuming missing at random missingness, and perform sensitivity analyses to examine missing not at random (MNAR) missingness because of illness severity.
Methods and Results—INTERMACS patients (n=3248) with a preimplantation profile of 1 (critical cardiogenic shock) or 2 (progressive decline) were assessed with the EQ-5D-3L visual analog scale and Kansas City Cardiomyopathy Questionnaire-12 summary scores pre-implantation and 3 months postoperatively. Mean and median observed and missing at random–imputed HRQOL scores were calculated, followed by sensitivity analyses. Independent factors associated with HRQOL scores and missing HRQOL assessments were determined using multivariable regression. Independent factors associated with preimplantation and 3-month HRQOL scores, and with the likelihood of missing HRQOL assessments, revealed few correlates of HRQOL and missing assessments (R2 range, 4.7%–11.9%). For patients with INTERMACS profiles 1 and 2 and INTERMACS profile 1 alone, missing at random–imputed mean and median HRQOL scores were similar to observed scores, before and 3 months after implantation, whereas MNAR-imputed mean scores were lower (≥5 points) at baseline but not at 3 months.
Conclusions—We recommend use of sensitivity analyses using an MNAR imputation strategy for longitudinal studies when missingness is attributable to illness severity. Conduct of MNAR sensitivity analyses may be less critical after mechanical circulatory support implant, when there are likely fewer MNAR data.
Patient-reported outcomes (PROs; eg, health-related quality of life [HRQOL], pain, and depression) are increasingly measured in cardiovascular clinical trials and registries because of their importance in illuminating the benefits of treatment for patients. However, they must be collected prospectively because there is no way to retrospectively capture these data. Accordingly, it is necessary to address missing HRQOL data. Although there can be noninformative, administrative reasons for missing HRQOL data, missingness can be problematic when attributable to the severity of patients’ illnesses. Although some challenges of missing data are loss of precision (ie, wider confidence intervals) and loss of statistical power (attributable to a reduction of sample size),1 the greatest concern is the bias that can be introduced when measures do not reflect true differences within and between groups across time.2 Standards for reporting missing PRO data have been proposed and include prevention and handling of missing data when missingness is unavoidable.2,3
Analytic strategies to handle missing PRO data are partially dependent on the mechanism of missingness. Rubin4 proposed the following missing data pattern nomenclature: missing completely at random, missing at random (MAR), and missing not at random (MNAR). Missing completely at random occurs if missingness is unrelated to the data that should have been obtained and to all other observed data. In this case, the complete-case data effectively form a random sample of the full (subject to missingness) data. Assuming that the correct data analysis methodology is used, a complete-case analysis will not incur any bias, although precision and power will be degraded as a consequence of sample size loss. Data are considered MAR if missingness is related to observed factors (eg, younger patients less likely to complete follow-up than older patients) but is unrelated to the missing data that should have been obtained after controlling for these variables.5 In contrast, when data are MNAR, missingness depends on the missing data themselves, even after adjustment for other variables (eg, patients with increased illness severity and poor HRQOL being least likely to complete HRQOL surveys).5 In this case, the observed data contain incomplete information about the nature of the unobserved data, and any assumptions made about the missing data are inherently untestable, resulting in conclusions that are highly sensitive to these assumptions.
It is important to note that all 3 types of missing data may be present in a given data set and even for a given variable, with varying implications for analyses. Therefore, when the amount of missing data is nontrivial (eg, >10%–20%), a thorough understanding of the possible causes of missingness, and appropriate methods for handling missing data, are essential. A variety of statistical methods are available for conducting MAR analyses, including maximum-likelihood estimation, multiple imputation, and inverse probability weighting,1,6 all of which can reduce bias caused by missingness.1 One of the most important considerations is the capture and inclusion of factors associated with both the variable of interest and the likelihood of missingness (although often one may not know in advance which variables these are). When it is believed that the data may be MNAR, sensitivity analyses are conducted to assess the impact of the assumptions made about missing data, which may include reducing imputed values by some amount or setting missing values to a worst-case (eg, HRQOL score of 0) value. Such analyses may help bracket the study results by quantifying the maximum possible impact of data MNAR. A major challenge to conducting and reporting MNAR sensitivity analyses is that because it is not possible to test the MNAR assumption, it is not possible to demonstrate that any given MNAR model is correct.1
A prototypical example of missing data is INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support), a prospective registry that collects data on patients who receive durable, Food and Drug Administration–approved mechanical circulatory support devices in the United States and Canada.7 PROs are an integral component of INTERMACS, which has previously reported improved HRQOL across time.8–10 These findings provide important information on outcomes for clinicians to share with advanced heart failure patients considering treatment options, including implantation of left ventricular assist devices (LVADs). However, changes in HRQOL from before to after implant have been related to preimplantation heart failure severity, comorbidities, implant strategy, and postimplantation adverse events, and the optimal approach to handling missing HRQOL data when patients are often missing these assessments because of the severity of their illness requires further elucidation.
It is likely that missing preimplantation HRQOL data in INTERMACS are at least MAR and oftentimes MNAR for patients who are too sick to respond to HRQOL surveys. After mechanical circulatory support implantation, however, many patients recover and improve, and missingness at follow-up assessments is likely attributable to other factors and MAR. Understanding the causes of missingness in the advanced heart failure population is essential to correctly characterize patients’ baseline and postoperative HRQOL. We, thus, sought to best inform handling of missing data in INTERMACS by (1) identifying factors associated with missing HRQOL, at time of implant and 3 months later, (2) examining the impact of these factors on estimated HRQOL under the assumption of MAR missingness, and (3) performing sensitivity analyses to examine the potential impact of MNAR missingness attributable to illness severity.
The data, analytic methods, and study materials will not be made available to other researchers for the purposes of reproducing the results or replicating the materials. Data sets are available through requests to INTERMACS.
Adult patients (age ≥19 years at implant) included in this study were from a pool (n=6770) of patients at 157 US and Canadian institutions participating in INTERMACS, who received primary implantation of a continuous-flow LVAD as a bridge to transplantation or destination therapy between June 2012 and February 2015. We included 3248 patients who had a preimplantation INTERMACS profile of 1 (critical cardiogenic shock) or 2 (progressive decline),11 as these were among the most ill and the ones in which the bias associated with missing HRQOL was likely the greatest (24% having incomplete baseline assessments because of illness, versus 4% of patients with INTERMACS profiles 3–7; Figure 1). Patients who were no longer in INTERMACS at 3 months because of death (n=335), transplantation (n=76), or recovery (n=4) were included in analyses of baseline HRQOL but not month 3 HRQOL.
HRQOL scores were from the EQ-5D-3L visual analog scale (VAS) score and Kansas City Cardiomyopthy Questionnaire (KCCQ)-12 summary score. The EQ-5D-3L is a generic health status instrument that measures overall health status using a vertical VAS (0=worst imaginable health state and 100=best imaginable health state).12 A difference of 10 points in the VAS score over time is clinically meaningful.13 The EQ-5D-3L also measures 5 dimensions of HRQOL: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression, although this component is primarily used for deriving health utility indices and so was not considered in the present analyses.12,14 Psychometric support for this instrument has been reported, including for patients with cardiovascular disease.15,16
The KCCQ-12 is a 12-item heart failure–specific health status questionnaire.17 It has 4 domains: physical limitations, symptom frequency, social limitations, and QOL, as well as a summary score that combines the 4 domain scores. Response options use Likert scales, wherein higher scores denote better health status (summary score range=0–100). A difference of 5 points in the summary score across time is clinically meaningful.18 The KCCQ-12, which was developed from the longer KCCQ-2319 that has been used in studies of mechanical circulatory support patients,20 to reduce response burden, has similar, robust, psychometric properties as the KCCQ-23.17
Approval was received from INTERMACS site Institutional Review Boards, and written consent was obtained from patients when required. Per INTERMACS protocol, patients were to complete the HRQOL surveys before and 3 months after implantation. Medical records data were abstracted at regular intervals.
Over 100 variables were evaluated for their association with HRQOL and with missing HRQOL assessments, both at baseline and 3-month follow-up. They included demographics; preimplantation clinical characteristics; postimplantation status; functional status (eg, New York Heart Association class and 6-minute walk test) at 1, 4, and 12 weeks after implantation; adverse events through 3 months; available HRQOL scores (ie, baseline scores for those missing 3-month scores, and vice versa); and 3-month and 1-year survival/follow-up status (Table I in the Data Supplement). Variables were compared by HRQOL completion status (completed, patient too ill to complete, not completed for other patient reasons, not completed for administrative reasons, or unknown [Table II in the Data Supplement]), using Kruskal–Wallis tests for continuous variables and χ2 or Fisher exact tests for categorical variables. Independent factors associated with both HRQOL scores and with missing HRQOL assessments were determined using multivariable linear and logistic regression. Least absolute shrinkage and selection operator variable selection was used to protect against overfitting,21 given the number of candidate variables considered, and bootstrap validation was used to ensure that models were not overfit.22
Missing HRQOL scores were multiply imputed based on all patient characteristics using fully conditional specification methods as implemented in the R package Mice (multivariate imputation by chained equations).23 In this approach, each incomplete variable is regressed in turn on all other covariates, and the process is iterated until regression estimates stabilize. Imputed values were chosen using predictive mean matching, which selects a value at random from among the observed data for patients whose predicted values are closest to those of the one with the missing value. This approach corrects for biases associated with observed data, preserves relationships among the variables, ensures that imputed values are within plausible ranges, and accounts for uncertainty in the missing values. Twenty such randomly imputed data sets were generated; analyses were conducted separately on each imputed data set and then pooled, thus incorporating uncertainty attributable to imputations in the results.24 The distributions of observed and imputed HRQOL scores were compared both quantitatively by descriptive statistics and graphically by nonparametric density plots of the observed data and of 5 randomly selected imputed data sets.
To examine the potential impact of MNAR data, we conducted sensitivity analyses lowering the imputed EQ-5D VAS scores and KCCQ-12 summary scores for those patients who were reported as too ill to complete assessments. We considered 3 strategies: (1) decreasing the imputed scores by 50%, (2) replacing the imputed score with a randomly selected score from the lowest quartile of observed scores, and (3) setting the imputed scores to zero. The first strategy represents a moderate decrease in scores, whereas the last strategy represents the worst-case scenario and provides a lower bound for the distribution of true HRQOL scores. The last strategy was used in previous HRQOL reports from INTERMACS8–10 based on the spread of VAS scores (<10) for INTERMACS profile 1 patients, so we wanted to test the veracity of this approach.
Analyses were conducted across all patients and then repeated for those with INTERMACS patient profile 1. Analyses were performed using SAS 9.4 (SAS Institute, Inc, Cary, NC) and R version 22.214.171.124
Demographic and Clinical Characteristics
Advanced heart failure patients receiving continuous-flow LVADs who were included in this study had a mean age of 57.0±13 years, 77.2% were male, and 884 were INTERMACS profile 1, whereas 2364 were INTERMACS profile 2 (total n=3248). The vast majority of these patients had a diagnosis of idiopathic dilated cardiomyopathy (32.8%) or ischemic cardiomyopathy (40.3%), with preimplantation device strategies of bridge to transplant (26.0%), possible bridge to transplant (31.8%), and destination therapy (42.2%). Before device implantation, most patients were NYHA class IV (86.9%), receiving intravenous inotropes (91.9%), and had an implantable cardioverter defibrillator (78.3%). After implant, 91.6% of patients were discharged alive with a device in place, primarily to home (69%). By 3 months, the majority of patients were NYHA class 1 or 2 (72.2%). Adverse events within 3 months after implant included bleeding (29.5%), infection (28.6%), arrhythmia (21.5%), and respiratory failure (17.8%). Almost 38% of these LVAD patients were rehospitalized within 3 months of surgery.
Rates of Instrument Completion
Preimplantation rates of HRQOL instrument completion, with scoring, were as follows: EQ-5D-3L VAS (n=1618, 49.8%) and KCCQ-12 (n=1625, 50.0%; Table 1). At 3 months after surgery, completion rates were somewhat improved: EQ-5D-3L VAS (n=1598, 56.4%) and KCCQ-12 (n=1647, 58.1%), although only 32% of patients completed HRQOL assessments at both time points. Substantially more patients were too sick to respond to both instruments (24%) immediately before implant than at 3 months after implant (4%), at which time the most frequent reason for instrument noncompletion was administrative (Table 1; eg, coordinator too busy or forgot to administer instruments; 12% and 13% for the EQ-5D-3L and KCCQ-12, respectively).
Patients who completed the KCCQ-12 pre-implantation (n=1714) were compared with noncompleters by reason: too sick (n=776), other patient-related reason (n=273), and administrative or unknown reason (n=485; Table 2). Significant differences in KCCQ-12 completion rates for these 4 groups were detected for some demographic characteristics (eg, age, education, and work status) and some preimplantation clinical characteristics (eg, device implant strategy, patient profile, inotrope therapy, presence of an intervention within the last 48 hours [eg, intra-aortic balloon pump and ventilator] and NYHA class). More patients who did not complete the KCCQ-12 before implant, because of being too ill, were INTERMACS profile 1, NYHA class IV, and had more interventions within the last 48 hours, compared with other groups. By 3 months after implantation, more patients who did not complete the KCCQ-12, because of being too ill, had respiratory failure and renal dysfunction than other groups. Similar between-group differences were found for patients who completed the EQ-5D-3L VAS and the 3 groups of EQ-5D-3L VAS noncompleters for both demographic and clinical variables, except there was only a trend for device implant strategy (Table III in the Data Supplement).
Factors Associated With HRQOL and the Likelihood of Missing HRQOL Assessments
Independent factors associated with HRQOL scores and with the likelihood of missing HRQOL assessments are listed in Table 3. These analyses revealed few correlates of HRQOL and of missing assessments in this population and generally weak associations, as measured by R2 values and C statistics (discrimination was strong for missing month 3 assessment because of illness [C=0.81], although this comprised only 4% of patients). The strongest correlates of baseline HRQOL were the corresponding 3-month HRQOL scores and vice versa. Importantly, there were few factors (ie, corresponding HRQOL assessments pre- and post-implantation, and at month 3, intubation and Intensive Care Unit/Cardiac Care Unit days post-implantation) that were jointly associated with both HRQOL and the likelihood of missing assessments, criteria that would indicate factors contributing to bias that should be included in imputation or other MAR analyses. Results were nearly identical when restricted to patients with INTERMACS profile 1.
Comparison of Observed HRQOL Scores With Multiply Imputed MAR Scores
We compared observed versus imputed HRQOL scores for all patients (ie, patients with INTERMACS profiles 1 and 2; Table 4) and only patients with INTERMACS profile 1 (Table 5) for the EQ-5D-3L VAS score and KCCQ-12 summary score pre-implantation and 3 months post-implantation. Considering >100 covariates, the distribution of imputed HRQOL scores did not change much from the observed scores for EQ-5D-3L VAS scores and KCCQ summary scores, either before or early after implantation in profiles 1 and 2 (Figure 2) or in profile 1 patients alone (Figure 3). For all patients, we found that the mean and median EQ-5D-3L VAS scores and KCCQ-12 summary scores were similar for observed and imputed data before and after LVAD implantation (preimplantation mean±SD scores for the EQ-5D-3L VAS were observed=42.2±24.9 and imputed=41.4±25.2; 3-month mean±SD scores for the EQ-5D-3L VAS were observed=70.0±21.6 and imputed=69.0±22.3; Table 4). For the KCCQ-12, the mean±SD scores before implantation were similar (observed=31.4±20.6 and imputed=30.8±20.7), as were the 3-month scores (observed=64.9±19.7 and imputed=63.7±20.9; Table 4).
Observed and multiply imputed mean and median EQ-5D-3L VAS scores and KCCQ-12 summary scores were also fairly similar both before and 3 months after implantation for patients with INTERMACS profile 1, except for the observation that the preimplantation observed median EQ-5D-3L score of 40 was 5 points higher than the imputed score of 35 (Table 5).
Distributions of imputed scores were also examined by reason for missingness (patient too ill, other patient reasons, administrative/unknown reasons; Figures 4 and 5). Before implantation, even among patients who were reported to be too ill to complete HRQOL assessments, imputation did not reveal any discernible difference from observed scores. At 3 months, imputed scores were lower for those too ill to complete the assessments, although this comprised only 4% of the study population and had little effect on the overall distribution reported above.
Sensitivity Analyses for MNAR Data
MNAR sensitivity analyses were conducted by decreasing imputed scores of patients who were too ill to complete HRQOL assessments using 3 strategies: (1) decreasing the imputed scores by 50%, (2) replacing the imputed scores with a randomly selected score from the lowest quartile of observed scores, and (3) setting the imputed scores to zero. For all preimplantation patients, when the MAR imputed mean±SD EQ-5D-3L VAS scores and KCCQ-12 summary scores were subjected to the 3 MNAR imputation strategies (1, 2, and 3), the scores differed modestly by strategy and instrument (EQ-5D-3L VAS scores: 36.8±24.8, 34.8±25.8, and 32.2±28.5 and KCCQ-12 summary scores: 27.3±20.0, 25.8±20.5, and 23.8±22.4), respectively (Table 4). When MNAR sensitivity analyses were conducted for only INTERMACS profile 1, larger differences in preimplantation mean±SD scores for all 3 strategies (1, 2, and 3) using the EQ-5D-3L VAS and KCCQ-12 summary scores were observed (EQ-5D-3L VAS score: 29.2±22.9, 25.6±23.7, and 20.3±26.9 and KCCQ-12 summary score: 21.4±17.8, 18.5±17.9, and 14.5±20.3) because of the larger proportion of patients who were too ill to complete HRQOL forms (Table 5). Notably, changes across time were substantially larger in the MNAR analyses because more patients were missing scores because of severity of illness at baseline.
Preimplantation MNAR-imputed median (as opposed to mean) scores, using the 3 strategies for all patients, were fairly similar (EQ-5D-3L range of median VAS scores=30–32 and KCCQ-12 range of median summary scores=19.8–23.4; Table 4). When only considering patients with INTERMACS profile 1, preimplantation MNAR-imputed median scores, using the 3 strategies 1, 2, and 3, differed substantially (EQ-5D-3L VAS scores: 25, 20, and 1 and KCCQ-12 summary scores: 16.7, 12.5, and 1.6), respectively (Table 5). Three months after implantation, imputed median scores for all patients and for only INTERMACS profile 1 patients were the same across the 3 MNAR strategies for each instrument, although interquartile ranges varied somewhat (Table 5).
HRQOL assessments, because they require direct contact with and input from patients, are particularly subject to missing data depending on patient availability and clinical status. In light of the potential biases associated with missing PRO data when patients are too ill to participate in surveys, we used the INTERMACS database of patients with severe heart failure undergoing LVAD implantation to provide empirical examples of alternative methods for handling missing data on observed PRO scores. Using MAR imputation, which accounts for biases associated with observed variables, we found a small impact on the distribution of either pre-implantation or 3-month postimplantation HRQOL scores, despite accounting for >100 patient factors and outcomes and regardless of the reason for missingness (although imputed scores at 3 months were lower for those too ill to complete the assessments). Surprisingly few factors were associated with missing HRQOL assessments, and even fewer associated both with missing assessments and with observed HRQOL scores, criteria that would identify a factor as a confounder of missing HRQOL.
We then considered 3 MNAR sensitivity analyses, imputing lower HRQOL scores to reflect the potential for informative missingness. The lowest mean MNAR-imputed scores, for both instruments at both time periods, occurred when missing scores were set to zero, as expected. This scenario represents the greatest possible impact that MNAR data could have on the distribution of HRQOL scores and would be considered an extreme sensitivity analysis, given the low likelihood that all such patients would really have an HRQOL score of zero. The other 2 scenarios are less extreme and likely more realistic, particularly for analyses that include all INTERMACS profiles. All MNAR-imputed scores were lower in patients with INTERMACS profile 1, reflecting the greater prevalence of missing HRQOL assessments for these patients, although whether these scores actually reflect such patients’ true HRQOL is unknown and untestable. By using analyses such as these to handle potentially informative missing data, it is possible to provide ranges of potential HRQOL outcomes that can help interpret the observed findings and assess their sensitivity to informative missingness.
Importantly, clinically meaningful differences, defined as a mean difference of ≥5 points for the KCCQ-1218 and ≥10 points for the EQ-5D-3L,13 were found in INTERMACS profile 1 patients using the 2 most extreme MNAR imputation methods for preimplantation scores, where the magnitude of missing data because of being too sick to complete the surveys was greatest. No clinically important differences in 3-month scores for the overall population were observed because of the smaller proportion of patients who were too ill to complete the PROs, as expected, because little data were missing because of illness at follow-up.
All models for handling missing HRQOL data, whether missing completely at random, MAR, or MNAR, are based on strong assumptions; thus, exploring mechanisms for missingness is critical when analyzing longitudinal data. A major challenge to conducting and reporting MNAR analyses is that because it is not possible to test the MNAR assumption, it is not possible to demonstrate that any given MNAR model is correct.1 Therefore, MNAR analyses are most useful as sensitivity analyses (wherein clinically plausible models that use different assumptions are created and compared), often after using MAR imputation modeling.1
Although our results provide empirical estimates of handling missing HRQOL data in a real-world clinical setting, Fairclough5 states that both primary prevention (ie, minimizing missing data during implementation of a registry or trial) and secondary prevention (ie, collecting data on reasons for missing data) are desirable. Regarding preventing missing data in INTERMACS, primary prevention strategies are in place to reduce missingness (eg, sharing of HRQOL data collection strategies by sites with minimal missing data and conducting regular site audits with feedback). Additionally, reasons for missingness are collected and used to inform analyses to handle missing data. We previously examined variation in HRQOL data collection across INTERMACS sites and found significant intersite variability in missingness because of both patient and administrative reasons, which supports current prevention strategies by INTERMACS and provides future opportunities for more targeted efforts to reduce missingness.26
This study should be interpreted in the context of the following potential limitations. First, there is no way to know the right scores, and our methods only provide an estimate of the potential biases that could be introduced. This supports ongoing efforts to maximize the completeness of data collection. Second, the EQ-5D-3L and KCCQ-12 are not specific for LVAD patients,27 and other studies with missing data for these instruments may have other factors associated with the completeness of data. This suggests that similar approaches to missing data may be warranted to better illuminate the best methods for handling potentially informative data in other heart failure populations. Finally, the precise cause of missing data is unknown. Nonetheless, our analyses provide guidance for future HRQOL studies using the INTERMACS database on strategies to handle missing HRQOL data.
Assessing the magnitude and reasons for missingness and reporting methods used to impute missing data are important to enhance the rigor and interpretation of HRQOL findings. In the setting of substantial missing data because of illness severity, MAR methods do not substantially alter the observed estimates, particularly when there is little correlation with available data for imputation and the missingness of data. MNAR models appear useful as sensitivity analyses to bracket the range of bias that might be introduced after using MAR imputation modeling.2 Although it is not possible to recommend one MNAR sensitivity analysis over another, we recommend use of an MNAR imputation strategy for longitudinal studies that include preimplantation patients with INTERMACS profiles 1 and 2, when MNAR data, attributable to severity of illness, are common. Conduct of MNAR sensitivity analyses may be less critical after mechanical circulatory support implantation, when there are substantially fewer data missing because of the severity of illness. Finally, regarding future studies, caution must be exercised, and uncertainty and the potential for bias must be acknowledged, when interpreting HRQOL findings, when more than trivial amounts of data are missing, especially if MNAR.
Sources of Funding
This work was sponsored by the National Institutes of Health, National Heart, Lung and Blood Institute (NHLBI), Registry of Mechanical Circulatory Support Devices for End-Stage Heart Failure (INTERMACS), Contract No. HHSN268200548198C.
Dr Naftel is a consultant to the HeartWare and Thoratec Corporations. Dr Spertus owns the copyright to the KCCQ and serves as a consultant to the Scientific Advisory Board of United Healthcare, Bayer, and Novartis. The other authors report no conflicts.
↵* S.A. Wissman is deceased.
Guest Editor for this article was Mathew J. Reeves, DVM, PhD.
The editors had no role in the evaluation of the manuscript or in the decision about its acceptance.
The Data Supplement is available at http://circimaging.ahajournals.org/lookup/suppl/doi:10.1161/CIRCOUTCOMES.116.003268/-/DC1.
- Received September 2, 2016.
- Accepted October 30, 2017.
- © 2017 American Heart Association, Inc.
- Li T,
- Hutfless S,
- Scharfstein DO,
- Daniels MJ,
- Hogan JW,
- Little RJ,
- Roy JA,
- Law AH,
- Dickersin K
- 3.↵National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press; 2010.
- Fairclough DL
- Mallinckrodt CH
- 7.↵The Interagency Registry for Mechanically Assisted Circulatory Support. http://www.uab.edu/medicine/intermacs/. Accessed October 10, 2017.
- Grady KL,
- Naftel D,
- Stevenson L,
- Dew MA,
- Weidner G,
- Pagani FD,
- Kirklin JK,
- Myers S,
- Baldwin T,
- Young J
- Grady KL,
- Naftel DC,
- Myers S,
- Dew MA,
- Weidner G,
- Spertus JA,
- Idrissi K,
- Lee HB,
- McGee EC,
- Kirklin JK
- Grady KL,
- Wissman S,
- Naftel DC,
- Myers S,
- Gelijins A,
- Moskowitz A,
- Pagani FD,
- Young JB,
- Spertus JA,
- Kirklin JK
- Spertus JA,
- Jones PG
- 18.↵CV Outcomes, Inc. http://www.cvoutcomes.org/topics/. Accessed October 10, 2017.
- Green CP,
- Porter CB,
- Bresnahan DR,
- Spertus JA
- Rogers JG,
- Aaronson KD,
- Boyle AJ,
- Russell SD,
- Milano CA,
- Pagani FD,
- Edwards BS,
- Park S,
- John R,
- Conte JV,
- Farrar DJ,
- Slaughter MS
- Hastie T,
- Tibshirani R,
- Friedman J
- Harrell FE
- Van Buuren S,
- Groothuis-Oudshoorn K
- Rubin DB
- 25.↵R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2105.
- Gupta BP,
- Grady KL,
- Fendler T,
- Jones PG,
- Spertus JA