doi: 10.1161/CIRCOUTCOMES.108.790634
Primer on Statistical Interpretation or Methods |
Primer on Statistical Interpretation or Methods Report Card on Propensity-Score Matching in the Cardiology Literature From 2004 to 2006
A Systematic Review
From the Institute for Clinical Evaluative Sciences; Dalla Lana School of Public Health, University of Toronto; and Department of Health Policy, Management and Evaluation, University of Toronto, all in Toronto, Ontario, Canada.
Correspondence to Peter C Austin, Institute for Clinical Evaluative Sciences, G1 06, 2075 Bayview Avenue, Toronto, Ontario, M4N 3M5, Canada. E-mail peter.austin{at}ices.on.ca
 |
Abstract |
|---|
 Top Abstract IntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
Background— Propensity-score matching is frequently used in the cardiology literature. Recent systematic reviews have found that this method is, in general, poorly implemented in the medical literature. The study objective was to examine the quality of the implementation of propensity-score matching in the general cardiology literature.
Methods and Results— A total of 44 articles published in the American Heart Journal, the American Journal of Cardiology, Circulation, the European Heart Journal, Heart, the International Journal of Cardiology, and the Journal of the American College of Cardiology between January 1, 2004, and December 31, 2006, were examined. Twenty of the 44 studies did not provide adequate information on how the propensity-score–matched pairs were formed. Fourteen studies did not report whether matching on the propensity score balanced baseline characteristics between treated and untreated subjects in the matched sample. Only 4 studies explicitly used statistical methods appropriate for matched studies to compare baseline characteristics between treated and untreated subjects. Only 11 (25%) of the 44 studies explicitly used statistical methods appropriate for the analysis of matched data when estimating the effect of treatment on the outcomes. Only 2 studies described the matching method used, assessed balance in baseline covariates by appropriate methods, and used appropriate statistical methods to estimate the treatment effect and its significance.
Conclusions— Application of propensity-score matching was poor in the cardiology literature. Suggestions for improving the reporting and analysis of studies that use propensity-score matching are provided.
Key Words: statistics • matching • observational study • propensity score • propensity-score matching
|
Introduction |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
There is a growing interest in using observational studies to examine the effect of exposures or treatments on outcomes; however, nonrandomized studies can be subject to confounding when subjects who receive the treatment differ systematically from untreated subjects. Confounding occurs when factors or characteristics that influence treatment selection are also associated with the outcome of interest. Propensity-score methods increasingly are being used to reduce the impact of confounding in the estimation of causal treatment effects with nonrandomized data.
The propensity score is defined as a subject’s probability of receiving a specific treatment conditional on the observed covariates.1,2 Subjects with the same propensity score have the same distribution of measured covariates, independent of treatment status1–3; however, treated and untreated subjects with the same propensity score can have different distributions of unmeasured variables.4,5 Propensity-score matching6 is a commonly used propensity-score method; however, its use requires appropriate statistical methods.
A recent review published in the statistical literature reported that propensity-score matching was frequently performed poorly in articles published in medical journals between 1996 and 2003.7 Of the 47 studies examined, only 2 fully described the methods used and applied appropriate statistical methods for the analysis of the propensity-score–matched sample. A similar review of the use of propensity-score matching in the cardiovascular surgery literature (Annals of Thoracic Surgery, European Journal of Cardio-thoracic Surgery, Journal of Cardiovascular Surgery, and Journal of Thoracic and Cardiovascular Surgery) between 2004 and 2006 found that of 60 articles examined, none completely and accurately described the statistical methods employed and used appropriate statistical methods for matched data.8
The objective of the present study was to systematically examine the use of propensity-score matching in the general cardiology literature. Suggestions are provided for the conducting and reporting of studies that use propensity-score matching.
|
Statistical Methods for Propensity-Score–Matched Samples |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
There are 3 different steps in a propensity-score–matched analysis.6 First, one must construct the propensity-score–matched sample. Second, one must assess the degree to which matching on the propensity score has resulted in a matched sample in which the distribution of measured baseline variables is similar between treated and untreated subjects. Third, one must estimate the effect of the treatment or exposure on the outcome. Each of these steps is briefly described below.
Propensity-Score Matching
The propensity score is frequently estimated by use of a logistic regression model in which treatment status is regressed on baseline characteristics. The estimated propensity score is the model-based predicted probability of exposure to the treatment. Once the propensity score has been estimated for each subject, treated and untreated subjects are matched on the propensity score. Frequently, nearest-neighbor matching within a specified caliper width is used. With this method, the first randomly selected treated subject is matched to the untreated subject with the closest propensity score within a specified range (the caliper width). If multiple untreated subjects are equally close to this treated subject, then 1 of these untreated subjects is randomly selected for matching to this treated subject. This process is repeated until all possible matches have been formed. If for a given treated subject, no available untreated subject lies within the specified caliper width, then that treated subject is excluded from the matched sample. Similarly, unmatched untreated subjects are excluded from the matched sample. Another common approach is to attempt to match treated and untreated subjects on the first 5 digits of the propensity score. If this cannot be done for a specific treated subject, then matching is attempted on the first 4 digits. If no suitable untreated subject exists, then matches are attempted on the first 3, first 2, and finally, the first digit of the propensity score.9 This method is referred to as 5
1 digit matching. The above strategies all use a "greedy" matching algorithm, regardless of the caliper width that is used. The term "greedy" implies that at a given step of the matching process, the nearest untreated subject is selected for matching to a given treated subject, even if that untreated subject would have served better as a match for a subsequent treated subject. An alternative to the use of greedy matching is to use "optimal" matching, in which matched pairs are formed to minimize the sum of the pairwise differences in the propensity score.10
Statistical Methods for Assessing Balance
The performance of propensity-score matching is assessed by the degree to which matching on the estimated propensity score results in a matched sample in which the distribution of measured baseline characteristic is similar between treated and untreated subjects. In prior systematic reviews, greater details were provided on appropriate methods for comparing balance in matched samples.7,8 Both Imai et al11 and Ho et al12 suggest that statistical hypothesis testing is not appropriate. There are 2 reasons for this. First, balance is a property of a sample and not of a hypothetical superpopulation. Second, significance testing and probability values are influenced by sample size. Matching results in a diminished sample size. Thus, when statistical hypothesis testing is used to assess balance in measured variables between treated and untreated subjects in the matched sample, apparent improvements in balance may occur solely because of a reduction in sample size.
A popular method for assessing balance in measured baseline covariates that is appropriate for use in matched samples is the standardized difference, which is the difference in means in units of the pooled standard deviation (SD).5,6,13 Unlike probability values, the standardized difference is not confounded with sample size, and thus, balance in the initial sample can be compared with that in the matched sample. The standardized difference can also be used to compare the relative balance of variables measured in different units.
Estimating the Treatment Effect
The final analytic step is to estimate the effect of treatment on the outcome. Propensity-score matching entails matching treated and untreated subjects with similar propensity scores. Rosenbaum and Rubin1 demonstrated that conditional on the propensity score, measured baseline covariates are independent of treatment assignment. Therefore, on average, treated and untreated subjects with the same propensity score do not differ systematically from one another. In observational studies, confounding results in treated subjects differing systematically from untreated subjects. Therefore, matched treated and untreated subjects will, on average, be more similar than randomly selected treated and untreated subjects. Thus, by design, the propensity-score–matched sample does not consist of independent observations but of pairs of subjects who share a correlation in their baseline covariates and thus likely have correlated outcomes. Appropriate statistical methods for estimating the magnitude and statistical significance of the effect of treatment on outcomes are described in the prior reviews.7,8 The performance of a selection of statistical methods for use of the propensity score to estimate the effect of treatment on binary, time-to-event, and count outcomes is described elsewhere.14–16
|
Survey of Propensity-Score Matching in the Cardiology Literature |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
Identification of Published Articles That Used Propensity-Score Matching
A search strategy similar to that of recently published systematic reviews of propensity-score methods was used.8,17 Both PubMed and the Science Citation Index were searched to identify studies that used propensity-score matching. PubMed was used to identify articles with the keyword "propensity." The search was restricted to articles published between January 1, 2004, and December 31, 2006, in the following general cardiology journals: American Heart Journal, American Journal of Cardiology, Circulation, European Heart Journal, Heart, International Journal of Cardiology, and Journal of the American College of Cardiology. In addition, the Science Citation Index was used to search for articles that cited at least 1 of several important papers on propensity-score methods (see a prior review for greater details on the search strategy17). The combined search identified 140 articles. A total of 140 articles were examined, and only those that used propensity-score matching were selected. One study that used propensity-score matching only to generate a sample for subsequent simulation analyses18 and 2 articles that reported using matching but that actually used stratification on the propensity score19,20 were excluded. The search resulted in the identification of 44 studies21–64 that used propensity-score matching in the following journals: American Heart Journal (6 articles), American Journal of Cardiology (16 articles), Circulation (12 articles), European Heart Journal (4 articles), International Journal of Cardiology (2 articles), and Journal of the American College of Cardiology (4 articles).
Abstraction of Analytic Methods in Propensity-Score–Matched Samples
The following information was abstracted from each of the published articles: (1) the method by which propensity-score–matched pairs were formed; (2) whether the authors assessed the balance in measured baseline characteristics between treated and untreated subjects in the matched sample, and when the authors did so, whether appropriate statistical methods were used; and (3) the statistical methods used to assess the effect of treatment on the outcome and whether the method was appropriate for matched data. The same criteria for evaluating articles were used as in the prior reviews of propensity-score matching.7,8
|
Results of Systematic Review |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
Propensity-Score Matching
Twenty-four studies fully described how matches were formed. Among these studies, 11 used 5
1 digit matching,* 1 study matched on the logit of the propensity score using calipers of width 0.2 SDs of the logit of the propensity score,44 and other studies matched on the propensity score using the following calipers: 0.02 (3 studies)41,49,58 and 0.01 (8 studies).38,47,48,54,56,61–63 One study required exact matching on the propensity score.53 Twenty (45%) of the studies did not report the manner by which propensity-score–matched pairs were formed.
Assessing Balance Between Treated and Untreated Subjects
Fourteen studies (32%) did not report whether matching on the propensity score resulted in a matched sample in which the distribution of baseline variables was similar between treated and untreated subjects. Seven additional studies reported that balance was achieved but did not report a table that allowed the reader to compare the distribution of baseline characteristics between treated and untreated subjects in the matched sample. The remaining 23 studies (52%) reported a table in which the distribution of baseline characteristics was compared between treated and untreated subjects.
Of the 30 studies (7 that did not report a table and 23 that did) that reported comparing the distribution of baseline characteristics between treated and untreated subjects, 19 used statistical significance testing, 4 used both standardized differences and significance testing,21,38,55,58 3 relied on visual comparison, and 4 did not report what was done. Thus, no studies used only methods appropriate for matched data. Each of the studies that used standardized differences supplemented this method with the use of statistical significance testing. We acknowledge, as a limitation, that in the 1 instance when significance testing that accounted for the matched nature of sample was done in addition to reporting of standardized differences,38 this insertion may have been either due to convention or at the journal’s request.
Estimating the Effect of Treatment or Exposure on the Outcome
Eleven (25%) of the 44 articles explicitly stated that methods appropriate for the analysis of matched data were used in estimating the treatment effect and its statistical significance. These studies used McNemar’s test,29,34,35 regression models estimated with generalized estimating equation methods to account for the matched-pairs nature of the sample,43,57 Cox proportional hazards regression stratified on the matched pairs,21,44,55,58 the signed rank test,34 and methods that used the bootstrap.41,60 Three additional studies used methods appropriate for correlated data for some outcomes but not for others.28,30,64
Twenty-one studies (48%) explicitly used inappropriate statistical methods to assess the statistical significance of the effect of treatment on the outcomes. Common errors included the use of the log-rank test to compare Kaplan-Meier survival curves in the matched sample, use of Cox proportional hazards models in the matched sample, use of logistic regression in the matched sample, use of 
2 tests to compare proportions in the matched sample, and use of Wilcoxon rank sum tests or standard t tests to compare continuous variables in the matched sample. Nine studies did not adequately describe the statistical methods that were used to compare the outcome between treated and untreated subjects.25,27,32,33,37,39,40,47,56 Frequently, these studies were comparing proportions or Kaplan-Meier survival curves between treated and untreated subjects.
|
Discussion |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
The objective of the present study was to critically examine the use of propensity-score matching in the general cardiology literature. Only 4 of the 44 studies compared the distribution of baseline variables between treated and untreated subjects in the matched sample and explicitly documented the use of appropriate statistical methods to assess whether measured characteristics were balanced between treated and untreated subjects in the matched sample. These four studies used standardized differences, which are appropriate in this context. Eleven (25%) of the 44 studies explicitly used appropriate statistical methods for all analyses that examined the impact of treatment on outcomes. Overall, only 2 studies55,58 described the matching method that was used and explicitly used appropriate statistical methods both for assessing balance in the matched sample and for assessing the statistical significance of the treatment effect; however, both of these studies, in addition to using standardized differences to assess baseline balance, also used statistical hypothesis testing that did not account for the matched nature of the sample. The remaining 42 articles were subject to either incomplete reporting of methods or inappropriate statistical methods. Below are recommendations for the design, analysis, and reporting of studies that use propensity-score matching. These recommendations are summarized in the Table.
|
Describe the Matching Method
The method by which the propensity-score–matched pairs were formed should be described explicitly. This allows other researchers to replicate the study methods. It is insufficient to state only that greedy matching was done or that nearest-neighbor matching was used. A recent study found that matching on the logit of the propensity score with a caliper width of 0.2 of the SD of the logit of the propensity score tended to have superior performance to other methods that were identified as being used in the medical literature.65 Furthermore, methods based on matching on the logit of the propensity score with calipers that are a proportion of the SD have stronger theoretical justification.66
Report the Balance of Baseline Variables Between Treated and Untreated Subjects in the Matched Sample
The propensity score is a balancing score: Matching treated and untreated subjects on the propensity score allows one to create a matched sample in which the distribution of baseline variables is similar between treated and untreated subjects. The test of whether the propensity-score model has been specified adequately is whether matching on the estimated propensity score has created a matched sample in which the distribution of measured baseline variables is similar between treated and untreated subjects. Describing the balance in measured variables between treated and untreated subjects in the matched sample allows both the researcher and readers to assess whether the propensity model was specified adequately. Researchers are encouraged to use standardized differences, rather than hypothesis testing, for assessing balance in baseline covariates between treated and untreated subjects. The receiver operating characteristic curve area (c-statistic) of the propensity-score model is not a measure of the goodness-of-fit of the propensity-score model. Indeed, recent research has shown that the receiver operating characteristic curve area of the propensity-score model provides no information on whether an important confounder has been omitted.5
Statistical Methods for Estimating the Effect of Treatment on Outcomes
Researchers should explicitly document that methods appropriate for the analysis of matched data were used. In 2 prior reviews,7,8 appropriate methods for estimating the effect of the treatment on the outcome in propensity-score–matched samples were described. Because matching on the propensity score allows one to mimic some of the characteristics of a randomized, controlled trial, researchers are encouraged to report measures of effect that are commonly reported in randomized, controlled trials: differences in means, risk differences, and relative risks. Because the matched sample has reduced or eliminated systematic differences in measured variables between treated and untreated subjects, regression-based methods may not be necessary. Researchers are encouraged to report risk differences or relative risks rather than odds ratios.
This constitutes the third systematic review of propensity-score matching that I have conducted. The first review was of articles that were published in an earlier era (1996–2003).7 In that earlier review, 4% of articles used appropriate methods to compare baseline characteristics between treated and untreated subjects. Furthermore, 28% of studies used appropriate statistical methods to estimate the significance of the effect of treatment on outcomes.7 In the present review, there was no meaningful improvement in statistical practice in a more recent era. The second review was of articles published in the same era as in the present review (2004–2006) but was restricted to articles in the cardiovascular surgical literature.8 In the cardiovascular surgery literature, no studies used appropriate methods to compare baseline characteristics between treated and untreated subjects, whereas 9% of articles in the cardiology literature used appropriate methods. In the cardiovascular surgery literature, 13% of studies explicitly used appropriate statistical methods to estimate the effect of treatment on outcomes, whereas 25% of studies in the cardiology literature used appropriate methods. Therefore, the implementation of propensity-score matching appears to be of somewhat higher quality in the cardiology literature than in the cardiovascular surgery literature; however, there still remains substantial room for improvements in both literatures. One of the primary objectives of the present systematic review is educational: to highlight and describe good statistical practice in the context of propensity-score matching.
In the present review, explicit criteria were used to assess the statistical methods used in studies that employ propensity-score matching. In particular, on the basis of arguments by Imai et al11 and Ho et al12 that significance testing is inappropriate for assessing balance in baseline covariates in matched samples, it was required that methods be used that were not based on statistical hypothesis testing. In addition, it was required that tests of the effect of treatment on outcomes should account for the matched-pairs nature of the sample. Although strong arguments can be made for both of these criteria, disagreement persists within the statistical community about acceptable practices for studies that use propensity-score matching. For example, in response to an earlier review, Hansen67 argued that the use of balance tests that incorporate hypothesis testing is acceptable. In a similar commentary, Hill68 acknowledged that matching on the propensity score induces dependence within the matched sample and that a matched-pairs analysis is an appropriate way to account for this lack of independence; however, she expressed disagreement with the requirement that a matched-pairs analysis is the only way to account for this lack of independence.68 The reader is referred elsewhere for the response to these commentaries.69
|
Summary |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
In conclusion, propensity-score matching tended to be poorly implemented in the cardiology literature. The majority of studies ignored the matched nature of the propensity-score–matched sample in the subsequent analyses. Suggestions for improving the analysis of propensity-score–matched samples and for improving the reporting of these analyses have been provided.
|
Acknowledgments |
|---|
Disclosures
The Institute for Clinical Evaluative Sciences is supported in part by a grant from the Ontario Ministry of Health and Long Term Care. The opinions, results, and conclusions are those of the author, and no endorsement by the Ministry of Health and Long-Term Care or by the Institute for Clinical Evaluative Sciences is intended or should be inferred. Dr Austin is supported by a Career Investigator Award from the Heart and Stroke Foundation of Ontario.
|
Footnotes |
|---|
*References 22–24, 31, 32, 36, 45, 46, 51, 55, 57.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
|
References |
|---|
|
TopAbstractIntroductionStatistical Methods for...Survey of Propensity-Score...Results of Systematic ReviewDiscussionSummaryReferences |
|---|
1. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983; 70: 41–55.
2. Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc. 1984; 79: 516–524.[CrossRef]
3. Blackstone EH. Comparing apples and oranges. J Thorac Cardiovasc Surg. 2002; 123: 8–15.
4. Austin PC, Mamdani MM, Stukel TA, Anderson GM, Tu JV. The use of the propensity score for estimating treatment effects: administrative versus clinical data. Stat Med. 2005; 24: 1563–1578.[CrossRef][Medline]
5. Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med. 2007; 26: 734–753.[CrossRef][Medline]
6. Austin PC, Mamdani MM. A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use. Stat Med. 2006; 25: 2084–2106.[CrossRef][Medline]
7. Austin PC. A critical appraisal of propensity score matching in the medical literature between 1996 and 2003. Stat Med. 2008; 27: 2037–2049.[CrossRef][Medline]
8. Austin PC. Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. J Thorac Cardiovasc Surg. 2007; 134: 1128–1135.
9. Parson LS. Reducing bias in a propensity score matched-pair sample using greedy matching techniques. In: Proceedings of the Twenty-Sixth Annual SAS Users Group International Conference. Cary, NC: SAS Institute; 2001: 214–216.
10. Rosenbaum PR. Observational Studies. New York, NY: Springer-Verlag; 1995.
11. Imai K, King G, Stuart EA. Misunderstandings between experimentalists and observationalists about causal inference. J R Stat Soc Ser A Stat Soc. 2008; 171: 481–502.[CrossRef]
12. Ho DE, Imai K, King G, Stuart EA. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Anal. 2007; 15: 199–236.
13. Flury BK, Riedwyl H. Standard distance in univariate and multivariate analysis. Am Stat. 1986; 40: 249–251.[CrossRef]
14. Austin PC. The performance of different propensity score methods for estimating marginal odds ratios. Stat Med. 2007; 26: 3078–3094.[CrossRef][Medline]
15. Austin PC, Grootendorst P, Normand SLT, Anderson GM. Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study. Stat Med. 2007; 26: 754–768.[CrossRef][Medline]
16. Austin PC. The performance of different propensity score methods for estimating relative risks. J Clin Epidemiol. 2008; 61: 537–545.[CrossRef][Medline]
17. Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006; 59: 437–447.[Medline]
18. Austin PC, Mamdani MM, Juurlink DN, Alter DA, Tu JV. Missed opportunities in the secondary prevention of myocardial infarction: an assessment of the effects of statin underprescribing on mortality. Am Heart J. 2006; 151: 969–975.[CrossRef][Medline]
19. Nishida H, Tomizawa Y, Endo M, Kurosawa H. Survival benefit of exclusive use of in situ arterial conduits over combined use of arterial and vein grafts for multiple coronary artery bypass grafting. Circulation. 2005; 112 (suppl I): I-299–I-303.[CrossRef][Medline]
20. Thielman M, Leyh R, Massoudy P, Neuhauser M, Aleksic I, Kamler M, Herold U, Piotrowski J, Jakob H. Prognostic significance of multiple previous percutaneous coronary interventions in patients undergoing elective coronary artery bypass surgery. Circulation. 2006; 114 (suppl I): I-441–I-447.[Medline]
21. Ahmed A, Perry GJ, Fleg JL, Love TE, Goff DC Jr, Kitzman DW. Outcomes in ambulatory chronic systolic and diastolic heart failure: a propensity score analysis. Am Heart J. 2006; 152: 956–966.[CrossRef][Medline]
22. Baskett RJ, O'Connor GT, Hirsch GM, Ghali WA, Sabadosa KA, Morton JR, Ross CS, Hernandez F, Nugent WC, Lahey SJ, Sisto D, Dacey LJ, Klemperer JD, Helm RE, Maitland A. The preoperative intraaortic balloon pump in coronary bypass surgery: a lack of evidence of effectiveness. Am Heart J. 2005; 150: 1122–1127.[CrossRef][Medline]
23. Wang A, Pappas P, Anstrom KJ, Abrutyn E, Fowler VG Jr, Hoen B, Miro JM, Corey GR, Olaison L, Stafford JA, Mestres CA, Cabell CH; International Collaboration on Endocarditis Investigators. The use and effect of surgical therapy for prosthetic valve infective endocarditis: a propensity analysis of a multicenter, international cohort. Am Heart J. 2005; 150: 1086–1091.[CrossRef][Medline]
24. Hassan A, Buth KJ, Baskett RJ, Ali IS, Maitland A, Sullivan JA, Ghali WA, Hirsch GM. The association between prior percutaneous coronary intervention and short-term outcomes after coronary artery bypass grafting. Am Heart J. 2005; 150: 1026–1031.[CrossRef][Medline]
25. Meine TJ, Roe MT, Chen AY, Patel MR, Washam JB, Ohman EM, Peacock WF, Pollack CV Jr, Gibler WB, Peterson ED; CRUSADE Investigators. Association of intravenous morphine use and outcomes in acute coronary syndromes: results from the CRUSADE Quality Improvement Initiative. Am Heart J. 2005; 149: 1043–1049.[CrossRef][Medline]
26. Ali IS, Buth KJ, Maitland A. Impact of preoperative intravenous nitroglycerin on in-hospital outcomes after coronary artery bypass grafting for unstable angina. Am Heart J. 2004; 148: 727–732.[CrossRef][Medline]
27. Moses JW, Nikolsky E, Mehran R, Cambier PA, Bachinsky WB, Leya F, Kuntz RE, Popma JJ, Schleckser P, Wang H, Cohen SA, Leon MB; SIRIUS 2.25 Investigators. Safety and efficacy of the 2.25-mm sirolimus-eluting Bx Velocity stent in the treatment of patients with de novo native coronary artery lesions: the SIRIUS 2.25 trial. Am J Cardiol. 2006; 98: 1455–1460.[CrossRef][Medline]
28. Stamou SC, White T, Barnett S, Boyce SW, Corso PJ, Lefrak EA. Comparisons of cardiac surgery outcomes in Jehovah’s versus Non-Jehovah’s Witnesses. Am J Cardiol. 2006; 98: 1223–1225.[CrossRef][Medline]
29. Bromberg-Marin G, Marin-Neto JA, Parsons LS, Canto JG, Rogers WJ; National Registry of Myocardial Infarction-4. Effectiveness and safety of glycoprotein IIb/IIIa inhibitors and clopidogrel alone and in combination in non-ST-segment elevation myocardial infarction (from the National Registry of Myocardial Infarction-4). Am J Cardiol. 2006; 98: 1125–1131.[CrossRef][Medline]
30. Shishehbor MH, Topol EJ, Mukherjee D, Hu T, Cohen DJ, Stone GW, McClure R, Roffi M, Moliterno DJ; TARGET Investigators. Outcome of multivessel coronary intervention in the contemporary percutaneous revascularization era. Am J Cardiol. 2006; 97: 1585–1590.[CrossRef][Medline]
31. Kandzari DE, Tuttle RH, Zidar JP, Jollis JG. Comparison of long-term (seven year) outcomes among patients undergoing percutaneous coronary revascularization with versus without stenting. Am J Cardiol. 2006; 97: 1467–1472.[CrossRef][Medline]
32. Kugelmass AD, Cohen DJ, Brown PP, Simon AW, Becker ER, Culler SD. Hospital resources consumed in treating complications associated with percutaneous coronary interventions. Am J Cardiol. 2006; 97: 322–327.[CrossRef][Medline]
33. Shishehbor MH, Seshadri N, Aktas M, Acharya N, Gillinov AM, Blackstone EH, Houghtaling PL, Migrino RQ, Ghaffari S. Comparison of outcomes in patients undergoing coronary bypass of patent versus restenosed bare metal stented coronary arteries. Am J Cardiol. 2005; 96: 1416–1419.[CrossRef][Medline]
34. Shavelle DM, Rasouli ML, Frederick P, Gibson CM, French WJ; National Registry of Myocardial Infarction Investigators. Outcome in patients transferred for percutaneous coronary intervention (a National Registry of Myocardial Infarction 2/3/4 analysis). Am J Cardiol. 2005; 96: 1227–1232.[CrossRef][Medline]
35. Fonarow GC, Wright RS, Spencer FA, Fredrick PD, Dong W, Every N, French WJ; National Registry of Myocardial Infarction 4 Investigators. Effect of statin use within the first 24 hours of admission for acute myocardial infarction on early morbidity and mortality. Am J Cardiol. 2005; 96: 611–616.[CrossRef][Medline]
36. Meine TJ, Patel MR, Washam JB, Pappas PA, Jollis JG. Safety and effectiveness of transdermal nicotine patch in smokers admitted with acute coronary syndromes. Am J Cardiol. 2005; 95: 976–978.[CrossRef][Medline]
37. Ferrieres J, Cambou JP, Gueret P, Boutalbi Y, Lablanche JM, Hanania G, Genes N, Cantet C, Danchin N. Effect of early initiation of statins on survival in patients with acute myocardial infarction (the USIC 2000 Registry). Am J Cardiol. 2005; 95: 486–489.[CrossRef][Medline]
38. Nunez E, Arnett DK, Benjamin EJ, Oakes JM, Liebson PR, Skelton TN. Comparison of the prognostic value of left ventricular hypertrophy in African-American men versus women. Am J Cardiol. 2004; 94: 1383–1390.[CrossRef][Medline]
39. Muhlestein JB, Anderson JL, Horne BD, Carlquist JF, Bair TL, Bunch TJ, Pearson RR; Intermountain Heart Collaborative Study Group. Early effects of statins in patients with coronary artery disease and high C-reactive protein. Am J Cardiol. 2004; 94: 1107–1112.[CrossRef][Medline]
40. Bartholomew BA, Harjai KJ, Dukkipati S, Boura JA, Yerkey MW, Glazier S, Grines CL, O'Neill WW. Impact of nephropathy after percutaneous coronary intervention and a method for risk stratification. Am J Cardiol. 2004; 93: 1515–1519.[CrossRef][Medline]
41. Klinke WP, Hilton JD, Warburton RN, Warburton WP, Tan RP. Comparison of treatment outcomes in patients > or =80 years undergoing transradial versus transfemoral coronary intervention. Am J Cardiol. 2004; 93: 1282–1285.[CrossRef][Medline]
42. Bhatheja R; Francis GS; Pothier CE; Lauer MS. Heart rate response during dipyridamole stress as a predictor of mortality in patients with normal myocardial perfusion and normal electrocardiograms. Am J Cardiol. 2005; 95: 1159–1164.[CrossRef][Medline]
43. Kapetanakis EI, Medlam DA, Petro KR, Haile E, Hill PC, Dullum MK, Bafi AS, Boyce SW, Corso PJ. Effect of clopidogrel premedication in off-pump cardiac surgery: are we forfeiting the benefits of reduced hemorrhagic sequelae? Circulation. 2006; 113: 1667–1674.
44. Guru V, Fremes SE, Austin PC, Blackstone EH, Tu JV. Gender differences in outcomes after hospital discharge from coronary artery bypass grafting. Circulation. 2006; 113: 507–516.
45. Tarakji KG, Brunken R, McCarthy PM, Al-Chekakie MO, Abdel-Latif A, Pothier CE, Blackstone EH, Lauer MS. Myocardial viability testing and the effect of early intervention in patients with advanced left ventricular systolic dysfunction. Circulation. 2006; 113: 230–237.
46. Mukamal KJ, Maclure M, Muller JE, Mittleman MA. Binge drinking and mortality after acute myocardial infarction. Circulation. 2005; 112: 3839–3845.
47. Zacharias A, Schwann TA, Riordan CJ, Durham SJ, Shah AS, Habib RH. Obesity and risk of new-onset atrial fibrillation after cardiac surgery. Circulation. 2005; 112: 3247–3255.
48. Toumpoulis IK, Anagnostopoulos CE, Katritsis DG, DeRose JJ Jr, Swistel DG. The impact of preoperative thrombolysis on long-term survival after coronary artery bypass grafting. Circulation. 2005; 112 (suppl I): I-351–I-357.[Medline]
49. Fukuta H, Sane DC, Brucks S, Little WC. Statin therapy may be associated with lower mortality in patients with diastolic heart failure: a preliminary report. Circulation. 2005; 112: 357–363.
50. Chen MS, Blackstone EH, Pothier CE, Lauer MS. Heart rate recovery and impact of myocardial revascularization on long-term mortality. Circulation. 2004; 110: 2851–2857.
51. Kaul P, Armstrong PW, Chang WC, Naylor CD, Granger CB, Lee KL, Peterson ED, Califf RM, Topol EJ, Mark DB. Long-term mortality of patients with acute myocardial infarction in the United States and Canada: comparison of patients enrolled in Global Utilization of Streptokinase and t-PA for Occluded Coronary Arteries (GUSTO)-I. Circulation. 2004; 110: 1754–1760.
52. Pan W, Pintar T, Anton J, Lee VV, Vaughn WK, Collard CD. Statins are associated with a reduced incidence of perioperative mortality after coronary artery bypass graft surgery. Circulation. 2004; 110 (suppl II): II-45–II-49.[Medline]
53. Mack MJ, Brown P, Houser F, Katz M, Kugelmass A, Simon A, Battaglia S, Tarkington L, Culler S, Becker E. On-pump versus off-pump coronary artery bypass surgery in a matched sample of women: a comparison of outcomes. Circulation. 2004; 110 (suppl II): II-1–II-6.[Medline]
54. Zacharias A, Habib RH, Schwann TA, Riordan CJ, Durham SJ, Shah A. Improved survival with radial artery versus vein conduits in coronary bypass surgery with left internal thoracic artery to left anterior descending artery grafting. Circulation. 2004; 109: 1489–1496.
55. Ahmed A, Husain A, Love TE, Gambassi G, Dell'Italia LJ, Francis GS, Gheorghiade M, Allman RM, Meleth S, Bourge RC. Heart failure, chronic diuretic use, and increase in mortality and hospitalization: an observational study using propensity score methods. Eur Heart J. 2006; 27: 1431–1439.
56. Spaulding C, Morice MC, Lancelin B, El Haddad S, Lepage E, Bataille S, Tresca JP, Mouranche X, Fosse S, Monchi M, de Vernejoul N; CARDIO-ARIF Registry Investigators. Is the volume-outcome relation still an issue in the era of PCI with systematic stenting? Results of the greater Paris area PCI registry. Eur Heart J. 2006; 27: 1054–1060.
57. Kapetanakis EI, Medlam DA, Boyce SW, Haile E, Hill PC, Dullum MK, Bafi AS, Petro KR, Corso PJ. Clopidogrel administration prior to coronary artery bypass grafting surgery: the cardiologist’s panacea or the surgeon’s headache? Eur Heart J. 2005; 26: 576–583.
58. Ahmed A, Rich MW, Love TE, Lloyd-Jones DM, Aban IB, Colucci WS, Adams KF, Gheorghiade M. Digoxin and reduction in mortality and hospitalization in heart failure: a comprehensive post hoc analysis of the DIG trial. Eur Heart J. 2006; 27: 178–186.
59. Ali IS, Buth KJ. Preoperative statin use and outcomes following cardiac surgery. Int J Cardiol. 2005; 103: 12–18.[CrossRef][Medline]
60. Delea TE, Stanford R, Hagiwara M, Edelsberg JS, Oster G. Death and hospitalization in heart failure patients receiving carvedilol vs. metoprolol tartrate. Int J Cardiol. 2005; 99: 117–124.[CrossRef][Medline]
61. Zacharias A, Schwann TA, Riordan CJ, Durham SJ, Shah A, Habib RH. Operative and late coronary artery bypass grafting outcomes in matched African-American versus Caucasian patients: evidence of a late survival-Medicaid association. J Am Coll Cardiol. 2005; 46: 1526–1535.
62. L'Allier PL, Ducharme A, Keller PF, Yu H, Guertin MC, Tardif JC. Angiotensin-converting enzyme inhibition in hypertensive patients is associated with a reduction in the occurrence of atrial fibrillation. J Am Coll Cardiol. 2004; 44: 159–164.
63. Quinn MJ, Aronow HD, Califf RM, Bhatt DL, Sapp S, Kleiman NS, Harrington RA, Kong DF, Kandzari DE, Topol EJ. Aspirin dose and six-month outcome after an acute coronary syndrome. J Am Coll Cardiol. 2004; 43: 972–978.
64. Ting HH; Raveendran G; Lenno RJ; Long KH; Singh M; Wood DL; Gersh BJ; Rihal CS; Holmes DR. A total of 1,007 percutaneous coronary interventions without onsite cardiac surgery. J Am Coll Cardiol. 2006; 47: 1713–1721.
65. Austin PC. The performance of different propensity-score matching methods used in the medical literature. Biom J. In press.
66. Cochran WG, Rubin DB. Controlling bias in observational studies: a review. Sankhya Ser A. 1973; 35: 417–446.
67. Hansen BB. The essential role of balance tests in propensity-matched observational studies: comments on "A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003" by Peter Austin. Stat Med. 2008; 27: 2050–2054.[CrossRef][Medline]
68. Hill J. Discussion of research using propensity-score matching: comments on "A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003" by Peter Austin. Stat Med. 2008; 27: 2055–2061.[CrossRef][Medline]
69. Austin PC. Rejoinder discussion of "A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003." Stat Med. 2008; 27: 2066–2069.[CrossRef]
This article has been cited by other articles:
 |
|
 |
|
  H. M. Krumholz One Year at Circulation: Cardiovascular Quality and Outcomes Circ Cardiovasc Qual Outcomes, September 1, 2009; 2(5): 399 - 401. [Full Text] [PDF]
|
|
|
|
 |
|
 T. A. Stukel and D. A. Alter Analysis methods for observational studies effects of cardiac rehabilitation on mortality of coronary patients. J. Am. Coll. Cardiol., June 30, 2009; 54(1): 34 - 35. [Full Text] [PDF]
|
|
|
|
|
|
H. M. Krumholz Circulation: Cardiovascular Quality and Outcomes: Scholarship to Improve Health and Health Care for Patients and Populations Circ Cardiovasc Qual Outcomes, September 1, 2008; 1(1): 1 - 3. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||