Use of Mixed Methods Research in Research on Coronary Artery Disease, Diabetes Mellitus, and Hypertension
A Scoping Review
Background—Mixed methods research, the use of both qualitative and quantitative methods within 1 program of study, is becoming increasingly popular to allow investigators to explore patient experiences (qualitative) and also measure outcomes (quantitative). Coronary artery disease and its risk factors are some of the most studied conditions; however, the extent to which mixed methods studies are being conducted in these content areas is unknown. We sought to comprehensively describe the characteristics of published mixed methods studies on coronary artery disease and major risk factors (diabetes mellitus and hypertension).
Methods and Results—We conducted a scoping review of the literature indexed in PubMed, Medline, EMBASE, and CINAHL. We identified 811 abstracts for screening, of which 254 articles underwent full-text review and 97 reports of 81 studies met criteria for inclusion. The majority of studies in this area were conducted in the past 10 years by nurse researchers from the United States and United Kingdom. Diabetes mellitus was the most common content area for mixed methods investigation (compared with coronary artery disease and hypertension). Most authors described their rationale for using mixed methods as complementarity and did not describe study priority or how they reconciled differences in methodological paradigms. Some mixed methods study designs were more commonly used than others, including concurrent timing and integration at the interpretation stage. Qualitative strands were most commonly descriptive studies using interviews for data collection. Quantitative strands were most commonly cross-sectional observational studies, which relied heavily on self-report data such as surveys and scales.
Conclusions—Although mixed methods research is becoming increasingly popular in the area of coronary artery disease and its risk factors, many of the more advanced mixed methods, qualitative, and quantitative techniques have not been commonly used in these areas.
- cardiovascular diseases
- coronary artery disease
- diabetes mellitus
- mixed methods research
- qualitative research
- research design
WHAT IS KNOWN
MMR is the use and integration of both qualitative and quantitative research methodologies within 1 research study.
MMR is especially important for understanding complex patient behaviors in chronic diseases, such as CAD, diabetes mellitus, and hypertension.
WHAT THE STUDY ADDS
The use of mixed methods in cardiovascular disease research is on the rise, especially by nurse researchers in the United States and United Kingdom and predominantly in studies focused on diabetes mellitus and CAD, rather than hypertension.
A variety of study designs have been used, but more advanced mixed methods designs and techniques (eg, early integration and multiphase study design) are used less frequently.
Those undertaking MMR should be cognizant to label their research as such and to follow recommended procedures for mixed methods studies, such as explicit integration of qualitative and quantitative data.
Mixed methods research (MMR)—the use of both qualitative and quantitative methods to address an omnibus research question—has been advanced as the third research paradigm alongside traditional research using singular qualitative or quantitative methods.1,2 In this new paradigm, both qualitative and quantitative data are collected within a single research agenda.3 The different methodological strands of the research may be conducted concurrently or sequentially,4 and relative emphasis may be on one component or the other,5 with some scholars advocating that equal weight may be applied to each component.6 A commonly described purpose of mixing research methods is to come to a more complete description of the phenomenon of interest.7
MMR was pioneered in the fields of education and social research approximately 25 years ago.3 More recently, this methodological approach has begun to be applied in the realm of health research. Despite the well-described methodological challenges of combining research paradigms,8,9 the uptake of mixed methods in health research has been quite swift. In fact, from 2000 to 2009, there were more mixed methods publications in health than in any other discipline.10 In part, the rapid adoption of MMR may be related to the complexity of health-related problems.11,12
Researchers have conducted metareviews of mixed methods studies, to summarize the numbers and types of mixed methods studies which have been published.10 Several health-related metareviews have examined the proposed use of MMR by analyzing data from funding bodies.13,14 One published metareview included an analysis of all published articles in 4 health services research journals,15 and another aimed to review all published mixed methods studies within the area of health research.16 The use of various mixed methods designs and techniques in published reports of healthcare research has been well summarized within the field of mental health,17 but focused metareviews of published mixed methods studies within other topic areas in health research, such as cardiovascular diseases, are lacking.
MMR has been advocated as being particularly useful in research focused on chronic diseases, such as coronary artery disease (CAD), diabetes mellitus, and hypertension, to quantify the effectiveness of treatments and qualify the illness experience.18 The primary and secondary prevention of CAD is complex and multifactorial encompassing both biophysical aspects, such as adequate blood pressure and glycemic control, as well as psychosocial aspects, such as tobacco cessation and emotional health, making this area especially amenable to MMR.19 MMR may provide valuable insights into individual behaviors which contribute to the uptake of preventive behaviors and may help increase understanding of both the magnitude of problems (via quantitative methods) and of patient-related issues, such as experiences and reasons for nonadherence (via qualitative methods). No review of the MMR literature related to CAD and its risk factors currently exists. Therefore, we sought to comprehensively describe the characteristics of published mixed methods studies on CAD and related risk factors (diabetes mellitus and hypertension).
We used the 5-stage protocol for conducting scoping studies described by Arksey and O’Malley,20 the sentinel article in describing scoping review methodology. Researchers undertake scoping reviews to examine the extent, range, and nature of research activity around a certain topic.20 A scoping review differs from a systematic review in that a broader range of study types are included (ie, not limited to randomized trials), and a formal assessment of study quality is not undertaken as part of the inclusion criteria.
Stage 1: Identifying the Research Question
We decided to broaden the topic of our review from CAD alone to also include several of its most important risk factors: diabetes mellitus and hypertension. The reason we chose these risk factors and not others (ie, smoking) is that the optimal management of each these conditions is managed in similar ways: adherence to preventive medications, regular medical and allied healthcare follow-up, and healthy behavior changes.
Among patients with CAD, diabetes mellitus and hypertension we sought to (1) determine the extent of use of MMR (by disease type and date of publication), (2) describe which researchers are using mixed methods designs (by country and discipline), (3) describe the MMR designs commonly used (including rationale for using MMR, study sequence, and priority), and (4) describe the commonly used study designs and data collection techniques for both qualitative and quantitative components.
Stage 2: Identifying Relevant Studies
We searched the following electronic databases: Pubmed, Medline, EMBASE, and CINAHL. We did not search the grey literature as we were explicitly interested in the use of MMR in the published academic literature. Searches were limited to the English language and full-text only. The search was not limited by date of publication (from inception to search date—November 2015). The search strategy used key words and title/abstract words for both the content area of interest and for the methodology of interest (mixed methods, multimethods, or multiple methods), combined together with the AND Boolean operator (Appendix 1 in the Data Supplement).
Stage 3: Study Selection
Two reviewers (D.J.T.C. and H.T.-T.) independently assessed each abstract that was identified by the initial search. All abstracts flagged as potentially relevant by either reviewer was advanced to full-text review. In the second stage, both reviewers read each full-text article and, based on a set of predefined criteria, determined which articles would be included. Any discrepancies were resolved through the involvement of a third reviewer (K.K.-S.).
The criteria for inclusion in this review were as follows:
reported mixed methods results (defined as at least 1 qualitative and 1 quantitative strand with independently collected data and clear integration);
pertained directly to the management of CAD, hypertension, or nongestational diabetes mellitus;
data were collected from adult respondents (≥18 years of age); and
focused on patient experience or patient-level data (ie, not healthcare system or healthcare provider-relevant data).
The criteria for exclusion included as follows:
solely reported program-related outcomes, such as cost or acceptability of a given program;
not a full-text article (ie, abstracts or summaries); and
did not describe original analyses (ie, review articles or study protocols).
Stage 4: Charting the Data
Both reviewers independently extracted data from each study advanced to full-text review. A review form was developed and used to collect prespecified elements of each study. The reviewers met to discuss data extraction, and where necessary, consensus was reached through discussion.
Extracted information included (1) study and author characteristics, (2) mixed methods features, and (3) qualitative and quantitative methodologies and data collection techniques. We extracted the following study characteristics: year of publication, disease being studied, country of first author’s primary institutional affiliation, type of funding body for the study, first author’s clinical discipline (categorized as physician, nursing, allied health or nonclinician/other)—based on degrees if published, otherwise we searched their institutional website to find information on clinical discipline.
The features of mixed methods studies which we assessed in this review included purpose for using MMR, reconciliation of paradigmatic differences, timing and priority of data collection, and point of data integration.14 Because the explicit focus of this review was study methodology, we did not extract information on study content or results.
There are many classifications for why one might undertake a mixed methods study, the most parsimonious example we found was by Sandelowski,21 modified from a previous taxonomy.22 These reasons include (1) complementarity—the use of distinct methods to provide a more complete understanding of a given phenomenon, (2) triangulation—the use of distinct methods to increase the credibility of the findings, and (3) development—the use of a second method to more fully explore and develop a finding found with the first method.
There has been considerable debate in the mixed methods community about if and how the different research paradigms used in qualitative (constructionism) and quantitative (postpositivism) research can or should be combined.23 We sought to assess whether researchers publishing mixed methods studies in this area described how they reconciled these paradigmatic differences.
Mixed methods studies can be conducted in a variety of ways and are often classified by the timing and priority of the 2 strands. The most common ways of classifying the timing of a mixed methods study are3 (1) concurrent (data for both strands collected simultaneously), (2) sequential (data from one strand informs the collection of data for the other strand—either exploratory [qualitative→quantitative], explanatory [quantitative→qualitative]), or (3) multiphase (>2 strands concurrently or sequentially collected). Some mixed methods studies prioritize one of the strands over the other, whereas others place equal priority on the 2 strands. We sought to describe both the reported timing and priority of included studies.
The data from the 2 strands must be merged or integrated in mixed methods studies. This integration can take place at various points3,24: development or study design—when the results of one strand inform the data collection for the second strand; data analysis—when the analysis of one strand is embedded within the analysis of the other strand; and interpretation—when each strand is analyzed independently and then considered together in the discussion section of a publication. We sought to describe the frequency with which different points of integration were used in the included studies. Because any given study may integrate at multiple points, we recorded the point of integration as the earliest point in a given study.
Finally, we also examined the study methodologies and data collection techniques used in the individual qualitative and quantitative strands of each included study.
Stage 5: Collating, Summarizing, and Reporting the Results
The results are presented in tabular and graphic form demonstrating the frequency and trends in MMR use, the most commonly used types of mixed methods, qualitative, and quantitative techniques in studies meeting our inclusion criteria. This presentation of results allows for comparison of the popularity of different methods in MMR studies in this topical area.
Over 800 abstracts were identified in the initial search (n=811). Review of abstracts led to the retrieval of 253 potentially relevant full-text articles. After full-text review, 84 reports of studies were identified that met all criteria for inclusion in the review (Figure 1). At this stage, the κ score for 2 reviewers’ agreement was 0.61, which is considered substantial agreement.25 Reasons for exclusion at the full-text review stage included: studies not directly related to patients (n=46), program/evaluation-specific outcomes (n=31); studies which were not focused on one of the conditions of interest (n=25); and studies not meeting our definition of MMR—including those which had added a small number of open-ended questions to an otherwise quantitative survey (n=46).
We identified 9 mixed methods studies where at least one of the strands was published as a separate article. In these instances, if the companion article was not found in the initial search, a focused literature search was performed to find the missing companion article(s). These publications (n=13) were added to the total pool of included studies, yielding a final pool of 97 reports of 81 distinct mixed methods studies. Of these 81 MMR studies, 72 were published in a single article (Table 1) and 9 published in 2 to 5 separate articles (Table 2).
Most studies were published recently, with >60% of the studies published since 2011, and only 5 studies were published before 2001 (Figure 2). Diabetes mellitus was the focus of 73% of studies, whereas CAD was the focus of 24% and hypertension was only investigated in 12% of studies included.
The bulk of MMR in this area came from researchers based out of the United States (45%) or United Kingdom (23%). Nurses (42%) and nonclinicians (eg, disciplines such as community health, sociology, anthropology, and public health; 29%) published the most mixed methods studies, whereas physicians (18%) and other allied healthcare providers (eg, occupational therapists, dieticians, and psychologists; 12%) published fewer MMR studies.
MMR seems to be funded predominantly by public sources and foundations, which contributed to 51 and 22 of the 97 included articles, respectively. Authors’ institutions contributed to 16 of the studies, and industry contributed to the fewest (n=7), whereas 20 articles were either unfunded or did not have a clear description of funding sources.
The features of the included mixed methods studies are summarized in Figure 3. The most commonly described rationale for using mixed methods was complementarity (80%; 65/81). Disparities in the philosophical assumptions between different paradigms which underpin qualitative and quantitative traditions were described and reconciled or justified in few studies (7%; 6/81).
The timing of most studies was concurrent (64%; 52/81). Of sequentially conducted studies, explanatory (quantitative study followed by qualitative exploration for depth) was more common (n=16) than exploratory (n=8). A multiphase design including >2 strands was used in 5 studies (6%). The priority of the strands was not explicitly identified in most studies (86%; 70/81).
Integration was most commonly undertaken in the interpretation phase of the study (64%; 52/81), where the results were presented separately and there was some integration that took place in the discussion and conclusion sections of the report. Integration at the study design stage was used in 17 of the sequentially conducted mixed methods studies.
In addition, fewer than half of the studies (48%; 39/81) explicitly described a theory or contextual framework underlying their work.
The predominant qualitative methodology used was basic qualitative description (91%; 74/81). We found few instances of more interpretive qualitative methodologies: ethnography (n=5), phenomenology (n=3), and narrative studies (n=3), with only 1 each of grounded theory and case study methodology. Within the qualitative strands, the most commonly used data collection techniques were individual and group interviews, with participant observation and documentary analysis being used much less frequently (Figure 4).
The most commonly used quantitative study designs were cross-sectional studies (56%; 45/81). There were fewer prospective observational studies (21%; 17/81) and experimental studies (20%; 16/81), with even fewer retrospective studies (5%; 4/81). The quantitative component of the included studies most commonly used surveys or scales for data collection (Figure 4). A minority of studies used objective measurements, including laboratory data, administrative data, and biophysical measures.
The use of MMR in studies of patients with or at risk of CAD is increasing over time. This could be because of multiple reasons, including the publication of articles and editorials promoting the importance of using MMR,18,19,26,27 the growing acceptability of using qualitative methods to reveal information unavailable when using quantitative methods,19 and the perceived propensity of funding agencies to support studies which include both qualitative and quantitative components.28–30 Of note, the use of MMR we report is also likely an underestimate of the true prevalence of this type of research because of the focus of our search parameters and inconsistencies with terminology in this field.
Our study supports previous research indicating that researchers from the United States and United Kingdom were conducting most of the published mixed methods studies.16 Notably, most of the mixed methods work in this area is being conducted by nursing researchers and published in nursing journals, with fewer studies conducted by physician and other allied health researchers. It is possible that this relates to the focus of the nursing profession, where value is placed on understanding patients’ views of the illness experience,31,32 compared with medicine, where evaluation of hard clinical outcomes often remains the priority.33 Another factor that likely contributes to the greatest representation of MMR in nursing journals is that they generally permit greater word count, allowing more space for authors to publish their methods and results from both strands of their research in the same article.
Notably, we found that public sources and nonprofit foundations were by far the most common funders of MMR in this area, whereas industry has not played a prominent role in funding MMR. This is likely because of the fact that the bulk of the research in this area is investigator initiated, whereas industry funders typically sponsor more traditional clinical trials of drug or device efficacy rather than health services or quality improvement research where MMR tends to be more prevalent.
We found that MMR was more commonly used in studies of diabetes mellitus, compared with hypertension and CAD. This may be related to the health behavior changes required to address and treat diabetes mellitus, and the fact that there is a significant experiential component to understanding these phenomena.
Because not all research questions warrant a mixed methods approach, it is important that scholars be explicit about their reason for choosing a mixed methods study design.3,34 Among the studies included in this review, the most commonly cited reason for choosing MMR was complementarity. Others have eschewed the notion of using mixed methods for the purpose of triangulation and advocated for its use in only specific circumstances,9 which may account for its relative unpopularity as a cited rationale for using MMR.
For years, the mixed methods literature was consumed by the controversy arising from the dilemma of trying to bring together the distinct worldviews of qualitative (constructivism) and quantitative (postpositivism) inquiry.35,36 More recently, several scholars have proposed means of reconciling this problem. Two of the most common means of doing so are (1) the pragmatic perspective—thinking that diverse methods can be used in the same study based on the premise of what works37 and (2) the dialectical perspective—that multiple paradigms may be used in an individual study if researchers firmly adhere to these individual traditions but cannot be reconciled into a single paradigm.38 Creswell and Plano Clark3 have stated that those who publish MMR must understand and be able to articulate the stance they are using with respect to the paradigm controversy. Despite this, we found that few of the included studies explicitly stated their perspective on reconciling the paradigms of the different strands, and only 2 studies described a pragmatic perspective, which is thought to be the most commonly adopted position.39 It is possible that some or many of those who did not state their stance were tacitly using a pragmatic approach, but omitted any discussion of this stance from their articles.
With respect to the mixed methods designs used, the majority used concurrent timing, consistent with previous metareviews of MMR in the health sciences.13,16 This may be because of the fact that those who conduct MMR in a sequential fashion may present the findings from each of their strands in separate publications. Although we have included sequentially published studies, in cases where the authors did not explicitly refer to at least 1 of the 2 as mixed methods, multimethod, or multiple methods, the studies would not have been detectable with the search strategy we used.
The integration of the 2 strands of a mixed methods study is a key feature, distinguishing mixed methods from simply using multiple research methodologies.40 Integration can be done in a variety of ways. The most basic of which is integration at the interpretation stage, when the researcher analyzes both strands separately and independently, subsequently comparing and contrasting the results of the 2 strands in the Discussion section. There are several other means of integrating findings from each strand in an iterative fashion, such as transformation (qualitizing quantitative data or quantitizing qualitative data),37 parallel presentation of results, or in a sequential study, 1 strand can inform the development of a subsequent strand.13 The vast majority of mixed methods studies used the more basic strategy of integrating at the interpretation stage whereas sequentially published studies predominantly used integration at the development stage. In future studies, consideration might be given to more proximal integration of qualitative and quantitative strands in CAD-related MMR—such as integration at the analysis stage.
In summarizing the mixed methods literature, it is also important to describe the various qualitative and quantitative techniques used. We found that the methodology of the qualitative strand in the vast majority of the included studies was qualitative description. Although this is a recognized and acceptable study design,41 many experts in qualitative research see qualitative description as a superficial method which lacks the theoretical and substantive depth that is afforded by other methodologies seldom used in the included studies (ie, phenomenology, grounded theory, and ethnography).42 However, others have noted that qualitative description is commonly used in Health Services Research because of its strength in obtaining patient perspectives and insights.43 Nearly all studies included in our review used solely interview methods (individual or group) for data collection. This is likely because of its strength for obtaining patient perspectives. In other disciplines, qualitative researchers may prefer combining interview methods with others, such as document analysis or participant observation, to provide greater contextual depth.42
As this was a scoping review, we did not use a tool to assess for study quality; instead, we described study designs and methods used. It is generally accepted that the most robust quantitative study designs are experimental studies and prospective cohort studies.44 These designs were used in only 41% of studies, whereas the most commonly used design was cross-sectional, which have several well-described limitations.45 Most of the included studies relied exclusively on self-report measures such as scales or surveys, with a minority incorporating objective measures (such as laboratory, administrative, or physiological data) which are less prone to systematic error such as reporting bias.46
This study was limited by inclusion of studies that simultaneously reported both qualitative and quantitative strands or explicitly reported in the abstract that it was an individual strand of a mixed methods study. This most certainly excluded numerous mixed methods studies and programs of research in which the strands were published sequentially without mention of using a mixed methods study design. Finding sequentially published articles that were not reported as part of a mixed methods program was beyond the scope of this study. Furthermore, sequentially conducted research studies for which another strand is not acknowledged may not be truly mixed methods because they likely lack the thorough integration of the 2 strands that is characteristic of a mixed methods study, as opposed to a series of studies using multiple research methods.47 However, we must acknowledge that because of this limitation our review likely underestimates of the full extent of MMR in this area of study. Finally, our search may have also underestimated the use of MMR in this area because of a lack of consistent terminology used to describe MMR, for example, our search may have missed studies described as blended research, integrative research, and triangulation research.48 The term mixed methods has been shown to be by far the most commonly used term, but there is a possibility that we missed some studies which used different terminology.48 The heterogeneity in terminology has been widely accepted as a limitation to the entire mixed methods movement and is part of what has driven the convergence toward the emergence of mixed methods as the preferred term to refer to this type of research. Finally, in attempting to discern the discipline of origin for each study, we attempted to assign this on the basis of the clinical discipline of the first author based on their degrees and affiliations—some nonclinicians may be affiliated with clinical departments, whereas those trained as clinicians may not be currently in practice.
MMR is becoming increasingly popular in health research. Although it is likely that the true use of MMR is underestimated in this scoping review, 81 mixed methods studies were identified that met the strict inclusion criteria for this study, most of which have been published in the past decade. CAD and its risk factors are certainly areas to which mixed methods can contribute to an understanding of patients’ perspectives. This review has shown that the optimization of CAD risk factor management, such as hypertension, may be underinvestigated from an MMR perspective. Also, we noted that the majority of mixed methods studies are published by nursing researchers and are funded publicly. We have shown that many of the more advanced mixed methods techniques (such as early integration and multiphase study designs) are not commonly used in this body of literature. Furthermore, those undertaking mixed methods studies in this area should be more explicit in describing the theoretical underpinnings of their studies and of the perspective used to reconcile diverse research paradigms in the various methodologies used. More interpretive qualitative methodologies (such as grounded theory and ethnography) and more contextual data collection techniques (ie, participant observation) are not commonly used. Similarly, the strongest quantitative methodologies (ie, experimental studies) and data collection techniques (ie, those that collect objective data) could be incorporated into future mixed methods studies on CAD, diabetes mellitus, and hypertension to strengthen the body of evidence in this field.
Dr Campbell conceived of the study. All authors shared in the design and scope of this review. Dr Campbell, H. Tam-Tham, and Dr King-Shier undertook the screening of abstracts and extraction of data from the articles. Dr Campbell wrote the first draft of the article, to which all authors contributed substantially. All authors approved the final version of this article.
Sources of Funding
Dr Campbell is supported by an Alberta Innovates-Health Solutions Clinician Fellowship award; H. Tam-Tham is supported by an Alberta Innovates-Health Solutions Graduate Studentship. Dr Manns is supported by an Alberta Innovates-Health Solutions salary award. Dr Manns is also supported by the Svare Chair in Health Economics. Dr Hemmelgarn is supported by the Roy and Vi Baay Chair in Kidney Research. Dr King-Shier is supported by the Guru Nanak Dev Ji DIL (heart) Chair in Cardiovascular Research. Drs Manns and Hemmelgarn were supported by an alternative funding plan from the Government of Alberta and the University of Calgary. The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the article.
The Data Supplement is available at http://circoutcomes.ahajournals.org/lookup/suppl/doi:10.1161/CIRCOUTCOMES.116.003310/-/DC1.
- Received January 13, 2014.
- Accepted November 17, 2016.
- © 2016 American Heart Association, Inc.
- Johnson B,
- Onwuegbuzie A.
- Greene J.
- Creswell J,
- Plano Clark V.
- Field P,
- Morse J.
- Morgan DL.
- Leech N,
- Onwuegbuzie A.
- Willig C,
- Stainton-Rogers W
- Yardley L,
- Bishop F.
- Rossman G,
- Wilson B.
- Tashakkori A,
- Teddlie C
- Ivankova N,
- Kawamura Y.
- Andrew S,
- Halcomb EJ
- Halcomb EJ,
- Andrew S,
- Brannen J.
- Plano Clark VL.
- Curry LA,
- Nembhard IM,
- Bradley EH.
- Teddlie C,
- Tashakkori A
- Teddlie C,
- Tashakkori A.
- Krumholz HM,
- Bradley EH,
- Curry LA.
- Curry LA,
- Krumholz HM,
- O’Cathain A,
- Plano Clark VL,
- Cherlin E,
- Bradley EH.
- Morse JM.
- Morse JM.
- Giddings L.
- Clarke PN,
- Yaros PS.
- Streubert H,
- Rinaldi Carpenter D.
- Creswell J,
- Klassen A,
- Plano Clark V,
- Clegg Smith K.
- Denzin N.
- Smith J.
- Tashakkori A,
- Teddlie C.
- Greene J,
- Caracelli V.
- Tashakkori A,
- Teddlie C
- Morse J.
- Green J,
- Thorogood N.
- 44.↵Centre for Evidence Based Medicine, University of Oxford. Levels of Evidence. 2009. http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009. Accessed August 15, 2016.
- Mann CJ.
- Tashakkori A,
- Teddlie C
- Morse J.