Advancing Quality Standards in Mixed Methods Research: Extending the Legitimation Typology

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Several scholars have proposed frameworks for assessing the quality of mixed methods research (MMR) studies. However, no general consensus has emerged. The legitimation typology developed by Onwuegbuzie and Johnson (2006) is one promising approach that addresses quantitative, qualitative, and MMR elements. The aim of this intrinsic, exploratory case study is to explore the use of the legitimation typology in empirical MMR studies and through interviews with the developers, MMR scholars, and researchers who applied the legitimation typology to an empirical MMR study. We conducted a systematic methodological review using multiple databases and identified 49 empirical MMR studies that addressed the legitimation typology. Using a critical case sampling approach defined by participants’ unique experiences with the legitimation typology, semi-structured interviews were conducted with five authors of empirical MMR studies, a mixed methods researcher who has written about the typology, and one of the authors of the original legitimation typology to expand on ways the legitimation typology is used in practice. Four overarching themes were identified: (a) comprehensive approach to assessing quality, (b) researchers’ interpretation of legitimation types, (c) value of divergent findings, and (d) strategies for applying the legitimation typology. This case study adds to the MMR literature by clarifying the use of emic-etic and conversion legitimations and by proposing a new legitimation type: divergent findings legitimation. Hence, this study elucidates the application of one quality framework (i.e., legitimation) in MMR and provides recommendations to the field to further advance discussions on quality criteria and their implementation in mixed methods research.

Related collections

Most cited references 34

Record: found
Abstract: not found
Article: not found

How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability

Greg Guest, Arwen Bunce, Laura Johnson (2016)

0 comments Cited 1882 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching

Neal Haddaway, Alexandra Collins, Deborah Coughlin … (2015)

Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable attention as a method for searching for literature, particularly in searches for grey literature, as required by systematic reviews. The reliance on GS as a standalone resource has been greatly debated, however, and its efficacy in grey literature searching has not yet been investigated. Using systematic review case studies from environmental science, we investigated the utility of GS in systematic reviews and in searches for grey literature. Our findings show that GS results contain moderate amounts of grey literature, with the majority found on average at page 80. We also found that, when searched for specifically, the majority of literature identified using Web of Science was also found using GS. However, our findings showed moderate/poor overlap in results when similar search strings were used in Web of Science and GS (10–67%), and that GS missed some important literature in five of six case studies. Furthermore, a general GS search failed to find any grey literature from a case study that involved manual searching of organisations’ websites. If used in systematic reviews for grey literature, we recommend that searches of article titles focus on the first 200 to 300 results. We conclude that whilst Google Scholar can find much grey literature and specific, known studies, it should not be used alone for systematic review searches. Rather, it forms a powerful addition to other traditional search methods. In addition, we advocate the use of tools to transparently document and catalogue GS search results to maintain high levels of transparency and the ability to be updated, critical to systematic reviews.

0 comments Cited 380 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources

Michael Gusenbauer, Neal Haddaway (2020)

Rigorous evidence identification is essential for systematic reviews and meta‐analyses (evidence syntheses) because the sample selection of relevant studies determines a review's outcome, validity, and explanatory power. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments. This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed, and Web of Science. A novel, query‐based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall, and reproducibility. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analyzed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.