Quality assessment tools used in systematic reviews of in vitro studies: A systematic review

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Systematic reviews (SRs) and meta-analyses (MAs) are commonly conducted to evaluate and summarize medical literature. This is especially useful in assessing in vitro studies for consistency. Our study aims to systematically review all available quality assessment (QA) tools employed on in vitro SRs/MAs.

Method

A search on four databases, including PubMed, Scopus, Virtual Health Library and Web of Science, was conducted from 2006 to 2020. The available SRs/MAs of in vitro studies were evaluated. DARE tool was applied to assess the risk of bias of included articles. Our protocol was developed and uploaded to ResearchGate in June 2016.

Results

Our findings reported an increasing trend in publication of in vitro SRs/MAs from 2007 to 2020. Among the 244 included SRs/MAs, 126 articles (51.6%) had conducted the QA procedure. Overall, 51 QA tools were identified; 26 of them (51%) were developed by the authors specifically, whereas 25 (49%) were pre-constructed tools. SRs/MAs in dentistry frequently had their own QA tool developed by the authors, while SRs/MAs in other topics applied various QA tools. Many pre-structured tools in these in vitro SRs/MAs were modified from QA tools of in vivo or clinical trials, therefore, they had various criteria.

Conclusion

Many different QA tools currently exist in the literature; however, none cover all critical aspects of in vitro SRs/MAs. There is a need for a comprehensive guideline to ensure the quality of SR/MA due to their precise nature.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01295-w.

Related collections

Most cited references 69

Record: found
Abstract: found
Article: not found

GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.

Gordon Guyatt, Andrew D Oxman, Elie A. Akl … (2011)

This article is the first of a series providing guidance for use of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system of rating quality of evidence and grading strength of recommendations in systematic reviews, health technology assessments (HTAs), and clinical practice guidelines addressing alternative management options. The GRADE process begins with asking an explicit question, including specification of all important outcomes. After the evidence is collected and summarized, GRADE provides explicit criteria for rating the quality of evidence that include study design, risk of bias, imprecision, inconsistency, indirectness, and magnitude of effect. Recommendations are characterized as strong or weak (alternative terms conditional or discretionary) according to the quality of the supporting evidence and the balance between desirable and undesirable consequences of the alternative management options. GRADE suggests summarizing evidence in succinct, transparent, and informative summary of findings tables that show the quality of evidence and the magnitude of relative and absolute effects for each important outcome and/or as evidence profiles that provide, in addition, detailed information about the reason for the quality of evidence rating. Subsequent articles in this series will address GRADE's approach to formulating questions, assessing quality of evidence, and developing recommendations. Copyright © 2011 Elsevier Inc. All rights reserved.

0 comments Cited 2084 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

Gary G. Wells, Jeremy M. Grimshaw, Lex Bouter … (2009)

Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

0 comments Cited 226 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Reviewing studies with diverse designs: the development and evaluation of a new tool.

Reema Sirriyeh, Rebecca Lawton, Peter Gardner … (2012)

RATIONALE, AIMS & OBJECTIVE: Tools for the assessment of the quality of research studies tend to be specific to a particular research design (e.g. randomized controlled trials, or qualitative interviews). This makes it difficult to assess the quality of a body of research that addresses the same or a similar research question but using different approaches. The aim of this paper is to describe the development and preliminary evaluation of a quality assessment tool that can be applied to a methodologically diverse set of research articles. The 16-item quality assessment tool (QATSDD) was assessed to determine its reliability and validity when used by health services researchers in the disciplines of psychology, sociology and nursing. Qualitative feedback was also gathered from mixed-methods health researchers regarding the comprehension, content, perceived value and usability of the tool. Reference to existing widely used quality assessment tools and experts in systematic review confirmed that the components of the tool represented the construct of 'good research technique' being assessed. Face validity was subsequently established through feedback from a sample of nine health researchers. Inter-rater reliability was established through substantial agreement between three reviewers when applying the tool to a set of three research papers (κ = 71.5%), and good to substantial agreement between their scores at time 1 and after a 6-week interval at time 2 confirmed test-retest reliability. The QATSDD shows good reliability and validity for use in the quality assessment of a diversity of studies, and may be an extremely useful tool for reviewers to standardize and increase the rigour of their assessments in reviews of the published papers which include qualitative and quantitative work. © 2011 Blackwell Publishing Ltd.

0 comments Cited 165 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Nguyen Tien Huy:

ORCID: http://orcid.org/0000-0002-9543-9440

tienhuy@nagasaki-u.ac.jp

Journal

Journal ID (nlm-ta): BMC Med Res Methodol

Journal ID (iso-abbrev): BMC Med Res Methodol

Title: BMC Medical Research Methodology

Publisher: BioMed Central (London )

ISSN (Electronic): 1471-2288

Publication date (Electronic): 8 May 2021

Publication date PMC-release: 8 May 2021

Publication date Collection: 2021

Volume: 21

Electronic Location Identifier: 101

Affiliations

[1 ]GRID grid.444918.4, ISNI 0000 0004 1794 7022, Institute of Fundamental and Applied Sciences, Duy Tan University, ; Ho Chi Minh City, 700000 Vietnam

[2 ]GRID grid.444918.4, ISNI 0000 0004 1794 7022, Faculty of Natural Sciences, , Duy Tan University, ; Da Nang City, 550000 Vietnam

[3 ]Asia Shine Trading & Service CO. LTD., Ho Chi Minh City, Vietnam

[4 ]Online Research Club, Nagasaki, Japan

[5 ]GRID grid.411303.4, ISNI 0000 0001 2155 6022, Faculty of Medicine, , Al-Azhar University, ; Cairo, 11884 Egypt

[6 ]Department of Internal Medicine, Texas Tech University Health Science Center at the Permian Basin, Odessa, TX USA

[7 ]GRID grid.174567.6, ISNI 0000 0000 8902 2273, Department of Immunogenetics, , Institute of Tropical Medicine (NEKKEN), Graduate School of Biomedical Sciences, Nagasaki University, ; 1-12-4 Sakamoto, Nagasaki, 852-8523 Japan

[8 ]GRID grid.174567.6, ISNI 0000 0000 8902 2273, School of Tropical Medicine and Global Health, , Nagasaki University, ; 1–12–4 Sakamoto, Nagasaki, 852-8523 Japan

Author information

Nguyen Tien Huy http://orcid.org/0000-0002-9543-9440

Article

Publisher ID: 1295

DOI: 10.1186/s12874-021-01295-w

PMC ID: 8106836

PubMed ID: 33964880

SO-VID: 54c00835-5b91-4b55-8808-5fa7a8ebe325

License:

Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

History

Date received : 2 June 2020

Date accepted : 26 April 2021

Custom metadata

ScienceOpen disciplines: Medicine

Keywords: quality assessment tool,systematic review,meta-analysis,in vitro study

Data availability:

ScienceOpen disciplines: Medicine

Keywords: quality assessment tool, systematic review, meta-analysis, in vitro study

Quality assessment tools used in systematic reviews of in vitro studies: A systematic review

Read this article at

Abstract

Background

Method

Results

Conclusion

Supplementary Information

Related collections

Karger: Nephrology

Most cited references 69

GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

Reviewing studies with diverse designs: the development and evaluation of a new tool.

Author and article information

Contributors

Journal

Affiliations

Author information

Article

History

Categories

Custom metadata

Comments

Comment on this article

Similar content 161

Cited by 21

Most referenced authors 868