20
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Quantifying Bias in Randomized Controlled Trials in Child Health: A Meta-Epidemiological Study

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Objective

          To quantify bias related to specific methodological characteristics in child-relevant randomized controlled trials (RCTs).

          Design

          Meta-epidemiological study.

          Data Sources

          We identified systematic reviews containing a meta-analysis with 10–40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews.

          Data Extraction

          Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias.

          Results

          We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors.

          Conclusions

          We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the number of trials included, in particular the small number of studies with low risk of bias. Until further evidence is available, reviewers should not exclude RCTs from systematic reviews and meta-analyses based solely on risk of bias particularly in the area of child health.

          Related collections

          Most cited references21

          • Record: found
          • Abstract: found
          • Article: not found

          Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.

          To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials. Retrospective review of publications and follow up survey of authors. Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000. Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance. 519 trials with 553 publications and 10,557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable--for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance. Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Generation of allocation sequences in randomised trials: chance, not choice.

            The randomised controlled trial sets the gold standard of clinical research. However, randomisation persists as perhaps the least-understood aspect of a trial. Moreover, anything short of proper randomisation courts selection and confounding biases. Researchers should spurn all systematic, non-random methods of allocation. Trial participants should be assigned to comparison groups based on a random process. Simple (unrestricted) randomisation, analogous to repeated fair coin-tossing, is the most basic of sequence generation approaches. Furthermore, no other approach, irrespective of its complexity and sophistication, surpasses simple randomisation for prevention of bias. Investigators should, therefore, use this method more often than they do, and readers should expect and accept disparities in group sizes. Several other complicated restricted randomisation procedures limit the likelihood of undesirable sample size imbalances in the intervention groups. The most frequently used restricted sequence generation procedure is blocked randomisation. If this method is used, investigators should randomly vary the block sizes and use larger block sizes, particularly in an unblinded trial. Other restricted procedures, such as urn randomisation, combine beneficial attributes of simple and restricted randomisation by preserving most of the unpredictability while achieving some balance. The effectiveness of stratified randomisation depends on use of a restricted randomisation approach to balance the allocation sequences for each stratum. Generation of a proper randomisation sequence takes little time and effort but affords big rewards in scientific accuracy and credibility. Investigators should devote appropriate resources to the generation of properly randomised trials and reporting their methods clearly.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials.

              Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials. Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions. We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant. Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56-79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect. Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.
                Bookmark

                Author and article information

                Contributors
                Role: Editor
                Journal
                PLoS One
                PLoS ONE
                plos
                plosone
                PLoS ONE
                Public Library of Science (San Francisco, USA )
                1932-6203
                2014
                4 February 2014
                : 9
                : 2
                : e88008
                Affiliations
                [1 ]Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
                [2 ]Clinical Pharmacology Unit, Faculty of Medicine, Instituto de Medicina Molecular, University of Lisbon, Lisbon, Portugal
                [3 ]Department of Pediatrics, Santa Maria Hospital, Lisbon, Portugal
                Canadian Agency for Drugs and Technologies in Health, Canada
                Author notes

                Competing Interests: Co-author Lisa Hartling is a PLOS ONE Editorial Board member. This does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.

                Conceived and designed the experiments: LH MPH RMF DMD BV. Performed the experiments: LH MPH. Analyzed the data: LH BV. Wrote the paper: LH. Edited manuscript: MPH RMF DMD BV.

                Article
                PONE-D-13-41505
                10.1371/journal.pone.0088008
                3913714
                24505351
                c774dee0-217a-461a-9bb8-0597131eaf97
                Copyright @ 2014

                This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 10 October 2013
                : 2 January 2014
                Page count
                Pages: 6
                Funding
                This study was funded through the Canadian Institutes of Health Research (CIHR). Dr. Hartling holds a New Investigator Salary Award through CIHR. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
                Categories
                Research Article
                Medicine
                Clinical Research Design
                Clinical Trials
                Meta-Analyses
                Systematic Reviews
                Pediatrics
                Public Health
                Child Health
                Science Policy
                Research Assessment
                Publication Practices
                Research Reporting Guidelines
                Research Integrity
                Publication Ethics

                Uncategorized
                Uncategorized

                Comments

                Comment on this article

                scite_
                39
                5
                45
                0
                Smart Citations
                39
                5
                45
                0
                Citing PublicationsSupportingMentioningContrasting
                View Citations

                See how this article has been cited at scite.ai

                scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

                Similar content226

                Cited by8

                Most referenced authors676