Reproducible Research Practices and Transparency across the Biomedical Literature

Abstract

There is a growing movement to encourage reproducibility and transparency practices in the scientific community, including public access to raw data and protocols, the conduct of replication studies, systematic integration of evidence in systematic reviews, and the documentation of funding and potential conflicts of interest. In this survey, we assessed the current status of reproducibility and transparency addressing these indicators in a random sample of 441 biomedical journal articles published in 2000–2014. Only one study provided a full protocol and none made all raw data directly available. Replication studies were rare ( n = 4), and only 16 studies had their data included in a subsequent systematic review or meta-analysis. The majority of studies did not mention anything about funding or conflicts of interest. The percentage of articles with no statement of conflict decreased substantially between 2000 and 2014 (94.4% in 2000 to 34.6% in 2014); the percentage of articles reporting statements of conflicts (0% in 2000, 15.4% in 2014) or no conflicts (5.6% in 2000, 50.0% in 2014) increased. Articles published in journals in the clinical medicine category versus other fields were almost twice as likely to not include any information on funding and to have private funding. This study provides baseline data to compare future progress in improving these indicators in the scientific literature.

Abstract

Examination of recent trends in reproducibility and transparency practices in biomedical research reveals an ongoing lack of access to full datasets and detailed protocols for both clinical and non-clinical studies.

Author Summary

There is increasing interest in the scientific community about whether published research is transparent and reproducible. Lack of replication and non-transparency decreases the value of research. Several biomedical journals have started to encourage or require authors to submit detailed protocols, full datasets, and disclose information on funding and potential conflicts of interest. In this study, we investigate the reproducibility and transparency practices across the full spectrum of published biomedical literature from 2000–2014. We identify an ongoing lack of access to full datasets and detailed protocols for both clinical and non-clinical biomedical investigation. We also map the availability of information on funding and conflicts of interest in this literature. The results from this study provide baseline data to compare future progress in improving these indicators in the scientific literature. We believe that this information may be essential to sensitize stakeholders in science about the need for improving reproducibility and transparency practices.