How should systematic reviewers handle conference abstracts? A view from the trenches

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

While identifying and cataloging unpublished studies from conference proceedings is generally recognized as a good practice during systematic reviews, controversy remains whether to include study results that are reported in conference abstracts. Existing guidelines provide conflicting recommendations.

Main body

The main argument for including conference abstracts in systematic reviews is that abstracts with positive results are preferentially published, and published sooner, as full-length articles compared with other abstracts. Arguments against including conference abstracts are that (1) searching for abstracts is resource-intensive, (2) abstracts may not contain adequate information, and (3) the information in abstracts may not be dependable. However, studies comparing conference abstracts and fully published articles of the same study find only minor differences, usually with conference abstracts presenting preliminary results. Other studies that have examined differences in treatment estimates of meta-analyses with and without conference abstracts report changes in precision, but usually not in the treatment effect estimate. However, in some cases, including conference abstracts has made a difference in the estimate of the treatment effect, not just its precision. Instead of arbitrarily deciding to include or exclude conference abstracts in systematic reviews, we suggest that systematic reviewers should consider the availability of evidence informing the review. If available evidence is sparse or conflicting, it may be worthwhile to search for conference abstracts. Further, attempts to contact authors of abstracts or search for protocols or trial registers to supplement the information presented in conference abstracts is prudent. If unique information from conference abstracts is included in a meta-analysis, a sensitivity analysis with and without the unique results should be conducted.

Conclusions

Under given circumstances, it is worthwhile to search for and include results from conference abstracts in systematic reviews.

Related collections

Most cited references 30

Record: found
Abstract: found
Article: not found

Publication bias and clinical trials.

K Dickersin, S. Chan, T.C. Chalmersx … (1987)

A study was performed to evaluate the extent to which the medical literature may be misleading as a result of selective publication of randomized clinical trials (RCTs) with results showing a statistically significant treatment effect. Three hundred eighteen authors of published trials were asked whether they had participated in any unpublished RCTs. The 156 respondents reported 271 unpublished and 1041 published trials. Of the 178 completed unpublished RCTs with a trend specified, 26 (14%) favored the new therapy compared to 423 of 767 (55%) published reports (p less than 0.001). For trials that were completed but not published, the major reasons for nonpublication were "negative" results and lack of interest. From the data provided, it appears that nonpublication was primarily a result of failure to write up and submit the trial results rather than rejection of submitted manuscripts. The results of this study imply the existence of a publication bias of importance both to meta-analysis and the interpretation of statistically significant positive trials.

0 comments Cited 129 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

State-of-the-evidence reviews: advantages and challenges of including grey literature.

Karen M Benzies, Shahirose Premji, K Alix Hayden … (2006)

Increasingly, health policy decision-makers and professionals are turning to research-based evidence to support decisions about policy and practice. Systematic reviews are useful for gathering, summarizing, and synthesizing published and unpublished research about clearly defined interventions. State-of-the-evidence reviews are broader than traditional systematic reviews and may include not only published and unpublished research, but also published and unpublished non-research literature. Decisions about whether to include this "grey literature" in a review are challenging and lead to many questions about whether the advantages outweigh the challenges. The primary purpose of this article is to describe what constitutes grey literature, and methods to locate it and assess its quality. The secondary purpose is to discuss the core issues to consider when making decisions to include grey literature in a state-of-the-evidence review. A recent state-of-the-evidence review is used as an exemplar to present advantages and challenges related to including grey literature in a review. Despite the challenges, in the exemplar, inclusion of grey literature was useful to validate the results of a research-based literature search. Decisions about whether to include grey literature in a state-of-the-evidence review are complex. A checklist to assist in decision-making was created as a tool to assist the researcher in determining whether it is advantageous to include grey literature in a review.

0 comments Cited 109 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards.

K Dickersin, Y I Min, C Meinert … (1992)

--To investigate factors associated with the publication of research findings, in particular, the association between "significant" results and publication. --Follow-up study. --Studies approved in 1980 or prior to 1980 by the two institutional review boards that serve The Johns Hopkins Health Institutions--one that serves the School of Medicine and Hospital and the other that serves the School of Hygiene and Public Health. --A total of 737 studies were followed up. --Of the studies for which analyses had been reported as having been performed at the time of interview, 81% from the School of Medicine and Hospital and 66% from the School of Hygiene and Public Health had been published. Publication was not associated with sample size, presence of a comparison group, or type of study (eg, observational study vs clinical trial). External funding and multiple data collection sites were positively associated with publication. There was evidence of publication bias in that for both institutional review boards there was an association between results reported to be significant and publication (adjusted odds ratio, 2.54; 95% confidence interval, 1.63 to 3.94). Contrary to popular opinion, publication bias originates primarily with investigators, not journal editors: only six of the 124 studies not published were reported to have been rejected for publication. --There is a statistically significant association between significant results and publication.

0 comments Cited 104 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Roberta W. Scherer:

ORCID: http://orcid.org/0000-0003-2493-1769

+1(410) 502-4636 , rschere1@jhu.edu

Ian J. Saldanha: ian_saldanha@brown.edu

Journal

Journal ID (nlm-ta): Syst Rev

Journal ID (iso-abbrev): Syst Rev

Title: Systematic Reviews

Publisher: BioMed Central (London )

ISSN (Electronic): 2046-4053

Publication date (Electronic): 7 November 2019

Publication date PMC-release: 7 November 2019

Publication date Collection: 2019

Volume: 8

Electronic Location Identifier: 264

Affiliations

[1 ]ISNI 0000 0001 2171 9311, GRID grid.21107.35, Center for Clinical Trials and Evidence Synthesis, Department of Epidemiology, , Johns Hopkins Bloomberg School of Public Health, ; 615 N. Wolfe Street, Room E6138, Baltimore, MD 21205 USA

[2 ]ISNI 0000 0004 1936 9094, GRID grid.40263.33, Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice (Primary), , Brown University School of Public Health, ; Providence, RI USA

[3 ]ISNI 0000 0004 1936 9094, GRID grid.40263.33, Department of Epidemiology (Joint), , Brown University School of Public Health, ; Providence, RI USA

Author information

Roberta W. Scherer http://orcid.org/0000-0003-2493-1769

Article

Publisher ID: 1188

DOI: 10.1186/s13643-019-1188-0

PMC ID: 6836535

PubMed ID: 31699124

SO-VID: f928f346-369f-4df7-8a20-7d891c1f3f44

License:

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

History

Date received : 3 April 2019

Date accepted : 5 October 2019

Custom metadata

ScienceOpen disciplines: Public health

Data availability:

ScienceOpen disciplines: Public health

Comments

Comment on this article

scite_

Smart Citations

Citing PublicationsSupportingMentioningContrasting

View Citations

See how this article has been cited at scite.ai

scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

How should systematic reviewers handle conference abstracts? A view from the trenches

Read this article at

Abstract

Background

Main body

Conclusions

Related collections

Global Health Next Generation Network

Most cited references 30

Publication bias and clinical trials.

State-of-the-evidence reviews: advantages and challenges of including grey literature.

Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards.

Author and article information

Contributors

Journal

Affiliations

Author information

Article

History

Categories

Custom metadata

Comments

Comment on this article

Similar content 561

Cited by 110

Most referenced authors 733