Full publication of results initially presented in abstracts

Abstract

Abstracts of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of results reported in these abstracts is based on the magnitude or direction of the results, publication bias may result. Publication bias creates problems for those conducting systematic reviews or relying on the published literature for evidence about health and social care. To systematically review reports of studies that have examined the proportion of meeting abstracts and other summaries that are subsequently published in full, the time between meeting presentation and full publication, and factors associated with full publication. We searched MEDLINE, Embase, the Cochrane Library, Science Citation Index, reference lists, and author files. The most recent search was done in February 2016 for this substantial update to our earlier Cochrane Methodology Review (published in 2007). We included reports of methodology research that examined the proportion of biomedical results initially presented as abstracts or in summary form that were subsequently published. Searches for full publications had to be at least two years after meeting presentation. Two review authors extracted data and assessed risk of bias. We calculated the proportion of abstracts published in full using a random‐effects model. Dichotomous variables were analyzed using risk ratio (RR), with multivariable models taking into account various characteristics of the reports. We assessed time to publication using Kaplan‐Meier survival analyses. Combining data from 425 reports (307,028 abstracts) resulted in an overall full publication proportion of 37.3% (95% confidence interval (CI), 35.3% to 39.3%) with varying lengths of follow‐up. This is significantly lower than that found in our 2007 review (44.5%. 95% CI, 43.9% to 45.1%). Using a survival analyses to estimate the proportion of abstracts that would be published in full by 10 years produced proportions of 46.4% for all studies; 68.7% for randomized and controlled trials and 44.9% for other studies. Three hundred and fifty‐three reports were at high risk of bias on one or more items, but only 32 reports were considered at high risk of bias overall. Forty‐five reports (15,783 abstracts) with 'positive' results (defined as any 'significant' result) showed an association with full publication (RR = 1.31; 95% CI 1.23 to 1.40), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; 95% CI 1.07 to 1.28) in 34 reports (8794 abstracts). Results emanating from randomized or controlled trials showed the same pattern for both definitions (RR = 1.21; 95% CI 1.10 to 1.32 (15 reports and 2616 abstracts) and RR = 1.17; 95% CI, 1.04 to 1.32 (13 reports and 2307 abstracts), respectively. Other factors associated with full publication include oral presentation (RR = 1.46; 95% CI 1.40 to 1.52; studied in 143 reports with 115,910 abstracts); acceptance for meeting presentation (RR = 1.65; 95% CI 1.48 to 1.85; 22 reports with 22,319 abstracts); randomized trial design (RR = 1.51; 95% CI 1.36 to 1.67; 47 reports with 28,928 abstracts); and basic research (RR = 0.78; 95% CI 0.74 to 0.82; 92 reports with 97,372 abstracts). Abstracts originating at an academic setting were associated with full publication (RR = 1.60; 95% CI 1.34 to 1.92; 34 reports with 16,913 abstracts), as were those considered to be of higher quality (RR = 1.46; 95% CI 1.23 to 1.73; 12 reports with 3364 abstracts), or having high impact (RR = 1.60; 95% CI 1.41 to 1.82; 11 reports with 6982 abstracts). Sensitivity analyses excluding reports that were abstracts themselves or classified as having a high risk of bias did not change these findings in any important way. In considering the reports of the methodology research that we included in this review, we found that reports published in English or from a native English‐speaking country found significantly higher proportions of studies published in full, but that there was no association with year of report publication. The findings correspond to a proportion of abstracts published in full of 31.9% for all reports, 40.5% for reports in English, 42.9% for reports from native English‐speaking countries, and 52.2% for both these covariates combined. More than half of results from abstracts, and almost a third of randomized trial results initially presented as abstracts fail to be published in full and this problem does not appear to be decreasing over time. Publication bias is present in that 'positive' results were more frequently published than 'not positive' results. Reports of methodology research written in English showed that a higher proportion of abstracts had been published in full, as did those from native English‐speaking countries, suggesting that studies from non‐native English‐speaking countries may be underrepresented in the scientific literature. After the considerable work involved in adding in the more than 300 additional studies found by the February 2016 searches, we chose not to update the search again because additional searches are unlikely to change these overall conclusions in any important way. Full publication of results initially presented in abstracts Key message Two important factors increase the probability that a study described in an abstract will subsequently be published in full, (1) the presence of 'positive' or statistically significant results in the abstract and (2) whether the team examining subsequent full publication were from an English‐speaking country or wrote their report in English. The consequence is that systematic reviews relying on fully published research may provide inaccurate or biased findings because of an over‐reliance on studies with positive results or from English‐speaking countries. Our question We reviewed the evidence about how often studies submitted as abstracts at a scientific meeting are published in full, usually as a journal article. We found 425 relevant reports, involving 307,028 abstracts. Background Investigators prepare and submit abstracts for presentation at scientific meetings. Abstracts selected for presentation are usually collated as conference proceedings, but these are not easily found. Thus, it is important to know whether the work submitted and presented is later published as a journal article, which can easily be identified and contains more study information than the abstract. It is also important to know if the publication of the study depends on the size or direction of results or other factors. If so, systematic reviews relying on the published literature for evidence about health and social care will have incomplete or unbalanced information, leading to inaccurate or biased estimates of the effects of the interventions studied. Study characteristics We included 425 research reports described in 551 articles, which had studied the subsequent full publication of 307,028 abstracts from a variety of biomedical and social sciences. Fifty‐four reports included data from abstracts describing randomized or controlled trials. Of the 425 reports, 376 were published in English, and 49 in other languages. Key results 1. Less than half of all studies, and about two‐thirds of randomized trials, initially presented as summaries or abstracts at meetings, are published as journal articles in the 10 years after presentation. 2. Studies with positive results are more likely to be published. 3. Studies with larger sample sizes are more likely to be published. 4. Studies with abstracts presented orally are more likely to be published than those presented as posters. 5. Studies accepted for presentation at a meeting are more likely to be published than those not accepted. 6. Studies describing basic science are more likely to be published that those describing clinical research. 7. Studies describing randomized trials are more likely to be published than those describing other types of studies. 8. Studies that took place in multiple centers are more likely to be published than those at a single center. 9. Studies classified as ‘high quality’ are more likely to be published than ‘low quality’ studies. 10. Studies with authors from an academic setting are more likely to be published than those with authors from other settings. 11. Studies considered by the report authors to have a high impact are more likely to be published than other studies. 12. Studies with funding source reported are more likely to be published than those not reporting funding. 13. Studies originating in North America or Europe are more likely to be published than those originating elsewhere. 14. Studies from English‐speaking countries are more likely to be published than studies originating elsewhere. Quality of the evidence We have confidence in our findings. We considered five criteria to constitute a risk of bias in the included reports, including methods to identify and match full publications to abstracts, and methods to determine whether a factor was associated with full publication. Overall, 7.5% (32/425) of the reports were scored as having an overall high risk of bias, 83.1% (353/425) had at least one criterion at high risk of bias, and 6.1% (26/425) had all criteria at low risk of bias. Search Date Our search updated our 2007 review and is current to February 2016. After the considerable work involved in including more than 300 additional studies from the February 2016 searches, we chose not to update the search again because additional searches are unlikely to change our overall conclusions in any important way.