For Publishers
DiscoveryMetadataPeer reviewHostingPublishing
For Researchers
JoinPublishReviewCollect
Register
Blog
About
Search
Advanced search
My ScienceOpen
Search
For Publishers
For Researchers
Blog
About
72
views
67
references
 Top references
cited by
44
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
0 collections
    0
    shares
      183
      similar
       All similar
      • Record: found
      • Abstract: found
      • Article: not found

      The frequency of excess success for articles in Psychological Science.

      Author(s): 1
      Publication date (Electronic):
      Journal: Psychonomic bulletin & review
      Publisher: Springer Nature

      Read this article at

      ScienceOpenPublisherPubMed
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Recent controversies have questioned the quality of scientific practice in the field of psychology, but these concerns are often based on anecdotes and seemingly isolated cases. To gain a broader perspective, this article applies an objective test for excess success to a large set of articles published in the journal Psychological Science between 2009 and 2012. When empirical studies succeed at a rate much higher than is appropriate for the estimated effects and sample sizes, readers should suspect that unsuccessful findings have been suppressed, the experiments or analyses were improper, or the theory does not properly account for the data. In total, problems appeared for 82 % (36 out of 44) of the articles in Psychological Science that had four or more experiments and could be analyzed.

          Related collections

          Most cited references67

          • Record: found
          • Abstract: found
          • Article: not found

          An Agenda for Purely Confirmatory Research.

          Eric-Jan Wagenmakers, Ruud Wetzels, Denny Borsboom … (2012)
          The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology's academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result-a procedure that invalidates the interpretation of the common statistical tests. The extent of the fine tuning varies widely across experiments and experimenters but is almost impossible for reviewers and readers to gauge. To remedy the situation, we propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label "confirmatory," and only for these analyses are the common statistical tests valid. Other analyses can be carried out but these should be labeled "exploratory." We illustrate our proposal with a confirmatory replication attempt of a study on extrasensory perception.
          Article 0 comments Cited 315 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            The Rules of the Game Called Psychological Science.

            Marjan Bakker, Annette van Dijk, Jelte Wicherts (2012)
            If science were a game, a dominant rule would probably be to collect results that are statistically significant. Several reviews of the psychological literature have shown that around 96% of papers involving the use of null hypothesis significance testing report significant outcomes for their main results but that the typical studies are insufficiently powerful for such a track record. We explain this paradox by showing that the use of several small underpowered samples often represents a more efficient research strategy (in terms of finding p < .05) than does the use of one larger (more powerful) sample. Publication bias and the most efficient strategy lead to inflated effects and high rates of false positives, especially when researchers also resorted to questionable research practices, such as adding participants after intermediate testing. We provide simulations that highlight the severity of such biases in meta-analyses. We consider 13 meta-analyses covering 281 primary studies in various fields of psychology and find indications of biases and/or an excess of significant results in seven. These results highlight the need for sufficiently powerful replications and changes in journal policies.
            Article 0 comments Cited 254 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              “Positive” Results Increase Down the Hierarchy of the Sciences

              Daniele Fanelli (2010)
              The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences in-between is nearly 200 years old. This order is intuitive and reflected in many features of academic life, but whether it reflects the “hardness” of scientific research—i.e., the extent to which research questions and results are determined by data and theories as opposed to non-cognitive factors—is controversial. This study analysed 2434 papers published in all disciplines and that declared to have tested a hypothesis. It was determined how many papers reported a “positive” (full or partial) or “negative” support for the tested hypothesis. If the hierarchy hypothesis is correct, then researchers in “softer” sciences should have fewer constraints to their conscious and unconscious biases, and therefore report more positive outcomes. Results confirmed the predictions at all levels considered: discipline, domain and methodology broadly defined. Controlling for observed differences between pure and applied disciplines, and between papers testing one or several hypotheses, the odds of reporting a positive result were around 5 times higher among papers in the disciplines of Psychology and Psychiatry and Economics and Business compared to Space Science, 2.3 times higher in the domain of social sciences compared to the physical sciences, and 3.4 times higher in studies applying behavioural and social methodologies on people compared to physical and chemical studies on non-biological material. In all comparisons, biological studies had intermediate values. These results suggest that the nature of hypotheses tested and the logical and methodological rigour employed to test them vary systematically across disciplines and fields, depending on the complexity of the subject matter and possibly other factors (e.g., a field's level of historical and/or intellectual development). On the other hand, these results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.
              Article 0 comments Cited 219 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Journal
                Journal ID (iso-abbrev): Psychon Bull Rev
                Title: Psychonomic bulletin & review
                Publisher: Springer Nature
                ISSN (Electronic): 1531-5320
                ISSN (Print): 1069-9384
                Publication date (Electronic): Oct 2014
                Volume: 21
                Issue: 5
                Affiliations
                [1 ] Department of Psychological Sciences, Purdue University, 703 Third Street, West Lafayette, IN, 47906, USA, gfrancis@purdue.edu.
                Article
                DOI: 10.3758/s13423-014-0601-x
                PubMed ID: 24638826
                SO-VID: e7cf0cae-0b17-40d4-b885-fe00a92acd77
                History

                Data availability:

                Comments

                Comment on this article

                scite_

                Similar content183

                See all similar

                Cited by43

                See all cited by

                Most referenced authors442

                See all reference authors