26
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: not found

      Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements

      Read this article at

      ScienceOpenPublisher
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Psychological science relies on behavioral measures to assess cognitive processing; however, the field has not yet developed a tradition of routinely examining the reliability of these behavioral measures. Reliable measures are essential to draw robust inferences from statistical analyses, and subpar reliability has severe implications for measures’ validity and interpretation. Without examining and reporting the reliability of measurements used in an analysis, it is nearly impossible to ascertain whether results are robust or have arisen largely from measurement error. In this article, we propose that researchers adopt a standard practice of estimating and reporting the reliability of behavioral assessments of cognitive processing. We illustrate the need for this practice using an example from experimental psychopathology, the dot-probe task, although we argue that reporting reliability is relevant across fields (e.g., social cognition and cognitive psychology). We explore several implications of low measurement reliability and the detrimental impact that failure to assess measurement reliability has on interpretability and comparison of results and therefore research quality. We argue that researchers in the field of cognition need to report measurement reliability as routine practice so that more reliable assessment tools can be developed. To provide some guidance on estimating and reporting reliability, we describe the use of bootstrapped split-half estimation and intraclass correlation coefficients to estimate internal consistency and test-retest reliability, respectively. For future researchers to build upon current results, it is imperative that all researchers provide psychometric information sufficient for estimating the accuracy of inferences and informing further development of cognitive-behavioral assessments.

          Related collections

          Most cited references45

          • Record: found
          • Abstract: found
          • Article: not found

          Coefficient alpha and the internal structure of tests

          Psychometrika, 16(3), 297-334
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation.

            Coefficient alpha is the most popular measure of reliability (and certainly of internal consistency reliability) reported in psychological research. This is noteworthy given the numerous deficiencies of coefficient alpha documented in the psychometric literature. This mismatch between theory and practice appears to arise partly because users of psychological scales are unfamiliar with the psychometric literature on coefficient alpha and partly because alternatives to alpha are not widely known. We present a brief review of the psychometric literature on coefficient alpha, followed by a practical alternative in the form of coefficient omega. To facilitate the shift from alpha to omega, we also present a brief guide to the calculation of point and interval estimates of omega using a free, open source software environment. © 2013 The British Psychological Society.
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Attentional bias in emotional disorders.

                Bookmark

                Author and article information

                Journal
                Advances in Methods and Practices in Psychological Science
                Advances in Methods and Practices in Psychological Science
                SAGE Publications
                2515-2459
                2515-2467
                November 06 2019
                : 251524591987969
                Affiliations
                [1 ]Department of Experimental Psychology, University of Oxford
                [2 ]Department of Psychology, Stockholm University
                Article
                10.1177/2515245919879695
                a03db6d9-4eff-4d1a-a9a0-48feb3f1fa2a
                © 2019

                http://journals.sagepub.com/page/policies/text-and-data-mining-license

                History

                Comments

                Comment on this article