8
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Reliability of COVID-19 data: An evaluation and reflection

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Importance

          The rapid proliferation of COVID-19 has left governments scrambling, and several data aggregators are now assisting in the reporting of county cases and deaths. The different variables affecting reporting (e.g., time delays in reporting) necessitates a well-documented reliability study examining the data methods and discussion of possible causes of differences between aggregators.

          Objective

          To statistically evaluate the reliability of COVID-19 data across aggregators using case fatality rate (CFR) estimates and reliability statistics.

          Design, setting, and participants

          Cases and deaths were collected daily by volunteers via state and local health departments, as primary sources and newspaper reports, as secondary sources. In an effort to begin comparison for reliability statistical analysis, BroadStreet collected data from other COVID-19 aggregator sources, including USAFacts, Johns Hopkins University, New York Times, The COVID Tracking Project.

          Main outcomes and measures

          COVID-19 cases and death counts at the county and state levels.

          Results

          Lower levels of inter-rater agreement were observed across aggregators associated with the number of deaths, which manifested itself in state level Bayesian estimates of COVID-19 fatality rates.

          Conclusions and relevance

          A national, publicly available data set is needed for current and future disease outbreaks and improved reliability in reporting.

          Related collections

          Most cited references27

          • Record: found
          • Abstract: found
          • Article: found
          Is Open Access

          Interrater reliability: the kappa statistic

          The kappa statistic is frequently used to test interrater reliability. The importance of rater reliability lies in the fact that it represents the extent to which the data collected in the study are correct representations of the variables measured. Measurement of the extent to which data collectors (raters) assign the same score to the same variable is called interrater reliability. While there have been a variety of methods to measure interrater reliability, traditionally it was measured as percent agreement, calculated as the number of agreement scores divided by the total number of scores. In 1960, Jacob Cohen critiqued use of percent agreement due to its inability to account for chance agreement. He introduced the Cohen’s kappa, developed to account for the possibility that raters actually guess on at least some variables due to uncertainty. Like most correlation statistics, the kappa can range from −1 to +1. While the kappa is one of the most commonly used statistics to test interrater reliability, it has limitations. Judgments about what level of kappa should be acceptable for health research are questioned. Cohen’s suggested interpretation may be too lenient for health related studies because it implies that a score as low as 0.41 might be acceptable. Kappa and percent agreement are compared, and levels for both kappa and percent agreement that should be demanded in healthcare studies are suggested.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV2)

            Estimation of the prevalence and contagiousness of undocumented novel coronavirus (SARS-CoV2) infections is critical for understanding the overall prevalence and pandemic potential of this disease. Here we use observations of reported infection within China, in conjunction with mobility data, a networked dynamic metapopulation model and Bayesian inference, to infer critical epidemiological characteristics associated with SARS-CoV2, including the fraction of undocumented infections and their contagiousness. We estimate 86% of all infections were undocumented (95% CI: [82%–90%]) prior to 23 January 2020 travel restrictions. Per person, the transmission rate of undocumented infections was 55% of documented infections ([46%–62%]), yet, due to their greater numbers, undocumented infections were the infection source for 79% of documented cases. These findings explain the rapid geographic spread of SARS-CoV2 and indicate containment of this virus will be particularly challenging.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: found

              Response to COVID-19 in Taiwan: Big Data Analytics, New Technology, and Proactive Testing

                Bookmark

                Author and article information

                Contributors
                Role: Data curationRole: InvestigationRole: Writing – original draftRole: Writing – review & editing
                Role: Data curationRole: InvestigationRole: Writing – original draftRole: Writing – review & editing
                Role: Data curationRole: Formal analysisRole: MethodologyRole: Writing – original draftRole: Writing – review & editing
                Role: Formal analysisRole: MethodologyRole: Writing – review & editing
                Role: Data curationRole: Writing – original draftRole: Writing – review & editing
                Role: Data curationRole: Writing – review & editing
                Role: Data curationRole: Writing – review & editing
                Role: Data curationRole: Writing – original draft
                Role: Writing – original draft
                Role: ConceptualizationRole: Formal analysisRole: MethodologyRole: Writing – original draftRole: Writing – review & editing
                Role: Data curationRole: Writing – original draftRole: Writing – review & editing
                Role: Funding acquisitionRole: Writing – review & editing
                Role: ConceptualizationRole: Data curationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: Project administrationRole: ResourcesRole: SupervisionRole: ValidationRole: VisualizationRole: Writing – original draftRole: Writing – review & editing
                Role: Editor
                Journal
                PLoS One
                PLoS One
                plos
                PLOS ONE
                Public Library of Science (San Francisco, CA USA )
                1932-6203
                3 November 2022
                2022
                3 November 2022
                : 17
                : 11
                : e0251470
                Affiliations
                [1 ] Department of Public Health, Simmons University, Boston, Massachusetts, United States of America
                [2 ] Department of Data Science and Neuroscience, Simmons University, Boston, Massachusetts, United States of America
                [3 ] Duke Global Health Institute, Duke University, Durham, North Carolina, United States of America
                [4 ] Department of Mathematics & Statistics, Boston University, Boston, Massachusetts, United States of America
                [5 ] Department of Physical Sciences, University of California San Diego, La Jolla, California, United States of America
                [6 ] Global School of Public Health, New York University, New York City, New York, United States of America
                [7 ] Division of Environmental Health Sciences, University of Minnesota, Minneapolis, Minnesota, United States of America
                [8 ] Department of Biology, Tougaloo College, Tougaloo College, Tougaloo, Mississippi, United States of America
                [9 ] Pediatric Ophthalmology and Adult Strabismus, Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States of America
                [10 ] Department of Management Science and Statistics, The University of Texas at San Antonio, San Antonio, Texas, United States of America
                [11 ] BroadStreet Health, Milwaukee, Wisconsin, United States of America
                New Mexico State University, UNITED STATES
                Author notes

                Competing Interests: The specific roles of these authors are articulated in the ‘author contributions’ section. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

                ‡ TAS worked as lead author to this work.

                ¶ All authors were members of (or participated in) the COVID-19 Data Project [ 31]

                Author information
                https://orcid.org/0000-0003-2612-993X
                Article
                PONE-D-21-19745
                10.1371/journal.pone.0251470
                9632841
                36327273
                83efa4a3-6211-48f4-a5fa-a39786eaa8ad
                © 2022 Miller et al

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 16 June 2021
                : 10 December 2021
                Page count
                Figures: 5, Tables: 4, Pages: 19
                Funding
                Funded by: BroadStreet Health
                Award Recipient :
                Funded by: BroadStreet Health
                Award Recipient :
                Funded by: BroadStreet Health
                Award Recipient :
                Funders for this study include only BroadStreet Health, which provided support in the form of salaries for authors EY, RWF, and ZJS, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
                Categories
                Research Article
                Medicine and Health Sciences
                Medical Conditions
                Infectious Diseases
                Viral Diseases
                Covid 19
                Physical Sciences
                Mathematics
                Statistics
                Statistical Data
                Medicine and Health Sciences
                Public and Occupational Health
                Medicine and Health Sciences
                Diagnostic Medicine
                Virus Testing
                Biology and Life Sciences
                Population Biology
                Population Metrics
                Death Rates
                Engineering and Technology
                Systems Engineering
                Quality Assurance
                Medicine and Health Sciences
                Epidemiology
                Pandemics
                People and places
                Geographical locations
                North America
                United States
                Custom metadata
                All the data underlying the findings of the study are available at the provided URL ( https://github.com/BroadStreet-Health/COVID-19-Cases-and-Mortalities).
                COVID-19

                Uncategorized
                Uncategorized

                Comments

                Comment on this article