Certainty about uncertainty in sleep staging: a theoretical framework

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Sleep stage classification is an important tool for the diagnosis of sleep disorders. Because sleep staging has such a high impact on clinical outcome, it is important that it is done reliably. However, it is known that uncertainty exists in both expert scorers and automated models. On average, the agreement between human scorers is only 82.6%. In this study, we provide a theoretical framework to facilitate discussion and further analyses of uncertainty in sleep staging. To this end, we introduce two variants of uncertainty, known from statistics and the machine learning community: aleatoric and epistemic uncertainty. We discuss what these types of uncertainties are, why the distinction is useful, where they arise from in sleep staging, and provide recommendations on how this framework can improve sleep staging in the future.

Related collections

Most cited references 28

Record: found
Abstract: found
Article: not found

Interrater reliability for sleep scoring according to the Rechtschaffen & Kales and the new AASM standard.

Heidi Danker-Hopfe, Peter Anderer, Josef Zeitlhofer … (2009)

Interrater variability of sleep stage scorings has an essential impact not only on the reading of polysomnographic sleep studies (PSGs) for clinical trials but also on the evaluation of patients' sleep. With the introduction of a new standard for sleep stage scorings (AASM standard) there is a need for studies on interrater reliability (IRR). The SIESTA database resulting from an EU-funded project provides a large number of studies (n = 72; 56 healthy controls and 16 subjects with different sleep disorders, mean age +/- SD: 57.7 +/- 18.7, 34 females) for which scorings according to both standards (AASM and R&K) were done. Differences in IRR were analysed at two levels: (1) based on quantitative sleep parameter by means of intraclass correlations; and (2) based on an epoch-by-epoch comparison by means of Cohen's kappa and Fleiss' kappa. The overall agreement was for the AASM standard 82.0% (Cohen's kappa = 0.76) and for the R&K standard 80.6% (Cohen's kappa = 0.68). Agreements increased from R&K to AASM for all sleep stages, except N2. The results of this study underline that the modification of the scoring rules improve IRR as a result of the integration of occipital, central and frontal leads on the one hand, but decline IRR on the other hand specifically for N2, due to the new rule that cortical arousals with or without concurrent increase in submental electromyogram are critical events for the end of N2.

0 comments Cited 121 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

Eyke Hüllermeier, Willem Waegeman (2021)

The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often referred to as aleatoric and epistemic . In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of attempts so far at handling uncertainty in general and formalizing this distinction in particular.