Repeated Clinical Assessment Using Sensory Modality Assessment and Rehabilitation Technique for Diagnosis in Prolonged Disorders of Consciousness

The kappa statistic is frequently used to test interrater reliability. The importance of rater reliability lies in the fact that it represents the extent to which the data collected in the study are correct representations of the variables measured. Measurement of the extent to which data collectors (raters) assign the same score to the same variable is called interrater reliability. While there have been a variety of methods to measure interrater reliability, traditionally it was measured as percent agreement, calculated as the number of agreement scores divided by the total number of scores. In 1960, Jacob Cohen critiqued use of percent agreement due to its inability to account for chance agreement. He introduced the Cohen’s kappa, developed to account for the possibility that raters actually guess on at least some variables due to uncertainty. Like most correlation statistics, the kappa can range from −1 to +1. While the kappa is one of the most commonly used statistics to test interrater reliability, it has limitations. Judgments about what level of kappa should be acceptable for health research are questioned. Cohen’s suggested interpretation may be too lenient for health related studies because it implies that a score as low as 0.41 might be acceptable. Kappa and percent agreement are compared, and levels for both kappa and percent agreement that should be demanded in healthcare studies are suggested.

The differential diagnosis of disorders of consciousness is challenging. The rate of misdiagnosis is approximately 40%, and new methods are required to complement bedside testing, particularly if the patient's capacity to show behavioral signs of awareness is diminished. At two major referral centers in Cambridge, United Kingdom, and Liege, Belgium, we performed a study involving 54 patients with disorders of consciousness. We used functional magnetic resonance imaging (MRI) to assess each patient's ability to generate willful, neuroanatomically specific, blood-oxygenation-level-dependent responses during two established mental-imagery tasks. A technique was then developed to determine whether such tasks could be used to communicate yes-or-no answers to simple questions. Of the 54 patients enrolled in the study, 5 were able to willfully modulate their brain activity. In three of these patients, additional bedside testing revealed some sign of awareness, but in the other two patients, no voluntary behavior could be detected by means of clinical assessment. One patient was able to use our technique to answer yes or no to questions during functional MRI; however, it remained impossible to establish any form of communication at the bedside. These results show that a small proportion of patients in a vegetative or minimally conscious state have brain activation reflecting some awareness and cognition. Careful clinical examination will result in reclassification of the state of consciousness in some of these patients. This technique may be useful in establishing basic communication with patients who appear to be unresponsive. 2010 Massachusetts Medical Society