The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview

Abstract

Aziz Sheikh and colleagues report the findings of their systematic overview that assessed the impact of eHealth solutions on the quality and safety of health care.

Abstract

Background

There is considerable international interest in exploiting the potential of digital solutions to enhance the quality and safety of health care. Implementations of transformative eHealth technologies are underway globally, often at very considerable cost. In order to assess the impact of eHealth solutions on the quality and safety of health care, and to inform policy decisions on eHealth deployments, we undertook a systematic review of systematic reviews assessing the effectiveness and consequences of various eHealth technologies on the quality and safety of care.

Methods and Findings

We developed novel search strategies, conceptual maps of health care quality, safety, and eHealth interventions, and then systematically identified, scrutinised, and synthesised the systematic review literature. Major biomedical databases were searched to identify systematic reviews published between 1997 and 2010. Related theoretical, methodological, and technical material was also reviewed. We identified 53 systematic reviews that focused on assessing the impact of eHealth interventions on the quality and/or safety of health care and 55 supplementary systematic reviews providing relevant supportive information. This systematic review literature was found to be generally of substandard quality with regards to methodology, reporting, and utility. We thematically categorised eHealth technologies into three main areas: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. We found that despite support from policymakers, there was relatively little empirical evidence to substantiate many of the claims made in relation to these technologies. Whether the success of those relatively few solutions identified to improve quality and safety would continue if these were deployed beyond the contexts in which they were originally developed, has yet to be established. Importantly, best practice guidelines in effective development and deployment strategies are lacking.

Conclusions

There is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies. In addition, there is a lack of robust research on the risks of implementing these technologies and their cost-effectiveness has yet to be demonstrated, despite being frequently promoted by policymakers and “techno-enthusiasts” as if this was a given. In the light of the paucity of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, it is vital that future eHealth technologies are evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle. Such evaluation should be characterised by careful attention to socio-technical factors to maximise the likelihood of successful implementation and adoption.

Please see later in the article for the Editors' Summary

Editors' Summary

Background

There is considerable international interest in exploiting the potential of digital health care solutions, often referred to as eHealth—the use of information and communication technologies—to enhance the quality and safety of health care. Often accompanied by large costs, any large-scale expenditure on eHealth—such as electronic health records, picture archiving and communication systems, ePrescribing, associated computerized provider order entry systems, and computerized decision support systems—has tended to be justified on the grounds that these are efficient and cost-effective means for improving health care. In 2005, the World Health Assembly passed an eHealth resolution (WHA 58.28) that acknowledged, “eHealth is the cost-effective and secure use of information and communications technologies in support of health and health-related fields, including health-care services, health surveillance, health literature, and health education, knowledge and research,” and urged member states to develop and implement eHealth technologies. Since then, implementing eHealth technologies has become a main priority for many countries. For example, England has invested at least £12.8 billion in a National Programme for Information Technology for the National Health Service, and the Obama administration in the United States has committed to a US$38 billion eHealth investment in health care.

Why Was This Study Done?

Despite the wide endorsement of and support for eHealth, the scientific basis of its benefits—which are repeatedly made and often uncritically accepted—remains to be firmly established. A robust evidence-based perspective on the advantages on eHealth could help to suggest priority areas that have the greatest potential for benefit to patients and also to inform international eHealth deliberations on costs. Therefore, in order to better inform the international community, the authors systematically reviewed the published systematic review literature on eHealth technologies and evaluated the impact of these technologies on the quality and safety of health care delivery.

What Did the Researchers Do and Find?

The researchers divided eHealth technologies into three main categories: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. Then, implementing methods based on those developed by the Cochrane Collaboration and the NHS Service Delivery and Organisation Programme, the researchers used detailed search strategies and maps of health care quality, safety, and eHealth interventions to identify relevant systematic reviews (and related theoretical, methodological, and technical material) published between 1997 and 2010. Using these techniques, the researchers retrieved a total of 46,349 references from which they identified 108 reviews. The 53 reviews that the researchers finally selected (and critically reviewed) provided the main evidence base for assessing the impact of eHealth technologies in the three categories selected.

In their systematic review of systematic reviews, the researchers included electronic health records and picture archiving communications systems in their evaluation of category 1, computerized provider (or physician) order entry and e-prescribing in category 2, and all clinical information systems that, when used in the context of eHealth technologies, integrate clinical and demographic patient information to support clinician decision making in category 3.

The researchers found that many of the clinical claims made about the most commonly used eHealth technologies were not substantiated by empirical evidence. The evidence base in support of eHealth technologies was weak and inconsistent and importantly, there was insubstantial evidence to support the cost-effectiveness of these technologies. For example, the researchers only found limited evidence that some of the many presumed benefits could be realized; importantly, they also found some evidence that introducing these new technologies may on occasions also generate new risks such as prescribers becoming over-reliant on clinical decision support for e-prescribing, or overestimate its functionality, resulting in decreased practitioner performance.

What Do These Findings Mean?

The researchers found that despite the wide support for eHealth technologies and the frequently made claims by policy makers when constructing business cases to raise funds for large-scale eHealth projects, there is as yet relatively little empirical evidence to substantiate many of the claims made about eHealth technologies. In addition, even for the eHealth technology tools that have proven to be successful, there is little evidence to show that such tools would continue to be successful beyond the contexts in which they were originally developed. Therefore, in light of the lack of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, the authors say that future eHealth technologies should be evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle, and include socio-technical factors to maximize the likelihood of successful implementation and adoption in a given context. Furthermore, it is equally important that eHealth projects that have already been commissioned are subject to rigorous, multidisciplinary, and independent evaluation.

Additional Information

Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000387.

• The authors' broader study is: Car J, Black A, Anandan C, Cresswell K, Pagliari C, McKinstry B, et al. (2008) The Impact of eHealth on the Quality and Safety of Healthcare. Available at: http://www.haps.bham.ac.uk/publichealth/cfhep/001.shtml

• More information is available on the World Health Assembly eHealth resolution

• The World Health Organization provides information at the Global Observatory on eHealth, as well as a global insight into eHealth developments

• The European Commission provides Information on eHealth in Europe and some examples of good eHealth practice

• More information is provided on NHS Connecting for Health