There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.
Abstract
KEY POINTS
Cyberattacks can incur privacy breaches and financial harm, as well as directly threaten
patient safety and health system functioning.
Reducing the risk of cyberattacks and managing those that do occur happens in 4 stages:
prevention, detection, response and recovery.
As novel areas of cyberthreats emerge (e.g., Internet-connected devices), clinicians
and health organizations should be vigilant for recalls, keep software up to date
and discuss possible risks with patients.
Keeping workflows efficient and maintaining a strong cybersecurity posture has trade-offs;
however, the minor inconveniences of security measures such as 2-factor authentication
are far preferable to recovering from an attack.
Canadian health systems have digitized considerably. In 2019, 86% of surveyed Canadian
family physicians reported using electronic medical records (EMRs).1 Digital tools
for virtual care and remote patient monitoring, wearables, care coordination platforms,
and Internet-of-things (IoT) devices are all permeating practice.2 The digitization
and integration of disparate health information systems on shared networks promises
greater convenience, access and quality of care, but also introduces risk for patients,
providers and health systems. Although some clinicians have dedicated information
technology (IT) training, most do not, and navigating increasingly complex health
information systems can create considerable stress.
Cyberattacks can incur privacy breaches and financial harm, as well as compromise
patient safety and health system functioning. Personal health information (PHI) can
fetch much higher prices on the dark Web than other personal information (e.g., credit
card details).3 In a 2021 international survey of health IT decision-makers, the average
cost of a ransomware attack was US$1.27 million. 4 Cyberattacks against health information
systems have been associated with delays in care, diversion of patients to other sites
and increased mortality.5 Cyberattacks against Canadian health information systems
are increasingly common, with 48% of all reported 2019 Canadian breaches occurring
in the health sector.6 Cyberattacks have also been increasing amid events such as
the COVID-19 pandemic and Russo–Ukrainian War.7,8 We outline the impact of cyberattacks
on Canadian health information systems and how clinicians, whether they practise in
large hospitals or individual clinics, can improve their cybersecurity posture.
How have cyberattacks affected Canadian health information systems?
Cyberattacks against health information systems are most commonly ransomware or data
breaches (Figure 1). At least 14 major cyberattacks on Canadian health information
systems have occurred since 2015, 9 of which attempted ransom and 6 of which compromised
PHI. Ransomware involves the installation and activation of a malicious program (i.e.,
malware) that locks or encrypts a computer system and its stored data until a financial
ransom is paid. Access to data is commonly lost even when ransoms are paid.4 The attack
can also entail data breaches, whereby PHI is exfiltrated off health information systems
and shared illicitly in online marketplaces. Another form of extortion relies on denial
of service, whereby an attacker overwhelms a site through fake traffic to make it
unavailable for authentic users (e.g., patients attempting to book an appointment)
until a payment is made.9 Although most cyberattacks against health organizations
are attributed to criminals, they can also be perpetrated by nation-states, terrorist
groups, online “hacktivists” and ideologically motivated violent extremists (e.g.,
those targeting abortion centres).10–12
Figure 1:
Recent cyberattacks on Canadian health information systems, including denial of service
(red), ransomware (green), data breach (blue), mixed (orange) and unknown (purple).
Note: IT = information technology, PHI = personal health information.
Health organizations, irrespective of their size, make attractive cyberattack targets.
First, they are financially lucrative targets because of the value of PHI but are
also likely well-resourced enough to pay ransoms. Since attackers adjust ransom amounts
to the perceived ability of the target to pay, attackers can hold health information
systems in individual physician offices for ransom in the Can$3000–Can$5000 range
and still expect a reasonable likelihood of payment.13 Canadian hospitals have not
been reported to pay ransoms, but American health systems have paid ransoms well into
the millions of dollars.14 Even if no money is paid, the extortion attempt can still
incur extended periods of downtime of the health information system with substantial
(and very public) impacts to IT and patient services. Second, the extensive media
coverage of cyberattacks on health systems increases the pressure on victims to pay
the ransom quickly before it becomes public. Third, health organizations often have
a history of underinvesting in IT systems and rely on outdated or legacy systems that
are vulnerable to exploitation. Finally, health organizations can also lack the capacity
to respond to cyberthreats, which increases the damage of hacks as well as the probability
of paying ransoms.
What can Canada learn from the cybersecurity practices of peer countries?
Comprehensive comparison of the burden of attacks between jurisdictions is difficult
since many cyberattacks on health information systems are unreported.15 Although effective
cyberhygiene (i.e., daily routines, good behaviours and occasional check-ups akin
to principles in health) strategies for end-users are essentially universal across
organizations, sectors and jurisdictions, cybersecurity policy in Canadian health
information systems has considerable room for improvement.
In June 2022, the House of Commons enacted the Critical Cyber Systems Protection Act
(CCSPA). The CCSPA defines critical cyber systems as those with serious implications
for public safety if compromised. These systems include telecommunications, pipelines,
nuclear energy, federally regulated transportation and banking — but not health organizations.16
In contrast, the United States Cybersecurity and Infrastructure Security Agency supports
a range of Sector Coordinating Councils that collaborate with the government for information
sharing, coordination and the establishment of voluntary practices to promote resilience.
The Healthcare and Public Health Sector Coordinating Council has dozens of members,
including health systems, advocacy groups, insurers and nonprofit organizations.17
Although the exclusion of health organizations from the CCSPA could be viewed as consistent
with the federal–provincial principles of the Canada Health Act, governance mechanisms
such as Sector Coordinating Councils could promote adherence to common standards while
also fostering innovation and experimentation.
Within the provinces and territories, considerable heterogeneity exists in cybersecurity
posture among broader public sector organizations, as smaller institutions often lack
requisite financial and human resources. Shared services models can help address disparities.
For example, Ontario Health is piloting 6 regional security operation centres.18 Each
centre would continuously monitor the security practices of member institutions, defend
against breaches and proactively isolate and mitigate security risks. Regional security
operation centres are similar to the well-received, health-related computer emergency
response teams in the United Kingdom, Norway and the Netherlands.10 As governments
establish these bodies, clinicians and health organizations must develop familiarity
with them and their incident reporting and escalation pathways. During establishment
of these bodies, governments should also endeavour to engage clinicians to ensure
their needs and perspectives are considered. Provinces and territories should be wary
of regulating cybersecurity practices beyond reporting at the level of the individual
provider or health organization (e.g., mandating biannual cybersecurity audits) as
top–down requirements can be overly onerous in terms of effort, capital and human
resources, especially for smaller practices. Finally, provinces and territories should
establish publicly available repositories of cyberattacks on health information systems.15
Such repositories can serve as a useful aid for research and guide consumer choice
as patients may preferentially seek out providers with strong cybersecurity track
records.
How can clinicians prevent and navigate cyberattacks?
The US National Institute of Standards and Technology outlines 5 stages to effectively
navigating cyberattacks: identification, protection, detection, response and recovery.19
For simplicity, we have combined the stages of identification and protection into
a single prevention stage (Figure 2).
Figure 2:
Four stages of cyber resilience, with suggested actions. Note: CMPA = Canadian Medical
Protective Association, EMR = electronic medical record, VPN = virtual private network.
Prevention
At the individual level, cyberhygiene prevents attacks. Clinicians should be vigilant
for phishing attacks via email or other suspicious behaviour (Figure 3). Phishing
refers to targeted, deceptive efforts to gain access to a victim’s device or network.
Once access has been obtained, an attacker can install malware to exfiltrate or encrypt
data for ransom. Clinicians should ensure they use unique, strong passwords (i.e.,
at least 8 characters with a mix of letters, numbers and special characters) and 2-factor
authentication for their logins, as well as set up verification questions and auto-lock
devices with access to PHI. Password managers can generate and store unique, strong
passwords for each site and provide notifications when user information is compromised.
Clinicians should avoid sensitive tasks without adequate network protections (e.g.,
accessing patient records on public Wi-Fi) as data can be intercepted or malware can
be installed in “man-in-the-middle” attacks. Software must be kept up to date as developers
release patches for security vulnerabilities on an ongoing basis. Health organizations
are notorious for relying on legacy systems (e.g., Windows XP) well past the date
of their security support deprecation.
Figure 3:
Anatomy of a hypothetical phishing attack.
At the institution or practice level, a key aspect of preventing cyberattacks is to
reduce the attack surface, or the number of entry points an intruder would have into
health information systems. This is especially important with setups in which individuals
can use their personal devices and with increasing numbers of IoT devices.20 Techniques
for reducing attack surfaces include auditing all devices on the network, ensuring
that their software (including operating systems) are up to date, installing antivirus
and antimalware software, and setting up a firewall to monitor both outbound and inbound
Internet traffic. Practices can also set up a virtual private network (VPN), which
encrypts and disguises online traffic, making it much more difficult to intercept.
Virtual private networks are particularly important for clinicians who wish to access
PHI from environments outside their health organization’s network, such as to complete
charting at home. Although clinicians in larger organizational settings will have
the benefits of a standardized approach, those in private practice will have to rely
on third-party vendors. Luckily, many traditional antivirus vendors now have comprehensive
bundles of services. Professional support from organizations such as the Ontario Medical
Association exists, including privacy breach and cyber coverage to assist with forensics,
public relations and legal services. These should be viewed as essential office expenses
and, in many jurisdictions, may be eligible for tax credits.
Detection
Suspicious behaviour can indicate a cyberattack. Examples include barred entry to
files or applications (e.g., EMRs, email clients), the deletion or installation of
unrecognized files and software, program auto-running and emails sent without the
user’s consent. Ransomware attacks are often accompanied by pop-up messages that indicate
to the user that they are being hacked and that provide instructions and a deadline
for ransom payment. Antivirus or antimalware software can also detect threats on routine
scans. Finally, users within the organization may report that they followed a link
in a phishing email or downloaded unknown files or applications.
Response
Once a cyberattack is detected, clinicians should first disconnect affected machines
from the Internet and shut them down. Quick action can prevent the exfiltration of
data, including PHI, from a health organization’s device and network. Once this is
done, practices should activate their cyberattack response plan. If access to computerized
systems such as EMRs is lost, staff should transition to back-up workflows such as
using paper records. Depending on the magnitude of workflow disruptions and the ability
of clinicians to maintain an adequate standard of care, contingency measures such
as cancelling clinics and transferring patients may be needed. Crucially, response
plans should not be improvised but rather be well documented, clear and deliberately
practised.21 Clinicians should practise their cyberattack response (i.e., their code
grey) like they would a fire (i.e., their code red). Although the pressure to do so
may be immense, health organizations should generally not pay ransoms to unlock and
decrypt systems, because restored access is not guaranteed and paying ransoms may
encourage future attacks.
The Canadian Medical Protective Association (CMPA) outlines the duty of custodians
to notify affected individuals of privacy breaches (e.g., patients), as well as the
provincial or territorial privacy commissioner and ministry of health.22 As the nuances
of expectations vary across jurisdictions, the CMPA recommends organizations and clinicians
initiate contact with the CMPA as soon as possible after a possible breach is discovered.
They should also contact law enforcement, especially in the event of a ransomware
attack. The Royal Canadian Mounted Police is currently pilot-testing a National Cybercrime
and Fraud Reporting System.23 The Canadian Centre for Cyber Security also has a reporting
system; however, it does not trigger an immediate response by law enforcement.24 As
part of their cyber response plan, practices should consult relevant authorities in
advance to ensure they clearly understand the obligations for breach reporting and
notification of law enforcement for their jurisdiction.
Recovery
After the acute threat of a cyberattack has subsided, clinicians and their organization
can then enter the recovery phase. Recovery is heavily dependent on having health
information systems that allow for restoration from back-ups. For smaller organizations
and independent practices without dedicated IT experts, clinicians should ask how
their vendors will protect their data and help recover it in case of an attack as
part of their due diligence when making a purchase. Organizations should also have
a focused debrief on the response, with emphasis on opportunities for improvement
and measures to improve ongoing cybersecurity posture.
Clinicians may feel that adhering to the outlined actions only adds to the burden
imposed on them by health information systems. In his famous The New Yorker essay,
Atul Gawande quipped that the EMR systems “that promised to increase my mastery over
my work [have], instead, increased my work’s mastery over me.”25 Especially for clinicians
in smaller practices, cybersecurity can become another dimension of task load, in
addition to documentation, computerized order entry and maintenance of licensing requirements
through mandatory e-modules, all of which contribute to burnout.26,27 Simulation training
has also become commonplace in medicine and some may ask if more are necessary. Measures
such as 2-factor authentication and VPNs add complexity to workflows; however, small
changes to daily practices that promote cyberhygiene are far preferable to recovering
from a cyberattack operationally, both financially and in terms of patient and community
trust.
What are emerging areas of cybersecurity in health care?
Emerging technologies require attention to ensure that the risk of compromise does
not grow with improvements in utility. Clinicians who are adopting a virtual care
platform should note that consumer video-conferencing solutions (e.g., Zoom, Face-Time)
often do not meet provincial privacy and security requirements. Instead, clinicians
should use tools built into their EMR or versions of videoconferencing solutions that
specifically meet health care standards such as Zoom for Healthcare.28 Provincial
health authorities provide lists of verified solutions for virtual care.29 Personal
medical devices — such as pacemakers, insulin pumps and blood glucose monitors — are
connected to the Internet for remote biomarker monitoring, as well as to receive software
updates. Hackers have shown the ability to rapidly drain device batteries, provide
too much stimulus (e.g., pacing, insulin bolus) or fail to provide a stimulus when
clinically indicated. 30 In 2019, Health Canada recalled several models of insulin
pumps that were susceptible to attack and encouraged patients to discuss switching
to other models with their physicians. 31 Finally, machine learning tools are actively
being developed and integrated into health care workflows.32 These tools can be vulnerable
to adversarial attacks or subtle changes to input data that are carefully designed
to mislead algorithms toward incorrect outputs.33 For example, a hacker can add very
small amounts of noise to pixels in a radiograph that would be imperceptible to humans
but change model outputs (e.g., from benign to pathologic or vice versa). Across these
novel areas, clinicians and health organizations should be vigilant for recalls, keep
software up to date and discuss possible risks with patients.
Conclusion
Preventing cyberattacks involves navigating trade-offs between keeping workflows efficient
and reducing risk amid threats that are growing in frequency, severity and sophistication.
As national and regional policies develop, health organizations, practices and individual
clinicians must take a proactive approach to improving their cybersecurity posture.
Methods for handling personal and professional risk go hand-in-hand, including leveraging
tools and best practices, being vigilant and having an incident response plan. With
respect to cybersecurity, a bit of prevention is worth a terabyte of cure.
KEY POINTS Machine learning has the potential to transform health care, although its current application to routine clinical practice has been limited. Multidisciplinary partnership between technical experts and end-users, including clinicians, administrators, and patients and their families, is essential to developing and implementing machine-learned solutions in health care. A 3-phase framework can be used to describe the development and adoption of machine-learned solutions: an exploration phase to understand the problem being addressed and the deployment environment, a solution design phase for the development of machine-learned models and user-friendly tools, and an implementation and evaluation phase to deploy and assess the impact of the machine-learned solution. Machine learning — the process of developing systems that learn from data to recognize patterns and make accurate predictions of future events1 — has considerable potential to transform health care. Machine-learned tools could support complex clinical decision-making and could automate many of the mundane tasks that may waste clinician time and lead to work dissatisfaction. 2 Despite growing interest in and regulatory approval of such technologies, for example smartwatch algorithms to detect atrial fibrillation,3 to date machine-learned tools have had only limited use in routine clinical practice.4 Developing and implementing machine-learned tools in medicine requires infrastructure and resources that can be difficult to access, such as large, real-time clinical data sets, technical skills in data science, computing power and clinical informatics infrastructure. Other barriers to adoption include challenges in ensuring data security and privacy, poorly performing mathematical models, difficulty integrating tools into existing workflows, low acceptance of machine-learned solutions by clinician users, and uncertainty about how to evaluate them.4 In this article we outline an approach to developing and adopting machine-learned solutions in health care. Related articles discuss some of the caveats of using this technology5 and the evaluation of machine-learned tools.6 Developing machine-learned solutions for clinical use requires a strong understanding of clinical care, data science and implementation science. A number of excellent frameworks support data analytics and quality-improvement initiatives, including the Cross-Industry Standard Process for Data Mining (CRISP-DM),7 the Model for Improvement developed by the Institute for Healthcare Improvement 8 and the Knowledge to Action9 framework. However, there is no clear, comprehensive framework specifically focused on adoption of machine-learned tools in health care. We propose a 3-phase framework to develop and implement machine-learned solutions in clinical care, illustrated by a case example (Box 1). The framework comprises an exploration phase, a solution design phase, and an implementation and evaluation phase (Figure 1). It can be used for a range of solutions, including computer vision–based projects, automation and optimization projects, and predictive analytics. The framework can also be applied when organizations are implementing machine-learned solutions that were developed elsewhere because the steps, other than model development, remain the same. Box 1: Case example A failure to recognize clinical deterioration in hospital is a leading cause of unplanned patient transfer to an intensive care unit (ICU).10 Early warning systems11,12 can predict a patient’s risk of clinical deterioration, and potentially allow clinicians to intervene earlier. Many existing early warning systems are based on traditional statistical approaches, such as logistic regression models that use a simple combination of a small number of inputs (most commonly, fewer than 10 parameters, such as vital signs), and they are prone to false-positive predictions.13 More advanced biostatistical models may identify at-risk patients with greater accuracy.13 However, implementation and evaluation of more advanced biostatistical or machine-learned models is uncommon. The General Internal Medicine (GIM) inpatient service at St. Michael’s Hospital, an academic health centre in Toronto, Ontario, cares for about 4000 patients each year. Roughly 7% of patients in the GIM service die or are transferred to an ICU.14 The hospital has a well-established critical care response team, staffed by a respiratory therapist, ICU nurse and ICU physician, which can be called by ward teams to urgently assess inpatients who may require transfer to the ICU. Beginning in 2017, the hospital developed a machine-learned early warning system for the GIM service. The aim was to predict and prevent clinical deterioration to reduce mortality. Implementation and evaluation of the intervention, which was rolled out iteratively in 2020, is under way. Figure 1: A framework for the development and adoption of machine-learned solutions in clinical practice. What are the key steps of the exploration phase? The development of successful machine-learned solutions requires a deep understanding of the problem at hand, relevant outcomes, the data that are available now and that will be available in the future, end-user needs, workflow, human factors and change management. For solutions designed to provide clinical decision support, implementation is strengthened by understanding in advance how the machine-learned solution will be paired with an evidence-based clinical intervention to improve care. Identify the problem and build a team The first step is to identify a problem that is important to end-users, such as clinicians or administrators, and to identify the specific, measurable outcomes they wish to change by modifying current practice. Machine-learned solutions may be geared toward replacing human effort (i.e., “do what I do”), in which case the outcomes may be time saved and measures of task performance. Alternatively, machine-learned solutions may be designed to address a clinical problem, in which case the outcome may be a measurable clinical improvement. Problems are usually first identified by end-users and then should be explored by a multidisciplinary team to determine whether a machine-learned solution might be appropriate. The team should include end-users who understand the clinical or operational problem and workflow; data engineers and information technology (IT) professionals who understand the available data and infrastructure and how a solution could be implemented; data scientists who understand how machine-learned models can be developed; and patients and caregivers when proposed solutions are patient-facing. Because developing and implementing machine-learned solutions is resource intensive, great care should be taken in selecting priority projects. First, the problem should be important, which could be determined by estimating how solving the problem would improve patient health, improve patient care experience, improve provider care experience, or reduce costs. Second, a machine-learned solution must be feasible, which is determined by whether the right quantity and quality of data are available with the right timeliness, whether the problem has a reasonable chance of being modelled successfully, and whether a potential solution can be implemented within existing IT infrastructure and clinical workflow. Finally, there must be a reasonable chance of improvement associated with the interventions that will accompany the solution. Ideally, the proposed interventions are evidence based and already known to be effective. Ultimately, end-user engagement is the key to success. End-users will adopt a machine-learned solution only if it fits into their workflow and is perceived to be useful. Understand the problem and set goals End-users may have identified a problem that they experience regularly, but they may not understand why the problem exists or how it could be solved. The multidisciplinary team should work to understand the problem and create a theory of change, which describes their best hypothesis of how a machine-learned solution will lead to improvement. Systematic approaches to understanding clinical and operational problems have been well described, including process mapping, cause-and-effect analysis, and failure modes and effects analysis. 15 This understanding of the problem will inform the development, implementation and evaluation of the solution. As with any improvement project, the team should set clear and measurable improvement goals by defining the relevant outcomes, describing the baseline state of performance, and setting a specific target for improvement. Unique to machine-learned solutions, the team should also set performance benchmarks to define the level of model performance that would be clinically actionable and useful. It may be helpful to answer the question, “What is the current level of performance of decision-makers and by how much should it be improved for a machine-learned solution to be worthwhile?” A highly accurate model that is no better than clinical judgment will be less useful than a modestly accurate model that is substantially better than clinical judgment. In the case example presented in Box 1, an exploration team (Figure 2) was established to consider various clinical events that could be predicted (e.g., sepsis, acute kidney injury, readmission) to improve care for patients in the General Internal Medicine (GIM) service. Based on available data and literature review, this team created a short list of options and then consulted with the full GIM Division, hospital administrators, and 3 of the hospital’s patient and family advisers before selecting clinical deterioration (i.e., death or ICU transfer) as the top priority. Data and IT experts determined that the project would be feasible. Literature review, discussions with GIM staff physicians and nurses, and a brief chart review of 10 randomly sampled16 cases of clinical deterioration helped the team better understand the problem. The proposed theory of change was that a machine-learned early warning system might improve care by enabling earlier detection of severe illness, allowing clinicians to intervene earlier, engage in proactive conversations regarding patient preferences and goals of care, and improve the timeliness of consultation by ICU teams or palliative care teams. The team set an aim to reduce mortality in patients admitted to the GIM ward by 10% in 1 year, which was considered achievable, given other studies of early warning systems.17 Figure 2: Team structure for each phase of development of an early warning system in the General Internal Medicine (GIM) service at St. Michael’s Hospital, Toronto, Ontario. Note: ICU = intensive care unit. How should machine-learned solutions be designed? Developing a machine-learned solution involves developing and testing a machine-learned model, and then testing its initial implementation. We suggest using a framework for algorithm development and testing, such as CRISP-DM.7 A key advantage of this approach is that it acknowledges the iterative nature of data science, which often requires cycling through 6 phases: understanding the use case, understanding the data, preparing the data, modelling, evaluating model performance, and deployment. The approach to model development is driven by several considerations, such as the problem that is being addressed; the quantity, quality and type of available data; and implementation considerations such as workflow and end-user acceptance. Developing a machine-learned solution often requires 3 complementary work streams, which could be led by 1 or more teams: model development, clinical implementation and evaluation (Figure 2). These workstreams are interrelated, as decisions made for one aspect affect the others. Focused teams can be developed for each workstream, so each receives sufficient attention and expertise, with overlapping membership to ensure coordination. Check the quality of the data Many problems encountered when deploying a machine-learned solution can be traced back to the data used to develop the model. The quality of input data can be assessed for completeness, correctness, concordance, plausibility and currency18 through relatively simple, automated approaches and targeted manual validation.19 Beyond these basic data-quality metrics, it is also important to understand the outcome data that models are trained on, and whether they truly reflect the intended prediction targets. A related article discusses problems related to model training data.5 Design the model with implementation in mind Data scientists have many options for developing effective models, including traditional regression techniques such as logistic regression and more modern machine-learning techniques that accommodate complex interrelationships of variables, such as neural networks.1 Although data scientists will select a modelling approach based on the nature of the desired output and the input features,20 the entire machine-learned solution should be designed by an interdisciplinary team with its implementation in mind.21 In the case example (Box 1), the solution involved a prediction model, a communication system to convey patient risk to clinicians, and a clinical care pathway for high-risk patients. All aspects of the solution were designed iteratively by the 3 teams (Figure 2), with periodic input from patient and family advisers. The teams decided that the prediction model should aim for no more than 2 false alarms for every true positive alarm in order to balance the time required to assess high-risk patients with other competing demands. Thus, the data scientists set the threshold for categorizing patients as high risk at a positive predictive value of 30%, based on historical data. At this threshold, the sensitivity was 50%, which clinicians considered would be a useful proportion of cases to detect. Clinicians felt that it would be most useful to predict outcomes that were likely to occur within 24–48 hours. A much shorter window would not leave enough time to intervene, and a longer window would make it difficult for clinicians to know how to respond. Thus, the data scientists trained models to predict events in the subsequent 48 hours. Develop a user-friendly tool For systems designed to provide decision support, models should be incorporated into user-friendly tools that provide key pieces of useful, action-oriented information and integrate into end-user workflow. This involves collaboration between end-users and experts in process improvement, human factors, design, and change management. Engagement with end-users is critical throughout this process, although the extent of engagement will vary depending on the issue being addressed. In the case example, based on human factors principles,22 a simple 3-level approach was selected to present actionable information to clinicians, with patients stratified into high-, medium- and low-risk groups. Clinicians receive updated patient risk predictions through the hospital’s electronic signout tool and through text paging alerts. Paging alerts are sent only when patients change from lower risk levels to the highest risk level, and if a patient remains at high risk, there are no repeat alerts, thereby minimizing alarm fatigue.23 As a result, there are typically between 0 and 2 alerts per GIM team (who usually care for 15–20 patients) per 24-hour period. Design a clinical intervention to integrate with workflow Introducing a new clinical tool, machine-learned or otherwise, may alter existing workflows.24 Such changes may be planned and welcome,25 or they may be disruptive and harmful.26 Various strategies, including interviews, focus groups, surveys and workflow analysis, may be employed to describe existing workflows and assess barriers and facilitators to implementation of a new tool.24,27 These can then be mapped to effective strategies to optimize implementation using approaches such as the Capability, Opportunity, Motivation, Behaviour (COM-B) model.28 In the case example, the implementation team included clinicians and administrators with first-hand experience of the existing workflows in GIM, ICU, palliative care and clinical informatics. Additional interviews and focus groups were conducted to inform the implementation team as needed. The team considered existing resources, such as hospital protocols for escalation of care and the critical care response team when designing the intervention. The methods and timing of alerts were designed to fit within existing processes for physicians and nurses in the GIM service, ICU and palliative care. For example, model predictions are reported to charge nurses at specific times, and in a specific format, so that patient risk can be factored into nursing assignments. A clinical pathway was designed with concrete actions and time targets for physicians and nurses to respond to high-risk patients while leaving room for clinical judgment (Figure 3). Figure 3: Clinical care pathway for patients in the General Internal Medicine (GIM) service with high predicted risk of clinical deterioration. Engage end-users to establish trust One common barrier to the adoption of machine-learned technology is whether clinicians trust the model’s output.29 One framework suggests trust can be built by demonstrating transparency, fairness and robustness of models.30 In the case example, the team used historical data from 2011 to 2020 to develop and validate the early warning system model. Multivariate adaptive regression spline models were developed using about 100 inputs related to patient demographics, vital signs and laboratory test results; this model was chosen after experimentation with numerous modelling techniques using more than 500 input variables.31 The large number of inputs and the complex ways they can interact make it difficult to explain the factors influencing any given prediction, although some machine-learned models may be more interpretable than others (i.e., it may be possible to report the relative importance of different predictors). It may be desirable for machine-learned models to be interpretable for some clinical applications,32 but interpretability is not essential for establishing trust33 and there is no consensus on the best methods to explain more complex models.34,35 Providing detailed explanations for model predictions could even hinder clinical decision-making in some situations through information overload or creating false impressions of causality. To establish trust in the GIM early warning system, we transparently reported to the front-line clinicians how we developed and validated these models, showing that models were not biased across patient age and sex (although there were limited sociodemographic data to explore other dimensions of fairness). We showed model robustness by validating the machine-learned models on historical cohorts using temporal split-sample validation, meaning that models trained on data from 2011 to 2019 were tested on data from 2020. We also compared model predictions to predictions made in real time by physicians and nurses about their patients over a 4-month period, to provide clinical validation of the model’s potential usefulness. To encourage engagement of end-users, the initiative was championed by well-regarded senior clinical leaders, including nursing leadership and the physician heads of the GIM, ICU and Palliative Care divisions. Engaging patients, family members and caregivers is important, particularly when developing patient-facing solutions. Engaging patients can improve the design, safety and satisfaction associated with new services.36,37 Methods for this engagement have been well described38,39 and should include clearly articulating the purpose of engagement, accommodating unique needs to make participation accessible, recruiting diverse partners, and embracing the opportunity for exchange between those with expert knowledge and those with lived experience. In the case example, patients and caregivers were recruited primarily from the hospital’s patient and family advisory group and were consulted at various stages of the project. We chose a consultative model of engagement in order to solicit feedback on key issues, including selecting clinical deterioration as a priority, designing the clinical intervention and addressing issues related to implementation. For example, a major topic of discussion was how patients and their families should be informed about the model’s predictions. These discussions led the clinical implementation team to conclude that the patient’s physicians should be responsible for discussing the model’s predictions when clinically appropriate and situating these in the broader context of the patient’s health and treatment plan. How should machine-learned solutions be implemented and evaluated? Phased implementation Widespread adoption of machine-learned solutions in health care immediately after their development is not advised. Instead, the machine-learned solution should be deployed in a “silent testing” period before formal implementation (i.e., without end-users being aware of the model predictions or recommendations). The length of this period should be determined by several factors, including the frequency of events being predicted, the nature of the specific clinical practice being targeted, and the number and heterogeneity of intended end-users. This time is used to ensure that data and IT infrastructure function well and to ensure that model performance in the real-world setting is sufficient for deployment. Once successfully completed, the results of the silent trial can be reported to end-users to strengthen trust. If unsuccessful, this testing phase can prevent a potentially harmful model from being deployed, or highlight the need for refinement before deployment. In the case example, the model was silently tested in real time without communicating predictions to clinicians for 9 months. We identified and corrected several issues; for example, we corrected a computing error where the algorithm recognized “Na” (the chemical symbol for sodium) as “NA” (denoting missing values), which affected model performance. Iterative evaluation Given the complexity of both model development and the health care environment, we suggest applying an iterative approach using frameworks that incorporate the Plan-Do-Study-Act (PDSA) cycle,40,41 described by the Model for Improvement developed by the Institute for Healthcare Improvement.8 This involves “planning” the solution deployment, its aims and key measures of effectiveness and safety; “doing” the implementation on a small scale; “studying” the implementation process and impact on the stated measures; and “acting” to refine the implementation process based on the study cycle. Evaluating the implementation of machine-learned models is an iterative process — described in more detail in a related article6 — that often requires several PDSA cycles before the solution is integrated effectively into routine workflow. After the silent test, we launched the early warning system in the case example in a phased roll-out with 2 GIM clinical teams in August 2020, expanded to all 5 GIM clinical teams in September, and then expanded to nurses and the palliative care team in October. The phased approach allowed us to monitor and correct any unanticipated problems that might have occurred related to the machine-learned model, IT environment or clinical workflow. During implementation, the 3 project teams that led the exploration and solution design phases were collapsed into a single implementation team (Figure 2) that met weekly to review process measures and outcome measures and iteratively refine the intervention, improve adherence to the clinical pathway and address unintended consequences. We corrected issues such as erroneous alert messages, revising the alert criteria and changing the education and training processes for physicians and nurses. Methods for evaluation Although randomized controlled trial (RCT) designs are ideal for studying the impact of interventions, non-RCT designs such as interrupted time series methods may also be suitable. In the case example, the option of conducting an RCT was explored, but the sample size required (more than 30 000 participants would be needed to detect a 10% relative mortality reduction, given baseline mortality of about 6%) was prohibitive. A pragmatic and mixed-methods approach is being adopted instead, which includes a qualitative evaluation to identify barriers to implementation and to study the effects of the machine-learned solution on clinical practice through in-depth interviews with nurses, residents and staff physicians. Time series methods and a matched cohort design will be used to compare outcomes for patients in the intervention period to historical controls. These two approaches may help address patient-level and secular confounding, but the confounding effects of the COVID-19 pandemic will remain an important limitation. Multisite trials networks dedicated to evaluating new machine-learned technologies are needed to enable rigorous evaluation. Conclusion The notion that machine learning can rapidly and radically transform health care by automating mundane tasks and enhancing clinical decision-making is glamorous. Unfortunately, the reality of machine learning in health care is sobering, with many instances of poor implementations of machine-learned tools.5 Finding machine-learned solutions that work requires careful engagement with the “messiness” of health care data and the complexity of clinical decisions and workflows. Machine learning does hold tremendous potential to meaningfully advance health care. A disciplined, inclusive, engaged and iterative approach to the development and adoption of these technologies is needed to truly benefit the patients we serve.
Title:
CMAJ : Canadian Medical Association Journal
Publisher:
CMA Impact Inc.
ISSN
(Print):
0820-3946
ISSN
(Electronic):
1488-2329
Publication date
(Print):
20
November
2023
Publication date
(Electronic):
20
November
2023
Volume: 195
Issue: 45
Pages: E1548-E1554
Affiliations
Temerty Faculty of Medicine (Harish), University of Toronto; Institute of Health Policy,
Management, and Evaluation (Harish), Dalla Lana School of Public Health, University
of Toronto; Department of Emergency Medicine (Ackery, Mehta), St. Michael’s Hospital,
Unity Health Toronto, Toronto, Ont.; Department of Emergency Medicine (Grant), Faculty
of Medicine, University of British Columbia, Vancouver, BC; Department of General
Internal Medicine (Jamieson), St. Michael’s Hospital, Unity Health Toronto; Institute
for Health System Solutions and Virtual Care (Jamieson), Women’s College Hospital;
Department of Emergency Medicine (Mehta), North York General Hospital, Toronto, Ont.
This is an Open Access article distributed in accordance with the terms of the Creative
Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and
reproduction in any medium, provided that the original publication is properly cited,
the use is noncommercial (i.e., research or educational use), and no modifications
or adaptations are made. See:
https://creativecommons.org/licenses/by-nc-nd/4.0/
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.