Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer : A Stepped-Wedge Cluster Randomized Clinical Trial

Key Points

Question

What is the effect of delivering machine learning mortality predictions with behavioral nudges to oncology clinicians on the rate of serious illness conversations with patients with cancer?

Findings

In this stepped-wedge cluster randomized clinical trial that included 14 607 patients with cancer, the intervention led to a significant increase in serious illness conversations from approximately 1% to 5% of all patient encounters and from approximately 4% to 15% of encounters with patients having high predicted mortality risk.

Meaning

Machine learning mortality predictions combined with behavioral nudges to clinicians led to an increased rate of serious illness conversations for patients with cancer.

Abstract

Importance

Serious illness conversations (SICs) are structured conversations between clinicians and patients about prognosis, treatment goals, and end-of-life preferences. Interventions that increase the rate of SICs between oncology clinicians and patients may improve goal-concordant care and patient outcomes.

Objective

To determine the effect of a clinician-directed intervention integrating machine learning mortality predictions with behavioral nudges on motivating clinician-patient SICs.

Design, Setting, and Participants

This stepped-wedge cluster randomized clinical trial was conducted across 20 weeks (from June 17 to November 1, 2019) at 9 medical oncology clinics (8 subspecialty oncology and 1 general oncology clinics) within a large academic health system in Pennsylvania. Clinicians at the 2 smallest subspecialty clinics were grouped together, resulting in 8 clinic groups randomly assigned to the 4 intervention wedge periods. Included participants in the intention-to-treat analyses were 78 oncology clinicians who received SIC training and their patients (N = 14 607) who had an outpatient oncology encounter during the study period.

Interventions

(1) Weekly emails to oncology clinicians with SIC performance feedback and peer comparisons; (2) a list of up to 6 high-risk patients (≥10% predicted risk of 180-day mortality) scheduled for the next week, estimated using a validated machine learning algorithm; and (3) opt-out text message prompts to clinicians on the patient’s appointment day to consider an SIC. Clinicians in the control group received usual care consisting of weekly emails with cumulative SIC performance.

Main Outcomes and Measures

Percentage of patient encounters with an SIC in the intervention group vs the usual care (control) group.

Results

The sample consisted of 78 clinicians and 14 607 patients. The mean (SD) age of patients was 61.9 (14.2) years, 53.7% were female, and 70.4% were White. For all encounters, SICs were conducted among 1.3% in the control group and 4.6% in the intervention group, a significant difference (adjusted difference in percentage points, 3.3; 95% CI, 2.3-4.5; P < .001). Among 4124 high-risk patient encounters, SICs were conducted among 3.6% in the control group and 15.2% in the intervention group, a significant difference (adjusted difference in percentage points, 11.6; 95% CI, 8.2-12.5; P < .001).

Conclusions and Relevance

In this stepped-wedge cluster randomized clinical trial, an intervention that delivered machine learning mortality predictions with behavioral nudges to oncology clinicians significantly increased the rate of SICs among all patients and among patients with high mortality risk who were targeted by the intervention. Behavioral nudges combined with machine learning mortality predictions can positively influence clinician behavior and may be applied more broadly to improve care near the end of life.

Trial Registration

ClinicalTrials.gov Identifier: NCT03984773

Abstract

This stepped-wedge cluster randomized clinical trial assesses the effect of a clinician-directed intervention combining machine learning mortality predictions with behavioral nudges vs usual care on motivating serious illness conversations between clinicians and patients with cancer.