Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy

Abstract

Objectives

Emergency departments (EDs) are an important point of contact for people with opioid use disorder (OUD). Universal screening for OUD is costly and often infeasible. Evidence on effective, selective screening is needed. We assessed the feasibility of using a risk factor-based machine learning model to identify OUD quickly among patients presenting in EDs.

Design/settings/participants

In this cohort study, all ED visits between January 2016 and March 2018 for patients aged 12 years and older were identified from electronic health records (EHRs) data from a large university health system. First, logistic regression modelling was used to describe and elucidate the associations between patient demographic and clinical characteristics and diagnosis of OUD. Second, a Gradient Boosting Classifier was applied to develop a predictive model to identify patients at risk of OUD. The predictive performance of the Gradient Boosting algorithm was assessed using F1 scores and area under the curve (AUC).

Outcome

The primary outcome was the diagnosis of OUD.

Results

Among 345 728 patient ED visits (mean (SD) patient age, 49.4 (21.0) years; 210 045 (60.8%) female), 1.16% had a diagnosis of OUD. Bivariate analyses indicated that history of OUD was the strongest predictor of current OUD (OR=13.4, CI: 11.8 to 15.1). When history of OUD was excluded in multivariate models, baseline use of medications for OUD (OR=3.4, CI: 2.9 to 4.0) and white race (OR=2.9, CI: 2.6 to 3.3) were the strongest predictors. The best Gradient Boosting model achieved an AUC of 0.71, accuracy of 0.96 but only 0.45 sensitivity.

Conclusions

Patients who present at the ED with OUD are high-need patients who are typically smokers with psychiatric, chronic pain and substance use disorders. A machine learning model did not improve predictive ability. A quick review of a patient’s EHR for history of OUD is an efficient strategy to identify those who are currently at greatest risk of OUD.