Beyond human expertise: the promise and limitations of ChatGPT in suicide risk assessment

Suicidal thoughts and behaviors (STBs) are major public health problems that have not declined appreciably in several decades. One of the first steps to improving the prevention and treatment of STBs is to establish risk factors (i.e., longitudinal predictors). To provide a summary of current knowledge about risk factors, we conducted a meta-analysis of studies that have attempted to longitudinally predict a specific STB-related outcome. This included 365 studies (3,428 total risk factor effect sizes) from the past 50 years. The present random-effects meta-analysis produced several unexpected findings: across odds ratio, hazard ratio, and diagnostic accuracy analyses, prediction was only slightly better than chance for all outcomes; no broad category or subcategory accurately predicted far above chance levels; predictive ability has not improved across 50 years of research; studies rarely examined the combined effect of multiple risk factors; risk factors have been homogenous over time, with 5 broad categories accounting for nearly 80% of all risk factor tests; and the average study was nearly 10 years long, but longer studies did not produce better prediction. The homogeneity of existing research means that the present meta-analysis could only speak to STB risk factor associations within very narrow methodological limits-limits that have not allowed for tests that approximate most STB theories. The present meta-analysis accordingly highlights several fundamental changes needed in future studies. In particular, these findings suggest the need for a shift in focus from risk factors to machine learning-based risk algorithms. (PsycINFO Database Record

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

Many countries are developing suicide prevention strategies for which up-to-date, high-quality evidence is required. We present updated evidence for the effectiveness of suicide prevention interventions since 2005.