Machine learned text topics improve drop-out risk prediction but not symptom prediction in online psychotherapies for depression and anxiety

Abstract

Objective: Internet-delivered cognitive behavior therapies (iCBT) are effective and scalable treatments for depression and anxiety. However, treatment adherence remains a major limitation that could be further understood by applying machine learning methods to during-treatment messages. We used machine learned topics to predict drop-out risk and symptom change in iCBT. Method: We applied topic modeling to naturalistic messages from 18,117 patients of nationwide iCBT programs for depression and generalized anxiety disorder (GAD). We used elastic net regression for outcome predictions and cross-validation to aid in model selection. We left 10% of the data as a held-out test set to assess predictive performance. Results: Compared to a set of reference covariates, inclusion of the topic variables resulted in significant decrease in drop-out risk prediction loss, both in between-patient and within-patient session-by-session models. Quantified as partial pseudo-R2, the increase in variance explained was 2.1–6.8 percentage units. Topics did not improve symptom change predictions compared to the reference model. Conclusions: Message contents can be associated with both between-patients and session-by-session risk of drop-out. Our topic predictors were theoretically interpretable. Analysis of iCBT messages can have practical implications in improved drop-out risk assessment to aid in the allocation of additional supportive interventions.