Automated Detection of Cognitive Distortions in Text Exchanges Between Clinicians and People With Serious Mental Illness

Objective:

The authors tested whether natural language processing (NLP) methods can detect and classify cognitive distortions in text messages between clinicians and people with serious mental illness as effectively as clinically trained human raters.

Methods:

Text messages (N=7,354) were collected from 39 clients in a randomized controlled trial of a 12-week texting intervention. Clinical annotators labeled messages for common cognitive distortions: mental filtering, jumping to conclusions, catastrophizing, “should” statements, and overgeneralizing. Multiple NLP classification methods were applied to the same messages, and performance was compared.

Results:

A tuned model that used bidirectional encoder representations from transformers (F1=0.62) achieved performance comparable to that of clinical raters in classifying texts with any distortion (F1=0.63) and superior to that of other models.

Conclusions:

NLP methods can be used to effectively detect and classify cognitive distortions in text exchanges, and they have the potential to inform scalable automated tools for clinical support during message-based care for people with serious mental illness.