Linguistic features within individuals’ text data may indicate their mental health. This trial examined the linguistic markers of depressive and anxiety symptoms in adults. Using a randomised cross over trial design, 218 adults provided eight different types of text data of varying frequencies and emotional valance. Linguistic features were extracted using LIWC-22 and correlated with self-reported symptoms. Machine learning was used to determine associations. No linguistic features were consistently associated with depressive or anxiety symptoms within or across all tasks. Features associated with depressive symptoms were different for each task and there was only some degree of reliability of these features within tasks. In all machine learning models, predicted values were weakly associated with actual values. Some text tasks had lower levels of engagement and negative impacts on mood. Overall, the linguistic markers of depression and anxiety shifted in response to contextual factors and the nature of the text analysed. This trial was prospectively registered with the Australian New Zealand Clinical Trials Registry (date registered: 15 September 2021, ACTRN12621001248853).
Depression and anxiety are common mental illnesses that negatively affect the health and wellbeing of millions worldwide, eroding individuals’ quality of life and productivity. Identifying depressive and anxiety symptoms among individuals and populations in an automated and wide scale manner may allow for timely intervention at a population level, triaging risk, and preventing potential escalation into severe disorders and suicidality. Digital phenotyping is the emerging field of science that mines the digital data generated by individuals in the daily course of their lives to uncover “objective, quantifiable characteristics of illness that can be measured accurately and reproducibly”. There has been a rapid increase in the number of research studies attempting to use this approach to understand various mental health states and the need for intervention among populations and individuals. Automated linguistic analysis of individuals’ text data may be an advantageous method of digital phenotyping because individuals generate large volumes of text data daily, and these can be readily collected and analysed at scale using low-cost real time software. By segmenting individuals’ text data into linguistic features that can be observed and quantified, linguistic analysis has enabled researchers to rapidly explore the relationships between linguistic expression and mental illness.
To date, several studies across psychology and computer science have suggested that linguistic features expressed in different forms of individuals’ text data may provide markers of the cognitive, emotional, and social deficits of depression and anxiety. It has been hypothesised by many researchers that mental illness significantly alters one’s linguistic style whereby the words used by individuals may indicate the reduced positive emotion, ruminative self-preoccupation, low levels of motivation and pleasure, and difficulty foreseeing positive future events that characterise depressive and anxious states. Indeed, a meta-analysis of 21 studies of individuals’ written and spoken communications including personal essays, social media posts and clinician interviews, found that higher depressive symptoms were marked by the increased use of first-person singular pronouns, irrespective of gender or other demographics. A second meta-analysis of 26 studies of individuals’ linguistic expression in varied writing tasks found that individuals with depressive symptoms were distinguished from non-depressed individuals by small to moderate differences in first-person singular pronouns use, negative emotion word use and positive emotion word use. Another systematic review of 25 studies on the linguistic features of depression also found that first-person singular pronouns and negative emotion words were significantly associated with depressive symptoms, although the strength and magnitude of the results varied. However, several studies attempting to identify linguistic markers of depression and anxiety have had conflicting findings. This is likely influenced by differences in the text forms analysed and variations in methodology such as clinical versus community samples. Furthermore, the strength of the associations between linguistic features and symptoms are consistently weak suggesting limited utility for prediction. While collectively the studies suggest that there may be linguistic markers of mental illness, the validity, reliability and overall utility of this approach remains in its infancy.
There is also increasing evidence of a task effect whereby contextual factors (i.e., the purposes and conditions under which text data is created) may influence the strength and direction of the relationships between linguistic features and mental health symptoms. For example, Havigerova found that linguistic features expressed by individuals in a fictional letter to a friend inviting them on a holiday had greater predictive utility for depression than the linguistic features expressed in fictional formal cover letters, complaints, or apology letters. Similarly, Minori and Cuteri found that written personal reflections of a complex picture was the most useful text task for generating valid linguistic markers of depression when compared to written self-reflection tasks about one’s self or one’s friends. In contrast, Kazmierczak et al. found that depressed individuals consistently used the same types of language and word choices regardless of what they were describing, suggesting that some aspects of linguistic expression may be stable. However, these past studies have not accounted for the potential intervention effects of expressive writing tasks on participants’ mood or their levels of engagement. This is pertinent to the study of depression, given that this illness is characterised by low levels of motivation, biased processing of emotional stimulus, enhanced recall of negative emotional experiences, and difficulty foreseeing positive future events. It remains unclear whether mental health symptoms have caused variations in linguistic expression, the task itself, or interaction effects, as most prior studies have not controlled for these factors.
With the growth of smartphone and digital-mediated communications, researchers have examined the utility of organic text data such as SMS and social media posts for generating markers of mental illness. When examining SMS text data, Stamatis et al., found that individuals with higher depressive symptoms used significantly fewer words related to anticipation, trust, social processes, and affiliation whereas the inverse use of words was found for high anxiety symptoms. Also in SMS data, Meyerhoff et al. found that individuals with higher depressive symptoms used significantly more first-person singular pronouns, filler words, sexual words, anger and negative emotion words, but only in their SMS with close contacts. Several studies have revealed potential linguistic markers of depression in individuals’ social media posts with several supervised and unsupervised machine learning models showing promise for detecting depressive symptoms. Notably however, Waterloo et al. found that different digital communication platforms (e.g., WhatsApp, Facebook, Twitter and Instagram) elicited significant variations in an individual’s linguistic expression. This is consistent with Tlachac and Rundensteiner, who found that that individuals’ SMS messages were more useful for detecting depression than their Twitter posts. Similarly, Liu also found significant cross-platform differences in linguistic expression, such that a Facebook-derived model of linguistic predictors of depression performed poorly when applied to SMS data from the same individuals. Smirnov also found that the number and significance of linguistics markers in social media messages and personal essays about oneself and their relations varied according to depression status (i.e., clinical depression diagnosis versus high depressive symptomatology) and text type. Based on their results, Smirnov and colleagues also posited that the linguistic markers of depression were most likely influenced by the text topic and the conditions under which the text was written, rather than mental health status alone.
Overall, variations in the types of text data, the number of linguistic features examined, and the analytical approaches used have made it difficult to assess the validity and reliability of linguistic markers of mental illness. Past studies have been largely observational, collecting text data from a single timepoint and source. In studies collecting and combining several types of text data, there has been little attempt to control for order effects, whereby the completion of certain study activities such as mental health surveys or negatively-anchored writing tasks may prime individuals to respond in particular ways. Many studies have not accounted for the varied contextual factors such as the individual, the communication modality, the audience domain, the task stimuli, or the study conditions, that may influence the emergence of linguistic markers of mental illness. Furthermore, the rapid changes in digital communication patterns among individuals as evidenced by generational shifts to short-form abbreviated messages and image-based content combined with the expansion of internet security measures to protect individuals’ data privacy, has made the automated extraction and mining of digital data by third parties increasingly difficult. As digital phenotyping gains popularity and data collection becomes more invasive, it is essential that researchers have a strong scientific and ethical rationale for extracting and analysing of large volumes of personal text data. A clearer understanding of the potential utility of text data for providing meaningful insight into the health and wellbeing of individuals will help to ensure this.
Using a randomised cross over experimental design to control for potential order and task effects, this trial aimed to examine the validity and reliability of linguistic features of symptoms of depression and anxiety in symptomatic adults by comparing several text types previously found to generate significant markers. This study controlled for potential reverse-causation and interaction effects by separating out the text tasks, randomising delivery and collecting multiple samples over time. Furthermore, this study explored whether markers found at the task-level remained significant when all text data was combined into one corpus. Using machine learning models, this study aimed to determine which linguistic features were the most strongly associated with depressive and anxiety symptoms. This study also examined the acceptability of the text tasks and the effects of these on participants’ mood and engagement in this type of digital phenotyping. Based on prior findings, it was hypothesised that several linguistic features would be significantly associated with symptoms of depression and anxiety but there would be variability in the significance of these markers depending on the task type. Consequently, it was hypothesised that markers found to be significant at the task level may not retain their significance when all text data was combined. It was also hypothesised that text tasks with negative sentiment would negatively impact participants’ mood and be less engaging to complete. The findings of this investigation will help to determine the validity and reliability of linguistic markers of depression and anxiety and the acceptability of collecting text data for this approach.
