Speech sentiment analysis shows proof-of-concept accuracy for classifying depressive symptoms among 116 Hong Kong family caregivers

A proof-of-concept study of 116 family caregivers in Hong Kong found that dimensional sentiment features derived from speech could distinguish participants with higher from lower depressive symptom scores, although performance remained limited and the analysis was not externally validated. A quadratic discriminant analysis using both valence and arousal features achieved an out-of-sample sensitivity of 0.725, specificity of 0.563 and area under the receiver operating characteristic curve of 0.700.

The secondary exploratory analysis used data originally collected to develop a conversation-based depression screening tool for community staff. Participants were 116 adults recruited from a single Hong Kong nonprofit community center between August 2018 and July 2019. Eligibility required at least 14 hours of unpaid care per week for one or more family members and the ability to communicate verbally in Cantonese. People whom registered social workers judged unable to complete the interview or instruments because of severe psychiatric disorders or cognitive impairment were excluded.

Depressive symptom severity was measured with the Chinese version of the Beck Depression Inventory II, a 21-item self-report scale ranging from 0 to 63. A score of 20 was used to divide participants into groups: 84 scored below 20 and were classified as having low depressive symptoms, while 32 scored 20 or above and were classified as having high depressive symptoms. The classification therefore concerned elevated depressive symptoms rather than formally diagnosed depression, which was not ascertained through a clinical interview.

All participants answered 12 open-ended questions about family circumstances and family resilience. Their recorded responses were transcribed with automatic speech recognition, normalized into Traditional Chinese and tokenized with Cantonese-language tooling. Words found in the Chinese Valence-Arousal Words dictionary were assigned baseline valence and arousal ratings. The researchers also modeled the effects of negation and gradable adverbs, then calculated distributional, temporal, frequency-domain and synchrony features from the resulting sequences.

Participants in the high-symptom group generally used words with more negative valence and lower arousal, but differences between the groups in mean valence and mean arousal were not statistically significant. Mean valence differed by a t value of −1.55, p = 0.124 and 95% confidence interval of −0.165 to 0.020, while mean arousal differed by a t value of −0.71, p = 0.482 and 95% confidence interval of −0.103 to 0.049. The study also reported that the high-symptom group had more irregular valence patterns and more high-arousal spikes, with statistically significant group differences and medium effect sizes for both measures.

Nine machine-learning classifiers were evaluated under repeated five-fold cross-validation with 100 independent repetitions. The validation procedure selected classification cutoffs by minimizing an expected-cost function that penalized false negatives more heavily than false positives. Across the models, sensitivity was generally higher than specificity. For classifiers using combined valence and arousal predictors, QDA produced a sensitivity of 0.725, specificity of 0.563 and AUC of 0.700, while k-nearest neighbors produced a sensitivity of 0.746 and specificity of 0.509.

Valence-only and arousal-only configurations produced different trade-offs. QDA based on valence alone had sensitivity of 0.763 and specificity of 0.521, while linear discriminant analysis based on arousal had sensitivity of 0.734 and specificity of 0.323. The valence-only QDA configuration also had a lower expected cost of 1.416 than the combined QDA configuration, which had an expected cost of 1.528, reflecting the greater penalty applied to missed high-symptom cases. The authors therefore characterized the results as evidence that emotional-word features can provide discriminatory information, rather than as validation of a screening tool.

Several limitations constrain the findings. The sample was small, specific and drawn from one organization, and the 84-to-32 class imbalance could affect reported performance metrics. Symptom groups were based on self-report rather than clinical diagnosis. Automatic transcription may have introduced errors, and the analysis did not include acoustic features such as tone, pitch or speech rate. The researchers also did not systematically assess whether participants found the family-resilience questions less stigmatizing or intrusive, and the study did not conduct external validation in other populations or settings.

Overall, the study provides exploratory evidence that valence and arousal patterns in caregiver speech can carry information associated with depressive symptom severity. The most favorable combined model found 72.5% of high-symptom participants and correctly identified 56.3% of those with lower symptom scores under cross-validation. The authors concluded that further refinement and validation in larger, more diverse and clinically characterized samples are needed before the method could be considered for practical use, including possible integration with other linguistic and acoustic features.

Subject of Research: Exploratory speech sentiment analysis of depressive symptoms in Hong Kong family caregivers

Article Title: Using valence and arousal scoring in speech to detect depressive symptoms: A study on family caregivers

Article References: So, M. K. P., Tsang, J. T. Y., Chan, L. S. H., Chan, J. N. L., Tiwari, A., & Chu, A. M. Y. (2026). Using valence and arousal scoring in speech to detect depressive symptoms: A study on family caregivers. PLOS One, 21(10), e0359556. https://doi.org/10.1371/journal.pone.0359556

Image Credits: AI Generated

DOI: 10.1371/journal.pone.0359556

Keywords: depressive symptoms, speech analytics, family caregivers, dimensional sentiment, valence, arousal, machine learning, cross-validation, Hong Kong, scoring, speech, detect