Chatbots or Counselors? New Study Reveals How People Really Compare AI and Human Therapy

Millions of people now turn to conversational artificial intelligence when they need someone to talk to, and a persistent question has shadowed this shift: do people actually feel understood by a machine, or does human counseling remain irreplaceable? A new study published in BMC Psychology offers one of the most direct answers to date, because it did not compare chatbot users with therapy patients as separate groups. Instead, researchers surveyed 233 Chinese adults who had personally experienced both AI-based psychological support and human counseling, allowing each participant to serve as their own comparison point. The results complicate the popular narrative in both directions. AI support was not simply a degraded substitute for therapy, nor was it a superior replacement. Rather, the perceived strengths and weaknesses of the two modalities split along surprisingly specific psychological fault lines, with machines winning on low-stakes accessibility and humans retaining a clear edge in genuine understanding and professional authority.

The research team, led by Hong Wu, Quzhi Liu, and colleagues at Hohai University in Nanjing, designed a cross-sectional online survey around two parallel 11-item experience scales. Each item probed a distinct dimension of perceived support quality: acceptance, understanding, goal alignment, collaboration, professional competence, response efficiency, safety and privacy, trust, reduced concern about being judged, willingness for deep self-disclosure, and disclosure of negative emotions. Because every participant rated both modalities on identical items, the design enabled paired statistical comparisons that control for the stable personality traits and life circumstances each individual brings to any evaluation. This within-person approach matters enormously in this field, because people who choose to use chatbots may differ systematically from people who seek therapy, and those pre-existing differences can masquerade as modality effects in cruder between-group studies.

The statistical machinery behind the study was deliberately layered. The primary analysis used a paired-samples t test on a 10-item composite score, constructed after the researchers discovered that the response-efficiency item was not strictly measurement-equivalent across the two modalities, meaning that a chatbot’s instant reply and a therapist’s considered response may simply not measure the same underlying quality. To guard against distortion from this problematic item, the team removed it from both modality scores and re-ran the comparison. They also applied paired Wilcoxon signed-rank tests with Monte Carlo two-tailed p values and Holm correction to the item-level data, a procedure that controls the inflated risk of false positives when many comparisons are tested simultaneously. Finally, a repeated-measures linear mixed-effects model served as a sensitivity analysis, explicitly modeling the within-person paired structure and testing whether the size of the modality difference varied according to individual characteristics such as attachment style.

The headline finding was a small but statistically robust overall advantage for AI-based support. On the 10-item composite, AI support scored a mean of 57.77 with a standard deviation of 6.55, while human counseling scored 56.80 with a standard deviation of 6.75. The mean difference, calculated as human minus AI, was −0.97 points, with a 95 percent confidence interval spanning −1.73 to −0.21, a paired t value of −2.53 on 232 degrees of freedom, and a p value of .012. The standardized effect size was modest, dz = 0.17, which the researchers emphasize is a small difference. Yet its persistence after the potentially non-equivalent response-efficiency item was excluded suggests the pattern is not merely an artifact of comparing a chatbot’s instant replies with a therapist’s slower turnaround. At the item level, AI support received higher ratings on five exploratory indicators, while human counseling won on exactly two: mutual understanding and professional competence.

That two-item advantage for human counselors is arguably the most psychologically meaningful part of the results. Mutual understanding and professional competence are precisely the qualities that define the therapeutic relationship in classical accounts of counseling, and the study suggests they remain the territory where human practitioners are perceived to excel, even among people who also use and generally rate AI support favorably. Conversely, the domains where AI scored higher map onto what the researchers describe as low-threshold and low-social-threat experiences. Talking to a chatbot appears to reduce the fear of being judged, makes deep self-disclosure feel safer, and lowers the barrier to seeking support in the first place. For people who find the prospect of sitting across from a professional intimidating, a machine that never sighs, never frowns, and never files a clinical note about your worst thoughts may feel like a genuinely safer opening move.

Attachment theory provided the study’s central individual-difference framework, and the regression results revealed a coherent pattern. Attachment anxiety, characterized by a heightened need for reassurance and fear of rejection, positively predicted perceived experience quality in both modalities, meaning anxiously attached participants rated both AI support and human counseling more favorably. Attachment avoidance, characterized by discomfort with closeness and interdependence, negatively predicted both outcomes, so avoidantly attached participants rated both forms of support less favorably. In other words, the same relational dispositions color how people experience support whether it comes from a person or a machine, suggesting that chatbots do not escape the attachment dynamics that shape human relationships. Notably, specialized mental health and therapy chatbot use was associated with higher AI-experience scores, while associations involving shorter AI-use duration were less consistent, hinting that purpose-built tools may deliver a better experience than general-purpose assistants pressed into therapeutic duty.

The mixed-effects sensitivity analysis added a subtler layer to the attachment findings. The interaction between modality and attachment avoidance was statistically significant, indicating that the relative gap between AI-based support and human counseling was not uniform across participants but varied according to how avoidant they were in close relationships. Although the study’s published summary does not detail the direction of this interaction, its existence implies that attachment avoidance may shape which modality feels more comfortable, a possibility with direct clinical implications for matching people to support formats. Meanwhile, the hierarchical regression on the human-minus-AI difference score showed improved model fit after the attachment variables were added, but the final full model was not significant at the omnibus level, so the researchers treat the difference-score analysis as suggestive rather than conclusive.

The authors are unusually explicit about the limits of their evidence, and these caveats deserve as much attention as the findings. The design was cross-sectional and entirely self-report, so the data capture perceived experiences rather than objectively measured therapeutic outcomes, and no causal claims about which modality actually helps more can be sustained. The distributions showed restricted upper-end variability, with medians generally high and often identical across modalities, which compresses the statistical room to detect differences. The AI tools participants reported using were heterogeneous, spanning whatever chatbots they happened to try, and the 11-item experience indicators were newly developed for this study and remain exploratory rather than validated instruments. The researchers also caution that the sensitivity analysis relied on a composite score derived from ordinal item ratings, a measurement choice that warrants care when interpreting the small mean difference that survived.

What the study ultimately delivers is a differentiated map rather than a verdict. Among dual users, perceived experiences differed by domain rather than showing a uniform preference for either machines or humans. AI-based psychological support appears to excel as a low-barrier, judgment-free channel that encourages disclosure, while human counseling retains perceived advantages in mutual understanding and professional competence, the relational core of formal therapy. For clinicians, policymakers, and the rapidly growing industry building mental health chatbots, the implication is that the two modalities may be complements rather than competitors: AI could serve as an accessible first step or an adjunct between sessions, while human professionals remain the reference point for depth and expertise. As conversational AI becomes more capable and more widely adopted, studies that compare experiences within the same individuals, as this one does, will be essential for understanding not just whether people use these tools, but how the tools make them feel, and for whom they work best.

Subject of Research: Perceived experiences of AI-based psychological support compared with human counseling among adults who have used both modalities

Article Title: Differences in perceived experiences of AI-based versus human psychological support among dual users: paired comparisons, hierarchical regression, and repeated-measures sensitivity analysis

Article References: Wu, H., Liu, Q., Shi, Y., Yang, M., Zhang, J., & Zhang, Y. (2026). Differences in perceived experiences of AI-based versus human psychological support among dual users: paired comparisons, hierarchical regression, and repeated-measures sensitivity analysis. BMC Psychology. https://doi.org/10.1186/s40359-026-05676-y

Image Credits: AI Generated

DOI: 10.1186/s40359-026-05676-y

Keywords: conversational AI, mental health chatbots, human counseling, attachment theory, self-disclosure, user experience, paired comparison, hierarchical regression, mixed-effects model, psychological support, trust, BMC Psychology