AI Companions May Widen Social Divides, New Framework Warns

Artificial intelligence companions have moved rapidly from science fiction into everyday life, offering conversation, emotional support and a sense of connection to millions of users. As these systems become more widespread, researchers from Singapore Management University and Duke-NUS Medical School argue that society is asking the wrong question about them. In a study published recently in Nature Human Behaviour, the team contends that the debate over whether AI companions are simply beneficial or harmful misses the point. The more urgent issue, they say, is who actually benefits from these technologies, who bears the greatest risks, and why those outcomes are so unevenly distributed across different groups of people.

The paper, titled How AI companions could deepen social inequality, introduces a new framework for understanding how AI companionship may reinforce existing social inequalities across three interconnected dimensions: the characteristics of users themselves, the design choices made by platforms, and the governance structures that surround the technology. Rather than treating risks in isolation, the framework shows how these factors interact to produce unequal outcomes, and it proposes practical interventions at each level. The study was led by first author Zhang Qiyang, Assistant Professor in Learning Analytics at SMU’s College of Integrative Studies, together with Zhang Renwen, Nanyang Assistant Professor at Nanyang Technological University’s Wee Kim Wee School of Communication and Information, and senior author Liu Nan, Associate Professor at Duke-NUS Medical School’s Centre for Biomedical Data Science.

At the heart of the research is a striking observation about human relationships that the authors describe as a rich-get-richer dynamic. People who already enjoy strong family ties and robust social networks are more likely to use AI companions as a supplement to their existing relationships. For these users, the technology can serve as a rehearsal space, a place to practise difficult conversations or manage stress before facing real interactions. Their social skills remain sharpened by regular human contact, and the AI adds convenience rather than replacing genuine connection.

The picture looks very different for people who are lonely, socially isolated or have limited access to mental health support. According to the study, these users are more likely to turn to AI companions as a substitute for human connection rather than a complement to it. Over time, this reliance may contribute to what the researchers call social deskilling, the gradual erosion of interpersonal skills that comes from having fewer opportunities to practise authentic human interaction. In this way, the technology may deliver its greatest advantages to those who need it least, while leaving the most vulnerable users increasingly exposed to harm.

To explain how such inequalities emerge, the study adapts the Swiss cheese model, a framework widely used in engineering and safety science to describe how multiple small failures align to produce larger systemic risks. The model, familiar from aviation and healthcare safety analysis, pictures each protective layer as a slice of cheese containing holes. Disasters occur only when the holes in successive slices line up, allowing a hazard to pass through every barrier at once. Applied to AI companionship, the framework suggests that harm rarely stems from a single cause.

Instead, the researchers argue that negative outcomes occur when several protective layers fail simultaneously. These layers include users’ own AI literacy, the strength of their social support networks, the design choices made by platform developers and the quality of regulatory oversight. Each layer contains weaknesses, and when those weaknesses align, vulnerable users become significantly more exposed to harmful outcomes. The authors present this systems perspective as a significant shift from existing approaches, which tend to focus on individual users, technology companies or regulation in isolation rather than examining how these elements interact.

Among the four protective layers examined, the study identifies governance as the most pressing concern. Despite being increasingly used for emotional support and mental well-being, AI companions currently occupy a regulatory grey area in many countries. Most are governed as consumer applications rather than as technologies with potential psychosocial or mental health impacts, a classification that leaves substantial gaps in protection. The researchers argue that recognising AI companions as health-related technologies could unlock stronger safeguards, including age-appropriate design requirements, transparent disclosure that users are interacting with AI rather than a human, limits on emotionally manipulative engagement features, clearer data governance and privacy standards, and mandatory crisis-response protocols for users who disclose self-harm or other high-risk situations.

The urgency of these questions is particularly visible in Singapore, which the study highlights as both well positioned and uniquely exposed to the rapid rise of AI companionship. The country’s advanced digital infrastructure, strong policymaking capabilities and history of proactive technology governance position it to lead internationally in developing responsible safeguards. At the same time, high smartphone adoption, widespread AI usage, an ageing population, growing numbers of older adults living alone, and increasing concerns around youth mental health and loneliness create conditions in which AI companions are likely to be adopted quickly, particularly among vulnerable groups. Zhang Qiyang is also developing a global policy dashboard tracking countries’ governance frameworks for AI mental health technologies, and early findings indicate that many jurisdictions have yet to establish dedicated policies or guidelines for AI companions.

Building on its framework, the study outlines practical measures for policymakers. First, it argues that AI companions should be recognised as technologies with psychosocial and mental health implications rather than regulated solely as consumer or entertainment applications, an approach that could support age-appropriate design standards, clearer disclosure of AI identity, stronger privacy protections and consistent crisis-response protocols. Second, it calls for a layered governance approach, arguing that no single intervention can adequately address the risks of AI companionship. Policymakers should combine regulation with efforts to strengthen AI literacy, encourage responsible platform design and support real-world social connection, reducing the likelihood that vulnerable users become overly reliant on AI for emotional support.

The researchers emphasise that the trajectory of AI companionship is not fixed by the technology itself but by the choices society makes about how to design, govern and use it. Liu Nan, who also directs the Duke-NUS AI + Medical Sciences Initiative, notes that the rapid evolution of AI companions presents an opportunity to shape their role in society before widespread adoption outpaces governance, and that young people in particular may be less equipped to recognise the limitations or commercial incentives behind these systems, making thoughtful safeguards especially important. The hope, the authors conclude, is that with thoughtful regulation, responsible product design and stronger public AI literacy, AI companions can complement human relationships and support well-being while strengthening, rather than replacing, meaningful human connection, and without deepening the social inequalities that already divide communities.

Subject of Research: How AI companions may deepen social inequality through user, design and governance factors

Article Title: SMU-Duke-NUS study offers new framework for understanding how AI companions may shape social inequality

Article References: SMU-Duke-NUS study offers new framework for understanding how AI companions may shape social inequality. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: AI companions, social inequality, Swiss cheese model, governance, mental health, social deskilling, Nature Human Behaviour, AI literacy, platform design, regulation, loneliness, Singapore