The research, showcased during the recent Bias-a-Thon competition at Penn State’s Center for Socially Responsible AI, reveals that traditional methods of examining bias—often reliant on sophisticated technical knowledge—might not adequately represent the day-to-day interactions between average users and AI. In contrast to expert-driven techniques—often resembling a cat-and-mouse game where programmers test the limits of AI’s guardrails—this new approach emphasizes the importance of understanding how everyday individuals engage with AI systems.

In participating in the Bias-a-Thon, a diverse group of fifty-two contenders—comprised largely of individuals without an in-depth background in tech—submitted challenges aimed at exposing bias within popular AI models, such as ChatGPT and Gemini. The intention was simple yet vital: to demonstrate that a straightforward, intuitive prompt is potent enough to trigger biased responses akin to those generated using advanced technical inquiries. This research digs deep into the biases that shape AI, encouraging a dialogue that transcends the esoteric barriers often associated with AI technology.

The researchers began their investigation by meticulously analyzing 75 unique prompts submitted to the contest. Each submission was accompanied by the participants’ insights into the discriminatory responses from the AI models. Interestingly, the analysis revealed that intuitive strategies employed by casual users were frequently just as capable of eliciting biased outputs as those used by technical experts, underscoring a crucial point: the accessibility of AI does not guarantee its fairness.

They examined the very nature of bias in AI systems, pointing out that such biases often stem from historical prejudices embedded in training data. These can range from language biases—where certain vernaculars are favored over others—to racial and gender biases that have permeated societal constructs. Beyond simply identifying these flaws, the research focused on how users perceive and manipulate AI capabilities, offering insights into how biases may be better recognized and addressed.

The research team initiated interviews via Zoom with a subset of participants, allowing them to expand on their prompting strategies and their conceptions of fairness, representation, and stereotypes. With systematic evaluation, they formulated a working definition of bias, encapsulating aspects such as prejudice towards specific groups, lack of representation, and the promotion of stereotypes. Through this user-informed lens, their work aims to bridge the gap between technical analysis and practical user experiences with AI.

The significance of the findings came to light further when they engaged with various large language models (LLMs) to test the reproducibility of the answers yielded from the prompts: a crucial aspect of validating their research. The inherent randomness that LLMs possess complicates consistent outcomes, as participants can receive wholly different responses to identical questions on separate occasions. The researchers meticulously filtered prompts that exhibited reproducible results, setting the stage for a structured exploration of the biases at play.

Notably, they identified eight distinct categories of bias that maltreated various societal groups: gender bias, racial and religious bias, age bias, disability bias, language bias, historical biases that favor Western ideologies, cultural biases, and political biases. Each of these categories provides a foundation for further analysis, revealing a spectrum of potential harm that AI-generated content can inflict on marginalized communities if left unchecked.

Equally compelling were the seven proactive strategies participants employed to elicit these biases. Some participants assumed personas to challenge the models, while others devised hypothetical scenarios designed to explore nuanced societal issues. This meant that casual users were effectively leveraging their personal knowledge and experiences to spotlight AI’s shortcomings, revealing just how impactful informed users can be in unveiling biases.

One of the most striking contributions of the competition was a new set of biases brought to light—an unexpected finding considering the established literature on AI bias. For instance, a revelation surfaced regarding conventional beauty standards; the AI models exhibited a troubling tendency to associate trustworthiness and employability with specific physical traits, clearly privileging individuals based on arbitrary aesthetic benchmarks. This finding symbolizes the potential for everyday users to uncover biases that may have escaped the analytical gaze of seasoned researchers.

The study’s implications stretch far and wide, prompting developers within the AI domain to reconsider their approaches to bias mitigation. The researchers approached the ongoing challenges of addressing biases within AI with a metaphor of a cat-and-mouse game, emphasizing the constantly evolving landscape of AI technology. Specific recommendations for developers include implementing rigorous classification filters to screen outputs prior to delivery, performing exhaustive testing on their models, and fostering user education on the nuances of AI interactions.

Moreover, the Bias-a-Thon holds intrinsic value beyond just highlighting shortcomings; it serves a broader educational purpose by elevating the discourse on AI literacy among general populations. With a clarion call for systematic awareness of AI shortcomings, the event reflects a growing recognition of the need for informed usage of such technologies.

As discussions on responsible AI development enter a new phase, researchers from Penn State—and various contributors from industry and academia—are working tirelessly to ensure AI evolves in ways that are cognizant of societal impact. Each step taken to understand and mitigate inherent biases is a stride towards a future where AI can be a beneficial tool for all, rather than a perpetuator of disparities.

The Bias-a-Thon not only encapsulates a novel methodology for critiquing AI but also acknowledges the critical role that engaged users play in refining these technologies. This engagement is pivotal; as more users become aware of the biases inherent in AI outputs, they can actively participate in the discourse around ethically responsible AI technologies. The ongoing dialogue and collaboration across various sectors will ultimately shape the trajectory of AI development, ensuring it becomes a robust ally in the promotion of fairness and equity in our increasingly digital society.

As the findings continue to circulate and gain traction, it is essential that both the tech industry and academic research communities take heed of the nuanced perspectives provided by everyday users. The complexities of AI biases require a multifaceted approach: informing the public, fostering responsible development practices, and continuously engaging users in this crucial dialogue. The future of AI should not just be a technological marvel; it must also be grounded in principles of equity and understanding, reflecting the diverse voices that populate our global landscape.

Through collaborative efforts such as the Bias-a-Thon, stakeholders are encouraged to join forces to illuminate blind spots in AI, ensuring that our technology not only evolves but grows to serve everyone fairly and justly.

Subject of Research: Bias in AI algorithms and user interactions
Article Title: Exposing AI Bias by Crowdsourcing: Democratizing Critique of Large Language Models
News Publication Date: 15-Oct-2025
Web References: http://dx.doi.org/10.1609/aies.v8i2.36620
References: Not available
Image Credits: Credit: CSRAI / Penn State

Keywords

The study, published in the journal Media Psychology, reveals that many AI models exhibit a troubling tendency: they misclassify emotions of individuals from minority racial groups, notably Black individuals, while performing relatively well for white subjects. This phenomenon arises because the training datasets behind these AI systems contain uneven representation of racial groups and their emotional expressions. For instance, datasets may have an overabundance of happy white faces, fostering an unintentional but damaging association of positive emotion predominantly with white race, while sad or negative emotions become linked with Black individuals.

S. Shyam Sundar, Evan Pugh University Professor and director of the Center for Socially Responsible Artificial Intelligence at Penn State, explains that AI “seems to have learned that race is an important criterion for determining whether a face is happy or sad” — not because developers intended this, but because the underlying data fed into the algorithms reflected existing societal imbalances. This unintended learning of racial biases within AI models echoes a wider concern in machine learning: the danger of “unanticipated correlations” when training data does not adequately represent diversity.

To explore whether ordinary users — laypersons — can recognize such biases embedded in AI systems, researchers designed a series of experiments. They constructed 12 versions of a prototype facial expression recognition AI, intentionally introducing skewed training data variations. Then, with a diverse pool of 769 participants from different racial backgrounds, the study tested whether individuals could detect biases in how the AI performed. Intriguingly, the overwhelming majority of participants failed to identify the racial bias, particularly if they belonged to the majority white group or were not affected by the misclassified emotions.

Lead author Cheng “Chris” Chen, an assistant professor at Oregon State University with a background in mass communications from Penn State’s Donald P. Bellisario College, highlights that although users tended to trust AI systems as neutral arbiters, this misplaced trust falters when discrepancies became evident. Black participants, especially when faced with biased AI outputs that misclassified their emotional expressions, were more inclined to note racial disparities. However, this recognition was often limited to specific emotions, notably negative affect such as sadness in Black faces, which were overrepresented in the biased training data.

The experiments were carefully designed with different scenarios. The first scenario presented participants with AI training data showing an uneven racial distribution in emotional categories—happy faces overwhelmingly white, sad faces predominantly Black. The second scenario stripped away minority representation altogether, showing only white faces across emotional categories. The third experiment combined these scenarios, featuring five distinct conditions from racially biased happy or sad faces to balanced, unbiased datasets. Participants’ perceptions of the AI’s fairness across racial groups were then assessed.

Results consistently demonstrated a troubling blindness to bias. Most participants did not perceive any unfairness or racial preference in the AI’s treatment of facial expressions when simply viewing the training data. It was only when the AI’s performance displayed overt bias — often by failing to accurately classify emotions in minority groups — that perceptions shifted. Black participants were more sensitive to these cues, suggesting that personal experience with misrepresentation sharpens awareness of bias, whereas others may overlook it entirely.

Sundar emphasizes the psychological dimension of these findings. The study underscores a wider societal issue: people tend to assume AI systems are objective and neutral, despite evidence to the contrary. This misplaced trust magnifies the potential harm caused by algorithmic bias, as users and developers may not scrutinize the underlying data or question how AI learns. Such oversight risks perpetuating racial stereotypes and unequal treatment, compounding social inequalities.

Chen further notes that the tendency to ignore training data biases and instead focus solely on AI’s apparent performance outcomes is problematic. When users observe racially biased results, they tend to form impressions based on these visible outcomes rather than understanding the root causes embedded in skewed datasets. This misperception highlights an urgent need for improved AI literacy — broadening public and expert understanding of how data shapes AI behavior and the societal implications of biased training.

Looking forward, the research team plans to pursue new ways of communicating bias in AI to a wide range of stakeholders, including users, developers, and policymakers. Better transparency and education about the origins and impacts of algorithmic bias are essential for fostering more equitable AI systems. The researchers view advancing media and AI literacy as a crucial defense against blind trust and uncritical acceptance of AI outputs, particularly when these technologies increasingly influence decision-making in consequential domains.

This study serves as a timely reminder that AI, despite its promise to enhance human capabilities, is only as fair and accurate as the data it learns from. If racial biases embedded in training datasets remain unrecognized or unchallenged, AI systems risk reinforcing and even amplifying existing prejudices. The path forward requires vigilance, interdisciplinary collaboration, and a commitment to equity that transcends technical innovation alone.

In essence, the research spotlights the human factor in AI bias: not only how machines learn from data, but how people perceive and react to AI behavior. Only by addressing both elements can societies hope to build AI that truly serves everyone — transcending the limitations imposed by skewed historical data and deep-seated social inequities.

Subject of Research: Racial bias in AI training data and laypersons’ ability to detect such bias
Article Title: Racial Bias in AI Training Data: Do Laypersons Notice?
News Publication Date: 16-Sep-2025
Web References: http://dx.doi.org/10.1080/15213269.2025.2558036
Keywords: Artificial intelligence, Generative AI, Psychological science, Behavioral psychology, Human social behavior, Group behavior, Communications, Mass media, Social groups, Social issues, Racial discrimination, Social discrimination