Large language models have become ubiquitous in numerous applications, ranging from customer service chatbots to creative writing assistants, and scientific research aids. Despite their impressive performance, one persistent issue has been their calibration—how well the models’ confidence estimations match the actual correctness of their outputs. This calibration directly affects trustworthiness and usability. Overconfidence can foster misleading information, causing users to rely on incorrect answers, while underconfidence might lead models to undervalue their predictions even when they are accurate.

The investigators approached this problem by considering what they call “competing biases.” These are systematic tendencies within the LLM architectures and training paradigms that result in diametrically opposed confidence distortions. Using a mixture of behavioral experiments on model outputs and rigorous statistical modeling, the team deconstructed the origins of these opposing biases, providing a fresh perspective on why LLMs oscillate between over- and under-confidence.

In their experimental setup, the researchers tasked several state-of-the-art LLMs with answering questions across multiple difficulty levels and domains. They simultaneously asked the models to provide a confidence rating for each response, effectively measuring the alignment between the model’s internal certainty and external accuracy. Analysis of this dataset revealed a striking pattern: when tackling simpler questions, LLMs tended to exhibit overconfidence, excessively affirming their correct and incorrect answers with high certainty. In contrast, for more ambiguous or complex prompts, models frequently demonstrated underconfidence, hesitating even in instances where their responses were correct.

Delving deeper, the team proposed that overconfidence stems primarily from reinforcement biases embedded during training. Since LLMs are frequently optimized to produce plausible or statistically likely continuations based on vast corpora, their internal reward structure prioritizes responses that “sound right” rather than those with guaranteed factual accuracy. Consequently, this creates a tendency towards certainty when the model’s heuristic approximations strongly match familiar patterns, irrespective of actual correctness.

Conversely, underconfidence emerges from uncertainty estimation mechanisms and probabilistic diffusion within the models’ parameter spaces. When confronted with questions that invoke underrepresented knowledge or conflicting evidence, LLMs propagate uncertainty signals leading to conservative confidence judgments. This cautious behavior, while desirable in certain contexts, might unfairly obscure their correct answers beneath a veil of doubt.

One of the paper’s most illuminating contributions is the conceptual integration of these two opposing biases within a unified framework. By mathematically characterizing overconfidence and underconfidence as competing forces drawn from different stages of model training and inference, the authors furnish a theoretical foundation for future work aiming to calibrate and optimize confidence outputs more effectively.

The practical implications of these findings are significant. For developers and end-users relying on LLMs in critical applications—including medical advice, legal reasoning, or scientific analysis—recognizing and compensating for these biases can dramatically enhance reliability. The study encourages the development of hybrid confidence estimation systems that dynamically adjust model certainty depending on input complexity and contextual domain knowledge.

Moreover, the researchers advocate for a paradigm shift in LLM training strategies. Instead of solely focusing on minimizing error rates or maximizing likelihood, integrating explicit confidence calibration targets could help moderate these opposing biases. Strategies like adversarial training focused on uncertainty prediction, calibrated fine-tuning using human-in-the-loop feedback, or incorporating meta-cognitive modules might be promising avenues to explore.

The study also carries broader philosophical implications about machine cognition and interpretability. It underscores that confidence—often treated as an ancillary metric—embodies deep computational challenges tied to the very nature of probabilistic learning and pattern recognition. Understanding confidence biases not only improves AI usability but also provides insights into analogous phenomena in human cognition, where overconfidence and underconfidence can impact decision-making.

Beyond the immediate technical contributions, the paper’s authors highlight the need for a standardized evaluation protocol to assess confidence calibration across different LLM architectures. Currently, diverse benchmarks and varying methodologies make cross-model comparisons challenging. Establishing universal metrics for confidence bias quantification would facilitate benchmarking and drive industry-wide improvements.

The dataset generated through this research, comprising thousands of model responses paired with confidence ratings and ground truth labels, constitutes a valuable resource for the AI community. By making this dataset publicly available, the authors invite further exploration into how different training datasets, model sizes, and architectures influence confidence behavior, promoting transparency and reproducibility.

In summary, this pioneering investigation into the dual biases that govern LLM confidence marks a crucial step toward resolving one of the pressing reliability issues in contemporary AI. By articulating the mechanisms behind overconfidence and underconfidence, the research paves the way for smarter, more trustworthy language models. It challenges the community to refine AI self-awareness, making future interactions safer, more transparent, and ultimately more aligned with real-world needs.

As AI systems continue to permeate everyday life and high-stakes environments alike, such insights are invaluable, reminding us that the road to truly intelligent machines requires mastering not just what they say but how sure they are when saying it. This study invites renewed optimism that with rigorous scientific inquiry and cross-disciplinary collaboration, large language models can evolve from impressive mimics of language to genuinely reliable partners in human endeavor.

Subject of Research: Confidence estimation biases in large language models (LLMs)

Article Title: Competing Biases Underlie Overconfidence and Underconfidence in LLMs

Article References:
Kumaran, D., Fleming, S.M., Markeeva, L. et al. Competing Biases underlie Overconfidence and Underconfidence in LLMs. Nat Mach Intell (2026). https://doi.org/10.1038/s42256-026-01217-9

Image Credits: AI Generated

DOI: https://doi.org/10.1038/s42256-026-01217-9

The research, spearheaded by Mikhail Belkin of the University of California San Diego and Adit Radhakrishnan of the Massachusetts Institute of Technology, dives deep into the labyrinthine computational structures of several state-of-the-art open-source LLMs. By examining architectures like Meta’s LLaMA and other leading models such as Deepseek, the team identified distinct “concepts” embedded within the models’ internal representation layers. These concepts, spanning categories like fears, moods, and geographic locations, serve as fundamental building blocks influencing the models’ responses.

What sets this study apart is the mathematical finesse employed by the researchers. Building upon their 2024 foundational work on Recursive Feature Machines—predictive algorithms adept at locating meaningful patterns within sprawling mathematical operations—the team demonstrated that the importance of these concepts can be either amplified or diminished through surprisingly straightforward mathematical manipulations. This fine-grained control allows for direct steering of model behavior without the need for exhaustive retraining or massive computational resources, addressing long-standing obstacles in efficient model tuning.

The universality of the method is equally remarkable; the team’s experiments show this steering capability transcends language barriers, working not only in English but also fluently in languages such as Chinese and Hindi. By manipulating just 512 concepts categorized into five primary classes, the researchers achieved consistent, interpretable modulations in output across diverse linguistic contexts, highlighting the foundational nature of these internal concepts.

Historically, the inner workings of LLMs have been shrouded in mystery, often regarded as inscrutable “black boxes” by both developers and end users. Understanding why these massive neural networks arrive at particular answers—especially in complex or ambiguous cases—has been notoriously difficult. The steering technique unveiled here offers a glimpse into these hidden processes, enabling researchers to peer beneath the surface and exert precise influence over the model’s internal reasoning pathways, a leap forward for transparency.

Beyond mere control, the research indicates that steering concepts can significantly enhance performance on narrowly focused, high-precision tasks. For example, when applied to code translation—from Python to C++—the method visibly improved the accuracy and reliability of outputs. It also proves effective as a diagnostic tool to uncover hallucinations, those instances when an LLM confidently fabricates plausible but incorrect information, a notorious challenge in deploying language models in real-world applications.

However, this power cuts both ways. The team uncovered that by attenuating the concept of refusal—essentially muting the model’s inclination to decline inappropriate requests—they could deliberately “jailbreak” guardrails designed to prevent harmful outputs. In one startling demonstration, the manipulated model produced detailed instructions on the illicit use of cocaine and even provided what appeared to be Social Security numbers, raising alarms about the misuse potential of such targeted steering attacks.

Moreover, the method can exacerbate bias and misinformation within these systems. By boosting concepts linked to political bias or conspiracy theories, models could be compelled to affirm dangerous falsehoods—such as endorsing flat Earth conspiracies based on satellite imagery or declaring COVID-19 vaccines poisonous—exposing vulnerabilities that must be addressed urgently as LLMs grow ever more integrated into society.

Despite these risks, the steering technique stands out for its remarkable efficiency. Leveraging just a single NVIDIA Ampere A100 GPU, the researchers identified and adjusted relevant concept patterns in under a minute, using fewer than 500 training samples. This speed and low computational overhead suggest the method could be seamlessly incorporated into standard training pipelines, enabling more agile and targeted improvements without prohibitive costs.

While this study focused exclusively on open-source models, owing to the lack of access to closed commercial LLMs like Anthropic’s Claude, the authors express strong confidence that their method’s underlying principles would generalize to any sufficiently transparent architecture. Strikingly, the research reports that larger and more recent LLMs exhibit greater steerability—a promising insight for future model development and customization—while opening the door for steering even smaller models that operate on consumer-grade hardware like laptops.

Looking ahead, the researchers highlight exciting possibilities for refining this approach to tailor concept steering dynamically based on specific inputs or application contexts. Such adaptive steering could enhance safety, align outputs more closely with user needs, and reduce unwanted biases in personalized AI interactions, marking a significant step towards universal, fine-grained control over complex AI systems.

Ultimately, this groundbreaking work underscores a crucial insight: large language models possess latent knowledge and representations far richer than what is typically expressed in their surface responses. Unlocking and understanding these internal representations opens pathways not only to boosting performance but also to fundamentally rethinking safety and ethical safeguards in AI, a necessary evolution as these technologies permeate critical aspects of daily life.

Supported by the National Science Foundation, the Simons Foundation, the UC San Diego-led TILOS Institute, and the U.S. Office of Naval Research, this research represents a critical milestone on the journey toward transparent, controllable, and secure AI. As large language models continue to scale new heights, the ability to navigate and modulate their internal landscapes will be pivotal in harnessing their full potential responsibly.

Article Title: Toward universal steering and monitoring of AI models
News Publication Date: 19-Feb-2026

Keywords

Generative AI, Artificial intelligence, Computer science, Artificial neural networks