Noah Golowich’s Theory Explains How AI Can Learn Stable Strategies in Games and Unfamiliar Worlds
When an artificial intelligence system defeated some of the world’s strongest professional poker players in 2017, the breakthrough was not simply a demonstration of faster calculation or superior memory. It reflected a deeper mathematical idea: an AI can search for a stable strategy in which no participant can improve their outcome by changing tactics alone. That concept, known as a Nash equilibrium, has become one of the central targets in computational game theory. Now, theoretical computer scientist Noah Golowich has developed new results that help explain why certain learning algorithms can discover such equilibria, how quickly they can do so, and what happens when an AI must make decisions with incomplete information. His doctoral thesis, “Theoretical Foundations for Learning in Games and Dynamic Environments,” has received the 2025 Hertz Thesis Prize from the Fannie and John Hertz Foundation.
Golowich completed his PhD at the Massachusetts Institute of Technology under the supervision of Constantinos Daskalakis and Ankur Moitra, and later carried out a postdoctoral research fellowship at Microsoft Research in New York City. He has now joined the University of Texas at Austin as an assistant professor, where he plans to study both the mathematical foundations and practical behavior of generative AI systems, including large language models. His work addresses a problem that has become increasingly urgent as AI systems move from isolated tasks into environments populated by other agents, human users, institutions and automated decision-makers. In those settings, intelligence is not only about predicting the world. It is also about anticipating how other decision-makers will respond.
One part of Golowich’s thesis examines situations in which multiple AI agents learn simultaneously while pursuing their own interests. Poker provides an intuitive example because players must make decisions with hidden information, limited observations and opponents whose behavior changes over time. Similar strategic interactions arise in online auctions, financial markets, automated negotiation, cybersecurity and decentralized networks. If each agent adjusts its behavior independently, the resulting system can oscillate indefinitely rather than settle into a predictable outcome. A strategy profile is considered an equilibrium when no single player can gain by unilaterally switching strategies. Finding such a point is difficult because every agent is learning against a moving target: the environment changes precisely because the other agents are learning too.
Golowich and his collaborators studied a family of algorithms based on Multiplicative Weights, a powerful method for repeatedly choosing among competing actions. In its basic form, the algorithm increases the probability of actions that perform well and decreases the probability of actions that perform poorly. The “optimistic” version adds a prediction about the next round’s result, allowing an agent to use information about expected future feedback rather than reacting only after an outcome has occurred. In strategic games, that extra predictive step can reduce the back-and-forth behavior that often slows learning. The researchers showed that when agents use Optimistic Multiplicative Weights, their collective behavior can approach equilibrium substantially faster than earlier theoretical analyses suggested. The result provides a mathematical explanation for why algorithms with a degree of anticipation can stabilize competition more efficiently than purely reactive methods.
The importance of this finding extends beyond the abstract question of whether a game eventually reaches equilibrium. The rate of convergence determines whether a method is usable in practice. In a small game, an algorithm may be allowed millions of rounds to learn, but a real market, auction or negotiation system may have only a limited number of interactions before decisions must be made. Faster convergence can reduce the amount of data and computation required before the agents behave predictably. It can also make the system less vulnerable to unstable feedback loops, in which one agent’s adjustment provokes another adjustment and the entire population continually swings between competing strategies. By improving the theoretical guarantees for equilibrium learning, Golowich’s work helps connect the elegant mathematics of game theory with the demands of large-scale AI systems.
A second major theme of the thesis concerns reinforcement learning, the approach through which an individual agent learns by taking actions, receiving feedback and gradually improving its policy. A robot entering an unfamiliar building, for example, must decide whether to exploit routes it already knows or explore unknown corridors that might lead to a better destination. The problem becomes dramatically harder as the number of possible states expands. An AI system may need to distinguish among countless combinations of locations, observations, actions and past events, while each experiment consumes time, energy or computing resources. Golowich’s theoretical work examines how an agent can explore efficiently, selecting actions that provide not only immediate rewards but also valuable information about the environment. These results are relevant to language models as well, which must learn from sequences of interactions and determine which forms of feedback reveal the most about how to act effectively.
The thesis also considers partially observed environments, in which the agent cannot directly see the complete state of the world. A physician may have to make a treatment decision without a full patient history, while an autonomous vehicle may need to act despite noisy cameras, blocked sensors or uncertain information about nearby traffic. In mathematical terms, the agent must maintain a belief about several possible underlying states and update that belief as new observations arrive. This creates a difficult combination of decision-making and inference: the system must determine what is happening while simultaneously choosing what to do. Golowich identified a method for finding a near-optimal strategy under these constraints and proved that its performance is essentially the best any algorithm could achieve in the same setting. Such lower-bound results are important because they show not merely that a technique works, but that substantial further improvement is impossible without additional assumptions or information.
The broader significance of these findings lies in their attempt to replace trial-and-error explanations of AI with precise guarantees. Modern machine-learning systems can produce striking results even when researchers do not fully understand why a particular training procedure succeeds. Golowich has argued that theoretical analysis can reveal the mechanisms hidden beneath that empirical success. In game-theoretic learning, the analysis clarifies how prediction changes the speed of convergence. In reinforcement learning, it identifies the cost of exploration and the limits imposed by partial information. These insights may eventually guide the design of more reliable systems, particularly AI agents that must operate in open-ended environments rather than respond to a fixed collection of examples.
Golowich said his thesis became broader as he encountered new problems and collaborated with researchers across the Hertz community. During his graduate work, he worked with fellow Hertz Fellows including Moitra and Robert Kleinberg, and he credited the Hertz Fellowship with giving him the freedom to pursue questions without forcing them into a predetermined plan. The Hertz Thesis Prize recognizes doctoral research judged to be exemplary, transformative and connected to real-world applications. Golowich joins more than 60 previous recipients. The 2025 committee also awarded honorable mentions to Alex Cohen and Nina Zubrilina. Cohen, who also earned his graduate degree at MIT, was recognized for work on higher-dimensional fractal uncertainty in harmonic analysis, while Zubrilina, a Princeton graduate, was honored for her study of convergence and correlations among coefficients of cusp forms in number theory.
At Austin, Golowich’s research will focus on understanding how generative AI systems, including language models, learn and make decisions. The engineering capabilities of these systems have advanced rapidly, but their internal behavior remains difficult to characterize with the same precision used in established areas of mathematics and computer science. The theoretical questions raised by his thesis could become increasingly important as language models act as autonomous agents, negotiate with one another, use external tools and make decisions under uncertainty. The central challenge is no longer simply whether an AI can produce an impressive answer. It is whether researchers can establish when its strategy will remain stable, how efficiently it learned that strategy, what information it lacks and whether any better method is possible. Golowich’s work offers a framework for asking—and beginning to answer—those questions.
Subject of Research: Theoretical foundations of learning in games, equilibrium computation, reinforcement learning, efficient exploration and decision-making under partial information.
Article Title: Noah Golowich’s Theory Explains How AI Can Learn Stable Strategies in Games and Unfamiliar Worlds
Web References: https://www.hertzfoundation.org/people/noah-golowich/; https://www.hertzfoundation.org/people/ankur-moitra/; https://www.hertzfoundation.org/people/robert-kleinberg/; http://hertzfoundation.org/hertz-community/awards-recognition/hertz-thesis-prize/; http://hertzfoundation.org/
References: Noah Golowich, “Theoretical Foundations for Learning in Games and Dynamic Environments”; Fannie and John Hertz Foundation, 2025 Hertz Thesis Prize announcement.
Keywords
Artificial intelligence, game theory, Nash equilibrium, reinforcement learning, Optimistic Multiplicative Weights, multi-agent learning, computational equilibrium, exploration, partial observability, large language models, theoretical computer science, machine learning, AI research
