In a study published in the journal Quantum Machine Intelligence, Oliver Sefrin of the German Aerospace Center (DLR) in Ulm, together with Manuel Radons and Lars Simon of Bundesdruckerei GmbH in Berlin and Sabine Wölk of DLR and the Center for Integrated Quantum Science and Technology at Ulm University, report the first systematic investigation of a hybrid quantum-classical reinforcement learning agent in non-stationary environments. The work, published as an open-access research article in Volume 8 of the journal, marks a departure from earlier studies of the so-called hybrid agent for quantum-accessible reinforcement learning, which had until now been tested only in stationary settings where the rules of the game never change.

Reinforcement learning, or RL, is the branch of machine learning in which an agent learns by trial and error, interacting with an environment that is formally described as a Markov decision process. The agent observes states, takes actions, collects rewards, and gradually adjusts its policy — the probability with which it chooses each action in each state — to maximize cumulative reward. From mastering Atari games and Go to fine-tuning today’s large language models through reinforcement learning from human feedback, the paradigm has become one of the most consequential tools in artificial intelligence. But in realistic scenarios, from traffic management to industrial control, the environment itself moves: lanes close, demand fluctuates, targets relocate. A policy that was optimal yesterday may be worthless today, and well-trained agents often fare worse after a sudden change than untrained ones, because what they have learned actively works against them.

The hybrid learning agent at the center of the new study, introduced in earlier work by Hamann and Wölk, offers a provable speed advantage in a special class of learning problems. It combines a classical RL algorithm with Grover-type amplitude amplification, a quantum technique that boosts the probability of sampling rewarded action sequences quadratically faster than random classical sampling. In the quantum part of the algorithm, actions and percepts are encoded in orthonormal quantum states, and the environment’s response is modeled as a unitary that produces a phase-kickback oracle marking rewarded action sequences with a sign flip. Applying Grover iterations to a superposition of action sequences weighted by the agent’s current policy raises the chance of measuring a rewarded sequence according to the well-known sine-squared amplification law, and the resulting candidate is then verified in a classical episode of play. The payoff is a quadratic reduction in the number of episodes needed to find rewarded behavior — but only for deterministic, strictly episodic tasks, such as the classic Gridworld maze, where the episode length is fixed and transitions are deterministic.

The catch, the researchers recognized, is that speed can become a liability. “In such dynamic learning scenarios, a well-trained agent often performs worse after a sudden change of the environment compared to an untrained or a barely trained agent,” the authors note in the paper. A fast-learning quantum agent that has converged hard onto one reward structure can find itself trapped, its confident policy pointing away from the new source of reward. To be competitive outside the laboratory, the hybrid agent needed a way to forget.

The team’s solution is elegantly simple: they paired the hybrid agent with Projective Simulation, a tabular RL algorithm developed by Hans Briegel and Gorka De las Cuevas, which stores experience in a network of clips connected by weighted edges. Projective Simulation includes a natural dissipation mechanism — a forgetting parameter that continuously pulls the edge weights back toward their initial values, driving the policy gradually toward uniform exploration. Two modifications were required to make the hybrid version work in a changing world. First, because executing k rounds of amplitude amplification consumes 2k+1 interaction episodes, the quantum agent updates its policy less frequently than a classical one; the researchers derived an equivalent update rule that scales the dissipation strength so that the hybrid agent forgets at the same rate as its classical counterpart, and they proved the equivalence formally by induction. Second, the agent’s internal lower-bound estimate of its success probability — needed to calibrate the number of Grover iterations — had to be maintained adaptively, purging previously rewarded action sequences from memory whenever they were discovered to be no longer rewarded after an environment change.

The testbed was a modified Gridworld with a moving target. In the first scenario, the reward moves along a fixed path within each episode, synchronously with the agent’s own steps, adding time dependence within episodes while keeping the overall reward structure stable. In the second, more demanding scenario, the reward path itself is swapped for a new, fully disjoint path partway through training — a hidden-mode Markov decision process in which the agent knows a switch may occur but is never told when. Crucially, the researchers designed the second layout so that the two reward paths share no rewarded action sequences at all, making it the harshest possible test: excellence on the first path maximally damages performance on the second.

The results, averaged over hundreds of simulated runs, are striking. In the fixed-path scenario, the hybrid agent found its first reward in an average of 17.9 episodes, compared with 58.4 episodes for classical Projective Simulation — better even than the theoretical quadratic-speedup upper bound of 34.4 episodes — and after just 100 training episodes it had already reached an average success probability of 99.3 percent, while the classical agent lagged at 69.3 percent. Classical Q-Learning, with its one-step back-propagation of reward information, needed far longer for its values to percolate back to the starting state.

The changing-path scenario delivered the study’s most interesting twist. Immediately after the reward path switched, the hybrid agent’s success probability collapsed by roughly an order of magnitude below that of the classical agent — exactly the vulnerability the researchers had anticipated. But the forgetting mechanism did its work. The hybrid agent’s curve then climbed steeply, converging to nearly 100 percent success, while classical Projective Simulation rose more gradually to 98.4 percent and Q-Learning, hampered by its fixed exploration rate, plateaued between 60 and 80 percent. With a dissipation value of 0.05, the hybrid agent achieved the shortest total learning time in the benchmark — about 113 episodes against 149 for Q-Learning and 187 for the best classical Projective Simulation configuration. The researchers also showed that their adaptive estimate of the success probability tracks the true value closely, overshooting only marginally in the first few episodes after a switch thanks to the purging mechanism.

“Given that the problem scenario in this article is chosen such that the hybrid agent’s initial good performance puts it at a maximal disadvantage after a change in the reward function, we expect the results of this toy problem to generalize well to a wider range of problem cases,” the authors write. They go further, conjecturing that the quadratic speedup in learning may itself be what enables faster adaptation — potentially opening the door to learning in fast-changing environments where a classical agent would be too slow to learn at all.

The work sits within the broader and still-uncertain landscape of quantum reinforcement learning. Much current research in the noisy intermediate-scale quantum era relies on variational quantum circuits as function approximators, but whether those approaches can deliver genuine quantum advantage remains an open question. The hybrid agent, by contrast, carries a proven speedup and has already been demonstrated in a proof-of-principle experiment on a nanophotonic processor, suggesting it may be a serious contender for the post-NISQ era. The authors caution that their dissipation mechanism is deliberately passive — it does not detect changes but absorbs them — and that combining the hybrid agent with continual-learning methods that prevent catastrophic forgetting, or extending the study to time-dependent transition dynamics, are natural next steps. They also call for standardized continual-RL benchmarks, perhaps as extensions of the popular Gymnasium package, to put such comparisons on firmer footing.

For a field often accused of chasing speedups on toy problems that never change, the message of this study is pointed: quantum-accelerated learners can not only learn faster, but also unlearn faster. In a world where the only constant is change, that may be the advantage that matters.

The hippocampus has long been recognized as a critical brain structure for spatial memory and navigation. One of the hallmark features of hippocampal activity during exploration is the presence of theta rhythms—rapid oscillations that organize neuronal firing into temporal sequences. These theta sequences are thought to represent coherent spatial sweeps of activity that effectively “simulate” future movements. Until now, most research had focused on experience-independent theta sequences that toggle between left and right paths, providing localized spatial sampling. However, their role in more complex and goal-directed planning remained elusive.

Tang and colleagues tackled this question by employing an open arena navigation task with rats trained to reach distant remembered goals. Unlike classic maze tests that limit the animal to predefined routes, this environment demanded flexible planning and continuous decision-making. Electrodes implanted in the hippocampus and prefrontal cortex recorded neural patterns while animals navigated toward spatial objectives. Through meticulous analysis, the research team identified a distinct class of theta sweeps that were strongly modulated by the animal’s specific goals and prior experience, distinguishing them from previously characterized randomized sweeps.

These goal-directed theta sweeps manifested as neuronal firing sequences that predicted upcoming trajectories toward remembered goals well before actual movement initiation. Beyond the hippocampus, coordinated activity with the prefrontal cortex suggested a distributed neural network integrating memory and executive functions to guide navigation. The authors observed that these sequences rapidly updated in accordance with changes in goal location, demonstrating an adaptive mechanism sensitive to current behavioral demands and environmental contingencies.

Importantly, these goal-directed theta sweeps were not isolated phenomena—they frequently coincided with sharp-wave ripple events during periods of immobility. Sharp-wave ripples have been implicated in memory consolidation and replay processes, but here they also appeared linked to the evaluation and reinforcement of planned trajectories. This coupling suggests a sophisticated dialogue between spontaneous replay and active navigation mechanisms, forming a continuous feedback loop critical for memory-guided decision making.

At a cellular and circuit level, the study offers an intriguing mechanistic model. The generation of goal-directed theta sweeps depends on integrating egocentric goal-direction signals with inhibitory feedback control within the hippocampal network. The authors propose that a reduction in feedback inhibition permits the selective amplification of neuronal ensembles representing learned goals. This disinhibition effectively biases theta sequences toward task-relevant spatial trajectories, enabling the animal to internally simulate feasible paths to desired targets.

The findings carry profound implications for how brains solve the perennial problem of path planning in complex three-dimensional environments. The flexible generation of theta sweeps tailored to specific behavioral goals provides a neural code capable of supporting real-time navigation and flexible strategy switching. This flexibility distinguishes the goal-directed sequences from more stereotyped, experience-independent sequences, suggesting an evolutionary advantage of modulating internal representations based on learned priorities.

Moreover, the coordination between hippocampal and prefrontal circuits highlights the importance of cross-regional communication in translating memory traces into actionable plans. The prefrontal cortex’s known involvement in working memory, attention, and decision conflict aligns well with its observed synchronization with hippocampal theta patterns during navigation. This interface likely enables context-dependent modulation of spatial representations, injecting behavioral relevance directly into hippocampal computations.

The study also prompts intriguing questions about how such mechanisms might extend beyond rodents into human cognition. Given the conservation of hippocampal-prefrontal pathways across mammals, similar theta-mediated predictive sequences may underpin complex human behaviors such as wayfinding, episodic memory retrieval, and prospective planning. Dysfunction in these systems could offer insights into disorders characterized by spatial disorientation and impaired executive function, including Alzheimer’s disease and schizophrenia.

Technologically, this research advances the methodological frontier by combining high-density electrophysiological recordings with sophisticated computational analyses capable of dissecting fine-grained temporal sequences within ongoing brain rhythms. The identification of goal-directed theta sequences required not only neural data acquisition but also innovative analytic frameworks attuned to the dynamic and context-dependent nature of neural activity patterns.

In essence, Tang et al.’s work bridges a crucial gap between the behavioral phenomena of flexible navigation and the underlying neural dynamics. By elucidating a learning-dependent form of theta sequence organization that selectively probes future paths toward remembered goals, this study redefines our understanding of spatial cognition’s neural architecture. It paves the way for future explorations into how memories shape imagined futures and guide purposeful actions in an ever-changing world.

As the field moves forward, potential applications might include developing neural-inspired algorithms for autonomous robotic navigation, incorporating biologically grounded models of flexible spatial reasoning. Furthermore, targeted interventions aimed at modulating theta dynamics could emerge as therapeutic strategies to restore spatial awareness and memory in clinical populations.

Ultimately, this discovery resonates broadly beyond neuroscience, touching on fundamental questions regarding how brains create internal maps not merely to reflect the environment but to imagine and evaluate potential futures. The brain’s exquisite capacity to internally replay and project paths toward desired outcomes embodies the essence of intelligent behavior—melding past experience with present needs to navigate an uncertain world effectively and efficiently.

This research thus unlocks a new dimension of understanding regarding the neural bases of planning, memory, and navigation—offering a vivid glimpse into the rhythms through which the brain choreographs the dance between memory and movement, past and future, knowledge and action.

Subject of Research: Neural mechanisms of goal-directed navigation and hippocampal theta sequences.

Article Title: Goal-directed hippocampal theta sweeps during memory-guided navigation.

Article References:
Tang, W., Mei, X., Harvey, R.E. et al. Goal-directed hippocampal theta sweeps during memory-guided navigation. Nat Neurosci (2026). https://doi.org/10.1038/s41593-026-02364-3

Image Credits: AI Generated

DOI: https://doi.org/10.1038/s41593-026-02364-3