AI Learns to Untangle the World’s Container Mountains

Every year, billions of tonnes of goods cross the oceans in steel boxes, and when those containers reach a port, they must be stacked, stored, and retrieved with almost military precision. Behind the scenes of every busy terminal lies a puzzle that has frustrated operations researchers for decades: the Container Relocation Problem. Containers arrive in a yard bay and are stacked in tiers, but the container a crane needs next may be buried under several others. Every extra lift needed to reach a buried box is called a rehandling operation, and each one costs time, fuel, and money. A new study published in Applied Intelligence by Zizhen Zhang and Fengwei Liu of Sun Yat-sen University, together with Mengchu Zhou of the New Jersey Institute of Technology, shows that a deep reinforcement learning system can learn to solve this puzzle better than traditional methods, and can even adapt on the fly when new containers keep arriving.

The classic version of the problem assumes the yard bay starts with a fixed set of containers and a list of retrieval requests. The goal is to find the sequence of moves that retrieves every container in the required order while performing the fewest possible rehandlings. The problem is computationally hard; researchers have spent years building exact algorithms based on branch and bound, integer programming formulations, and carefully engineered heuristics that encode expert knowledge about which containers are safe to place on top of others. The new work takes a different route. Instead of hand-crafting rules, the authors let a neural network learn a policy, a mapping from the current state of the yard to the next move, trained purely by trial, error, and reward.

What makes the study technically interesting is its treatment of dynamics. In real terminals, the yard is not a frozen snapshot. New containers arrive continuously, and each arrival can change which moves make sense. This dynamic version, the Dynamic Container Relocation Problem, forces the retrieval sequence to be rescheduled as the layout evolves. Static solvers struggle here because a plan that was optimal a moment ago can become obsolete the instant a new box lands on the bay. The authors therefore built a single policy network that handles both the static and dynamic cases, rather than training separate models for each scenario.

At the heart of the architecture is a novel dynamic attention mechanism. Attention layers, made famous by the transformer family of models, allow a network to weigh the relevance of different parts of its input when making a decision. Here, the input is the container yard itself: every stack, every tier, and every container with its retrieval priority. The researchers combined multi-head attention with long short-term memory, or LSTM, layers. The attention heads capture spatial dependencies, the relationships among containers across the stacks at a given moment, while the LSTM captures temporal dependencies, how the layout has changed over the sequence of moves so far. This fusion lets the model adaptively re-evaluate the yard after every single action, which is exactly what the dynamic problem demands.

Training relies on policy gradient optimization, a family of reinforcement learning techniques that traces back to Williams’ REINFORCE algorithm from 1992. In essence, the network proposes moves, the environment responds with a new yard configuration, and the total number of rehandling operations serves as the feedback signal. Moves that lead to fewer rehandlings are reinforced; costly moves are discouraged. Over many training episodes, the network learns to balance immediate placement decisions against long-term retrieval efficiency. A seemingly cheap move now, such as dropping a container onto a convenient stack, may create a costly blockage later, and the policy gradient method teaches the network to see that far ahead without any domain-specific heuristics being written into it.

The significance of this approach becomes clearer when set against the history of the field. Exact algorithms from researchers such as Caserta, Schwarze, and Voß, Tanaka and Mizuno, and Quispe, Lintzmayer, and Xavier have pushed the boundaries of what can be solved optimally, and improved mathematical formulations and branch-and-price methods have refined the lower bounds that certify optimality. But exact methods scale poorly as bays grow larger, and heuristics such as Kim and Hong’s relocation rule or the ant colony optimization of Jovanovic and colleagues depend on assumptions that may not hold in every terminal. Machine learning has entered the arena before, with studies applying deep learning assisted tree search to container pre-marshalling and reinforcement learning to relocation operations, but the new paper’s unified treatment of static and dynamic variants under one attention-based policy is a notable step forward.

The experiments reported in the study show that the learned policy outperforms traditional methods on both the classical Container Relocation Problem and its dynamic counterpart. The benchmark instances draw on widely used problem sets, including those published by Caserta and colleagues in 2012 and the instance library maintained by Tanaka, which have served as the standard testing grounds for relocation algorithms for over a decade. Beating established baselines on these instances without embedding expert knowledge is a strong result, because it suggests the network discovered useful strategies on its own, strategies that human experts spent years articulating as rules.

There is also a broader story here about the rise of neural combinatorial optimization. The lineage runs from AlphaGo’s demonstration that deep networks and search can master combinatorial games, through neural solvers for the traveling salesman and vehicle routing problems, to attention-based models that learn to construct solutions step by step. Recent work has explored multi-task learning, mixture-of-experts solvers, meta-learning for scale shifts, and universal neural solvers that generalize across problem families. The container relocation study adds a distinctive twist: the state is not a graph of cities or customers but a physical stack yard whose geometry changes with every crane move, and the dynamics are not just stochastic demand but the literal arrival of new objects into the problem space.

For port operators, the practical implications are considerable. Rehandling operations are a major source of cost and delay in terminal operations, and the dynamic version of the problem mirrors what crane drivers actually face: retrieval requests that must be served while the yard keeps changing. A policy network that can reschedule rapidly as containers arrive could be integrated into terminal operating systems, offering decision support that updates in real time. Because the approach does not rely on extensive expert knowledge, it could be adapted to terminals with different layouts, stacking heights, and arrival patterns without teams of specialists rewriting heuristics for each site. The authors note that their experiments highlight the potential of deep reinforcement learning for solving relocation problems without that heavy dependence on domain expertise.

The research was supported by the National Natural Science Foundation of China and the Guangdong Basic and Applied Basic Research Foundation, and it arrives at a moment when ports worldwide are under pressure to handle ever-larger vessels and tighter turnaround windows. As artificial intelligence continues to colonize the messy, physical corners of operations research, the image of a neural network staring at a container yard, weighing which box to lift next, captures something genuinely new: machines learning the tacit judgment of crane planners, one rehandling operation at a time. If the approach matures from benchmark bays to live terminals, the humble stack of steel boxes may become one of the clearest showcases of reinforcement learning in the physical economy.

Subject of Research: Deep reinforcement learning with dynamic attention tackles static and dynamic container relocation problems in port terminals

Article Title: Dynamic attention for container relocation problems: a deep reinforcement learning approach

Article References: Zhang, Z., Liu, F., & Zhou, M. (2026). Dynamic attention for container relocation problems: a deep reinforcement learning approach. Applied Intelligence, 56(16), Article 500. https://doi.org/10.1007/s10489-026-07525-2

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07525-2

Keywords: deep reinforcement learning, container relocation problem, dynamic attention, port logistics, combinatorial optimization, LSTM, multi-head attention, policy gradient, Dynamic, attention, container, relocation