Reinforcement Learning Coordinates Three Swarm Algorithms to Improve Continuous Optimization

A Reinforcement-Learning System Lets Three Nature-Inspired Algorithms Cooperate to Tackle Difficult Optimization Problems

Many of the hardest problems in engineering and computing share an apparently simple question: among an enormous number of possible choices, which one is best? Designing a lightweight structure, tuning the settings of an industrial process, planning a delivery route, or allocating resources across a supply chain can all require searching through landscapes so vast that examining every possibility is impossible. A new study reports a computational strategy that allows three widely used “nature-inspired” optimization methods to work together, with reinforcement learning deciding which approach should lead at each stage. In tests involving dozens of standard mathematical challenges, the coordinated system achieved the desired accuracy on every run for most of the problems considered, according to its developers.

The research, published in Knowledge and Information Systems, combines particle swarm optimization, the grey wolf optimizer and the artificial bee colony algorithm. These methods are known as metaheuristics: general-purpose procedures that search for good solutions without requiring a problem to have a simple mathematical structure. They do not guarantee the perfect answer in every situation, but they can be effective when conventional methods become impractical. The three algorithms draw inspiration from different collective behaviors. Particle swarm optimization models a group of particles moving through a search space, grey wolf optimization imitates a pack encircling and pursuing prey, and artificial bee colony optimization represents bees exploring food sources and sharing information about promising locations.

Each algorithm has characteristic strengths and weaknesses. Particle swarm optimization, or PSO, moves candidate solutions according to combinations of their own previous experience and the best position found by the swarm. This can make it fast at exploiting a promising region, but the swarm may converge too early around a mediocre solution. In mathematical optimization, “exploitation” means intensifying the search near an apparently good answer, while “exploration” means probing less familiar regions in the hope of finding something better. A method that explores too little can become trapped in a local optimum—a solution that is better than nearby alternatives but inferior to the global best. One that explores too much may fail to refine a strong candidate efficiently.

The grey wolf optimizer, or GWO, uses a hierarchy of candidate solutions and mathematical rules that simulate the hunting behavior of grey wolves. The best solutions act as leaders, guiding the rest of the population as it contracts around likely targets. The artificial bee colony, or ABC, divides its search behavior among employed bees, onlooker bees and scouts. Employed bees investigate known food sources, onlookers favor sources judged more attractive, and scouts abandon unproductive locations to search elsewhere. Although the biological metaphors are different, all three algorithms repeatedly update a population of possible answers. Their performance can vary substantially from one problem to another, because the shape of the objective function—the numerical measure being minimized or maximized—determines whether aggressive local refinement or broader wandering is more valuable.

Rather than selecting a single winner in advance, the researchers created modified versions of two methods and organized all six algorithms into three families: PSO and modified PSO, GWO and an extended GWO, and ABC and modified ABC. The redesigned particle swarm and bee-colony variants incorporate additional perturbation rules, intended to disturb candidate solutions and increase the chance of escaping stagnation. The grey-wolf family includes ExGWO, a previously developed extension used in the new coordination framework. These modifications give the system more than three choices: at each decision point, it can select among three sets of related search behaviors, each containing an original algorithm and a strengthened variant.

The selection mechanism is reinforcement learning, a branch of machine learning in which an agent learns by taking actions and receiving rewards. Here, the agent’s action is to choose one of the three algorithm families for the next phase of optimization. After the selected family is used, the system evaluates whether the resulting search produced an improvement. A reward table records the outcomes, and a Q-table is updated as a running performance indicator. In reinforcement learning, a Q-value estimates how useful an action is under a particular state or situation; higher values signal choices expected to produce greater future reward. In this study, the table is not learning a physical law or a language pattern. It is learning which search family appears most effective for the current optimization problem.

The approach is designed to adapt over time. During early iterations, the system gathers evidence about how the three families behave on the target function. After several iterations, the Q-table begins to favor the family that has produced the most useful improvements, and that family is selected for subsequent iterations. The process can therefore shift the balance between exploration and exploitation without relying entirely on a fixed schedule chosen by the programmer. A method that performs well at the beginning may be replaced if it stops making progress, while a previously less successful family may receive greater attention when the search enters a different region of the solution landscape. This makes the framework a hyper-heuristic: rather than directly solving the optimization problem with one heuristic, it learns how to coordinate multiple heuristics.

The researchers evaluated the system across 50 benchmark instances spanning different categories of single-objective continuous optimization problems. They also tested it on 30 functions from the CEC2017 benchmark suite and 12 functions from CEC2022. Such benchmark suites are widely used to compare optimization algorithms because they include functions with different levels of difficulty, dimensionality, multimodality and interaction among variables. A multimodal function, for example, may contain many local optima, making premature convergence particularly dangerous. Continuous optimization differs from discrete optimization in that decision variables can take values across ranges rather than being restricted to separate choices, although practical engineering problems may later impose boundaries or other constraints.

According to the study, the coordinated method achieved a 100 percent success rate with the desired accuracy on most of the tested instances. The authors also report that statistical comparisons established its superiority over a set of contemporary heuristics and hyper-heuristics for the benchmark problems. The result is notable because the system’s central advantage is not a new search rule alone, but the ability to select among existing rules according to observed performance. Still, benchmark success does not automatically translate into superiority for every real-world application. The tests concern mathematical functions, and the computational cost of maintaining reward and Q-tables, running multiple algorithmic variants and tuning their parameters may differ across applications. The study also states that no datasets were generated or analyzed, consistent with an evaluation based on benchmark functions rather than empirical measurements from a physical system.

The work highlights a broader movement in optimization research: algorithms inspired by swarms, animals or evolutionary processes are increasingly being paired with learning systems that can regulate their behavior. The attraction is practical. No single metaheuristic consistently performs best across all problem landscapes, and manually choosing an algorithm or switching strategy can require specialist knowledge. A reinforcement-learning controller offers a way to make those choices adaptively, using feedback from the search itself. If the idea proves robust beyond standardized functions, it could help computational systems adjust automatically while optimizing complex designs, schedules or control settings. For now, the study provides evidence that PSO, GWO and ABC need not compete as isolated methods. Guided by a learned performance table, they can become a coordinated team—one that changes tactics as the mathematical terrain changes.