The core problem the researchers set out to solve is well known to anyone who has trained a multilayer perceptron. There is no analytical formula that tells a designer what the optimal architecture for a given dataset looks like, so practitioners fall back on rules of thumb, trial and error, or exhaustive grid searches. Even once an architecture is fixed, the standard training procedure, backpropagation of errors, relies on gradient descent, which is vulnerable to getting trapped in local minima of the error surface and can converge painfully slowly on difficult problems. These two weaknesses compound each other: a poorly chosen architecture can make the training landscape even harder to navigate, while a bad training run can make a good architecture look useless.

Neuroevolution, the field that applies evolutionary computation to neural network design, has spent more than three decades trying to break this deadlock. Landmark work such as Stanley and Miikkulainen’s NEAT, which evolved network topologies through augmenting them over generations, demonstrated that architectures themselves can be treated as evolvable objects. Since then, a long line of metaheuristics, including genetic algorithms, particle swarm optimization, grey wolf optimizers, whale optimizers, and artificial bee colonies, has been applied to everything from selecting hidden neurons to tuning connection weights. Reviews of this literature, including Ojha, Abraham, and Snášel’s survey of two decades of metaheuristic design of feedforward networks, note a persistent fragmentation: most methods optimize only one or two components of the network, leaving the rest to human judgment or to a separate training algorithm.

NN-IMODE takes aim at that fragmentation. It is built on differential evolution, a population-based optimizer introduced by Storn and Price in 1997 that evolves candidate solutions by generating mutant vectors from weighted differences of existing population members and crossing them with parent solutions. What makes the new approach distinctive is its improved multi-operator structure, drawing on the IMODE framework developed by Karam M. Sallam and colleagues at the 2020 IEEE Congress on Evolutionary Computation. Rather than committing to a single mutation strategy, the algorithm deploys several operators in parallel, each maintaining its own subpopulation, and lets the search process itself determine which operators earn a larger share of the population. This division of labor allows the algorithm to balance broad exploration of the search space against fine-grained exploitation of promising regions, a balance that the authors have studied extensively in their earlier work on metaheuristic behavior.

The technical heart of the method is a novel solution representation. Because NN-IMODE must encode discrete architectural choices, such as the number of hidden layers and the count of neurons in each layer, alongside continuous parameters, such as real-valued weights and biases, and categorical choices, such as which activation function each layer applies, a conventional fixed-length numeric vector will not suffice. The researchers designed an encoding that captures all of these elements within a single evolvable structure, so that a differential evolution operator acting on two candidate networks can blend not only their weights but also their architectural genes. This means that a network with two hidden layers and sigmoid activations can, in effect, breed with a network that has three hidden layers and rectified linear units, producing offspring that inherit and recombine traits from both parents.

Evaluating such a method requires a demanding experimental protocol, and the team put NN-IMODE through one of the more comprehensive benchmarks in this literature. The algorithm was tested on 21 benchmark datasets drawn from the UCI Machine Learning Repository, a standard public collection spanning domains from medical diagnosis to image recognition. The comparison set was formidable: Levenberg-Marquardt, the second-order gradient method that remains a workhorse for small multilayer perceptrons; Scaled Conjugate Gradient, another fast gradient-based trainer; Genetic Algorithms; Differential Evolution in its classic form; and hybrid combinations of these methods. Across this gauntlet, NN-IMODE achieved the highest classification accuracy and the lowest test error rates on 18 of the 21 datasets, a result the authors describe as establishing the method as a transformative approach in neuroevolution.

The significance of beating Levenberg-Marquardt and Scaled Conjugate Gradient deserves unpacking. Gradient-based trainers are extremely efficient when they work, but their performance is hostage to the error surface of the architecture they are given. By searching architectures and parameters jointly, NN-IMODE effectively reshapes the landscape it is searching, discarding architectures whose error surfaces are pathological and concentrating its evolutionary effort on network shapes that train well. The multi-operator mechanism adds a second layer of robustness, because no single mutation rule has to succeed on every dataset; operators that happen to suit a given problem’s structure are automatically amplified. This adaptivity is likely what allows the method to generalize across such a heterogeneous collection of datasets rather than excelling only on problems of one particular type.

The work also arrives at a moment of renewed interest in automated neural network design. Neural architecture search has produced striking results in deep learning, but its most celebrated successes rely on enormous computational budgets that put them out of reach for most laboratories. Metaheuristic approaches such as NN-IMODE occupy a complementary niche: they target feedforward networks of the kind used for tabular classification and regression, where a compact, well-tuned multilayer perceptron can still match or beat far larger models, and where the cost of an evolutionary search is measured in hours rather than weeks. The authors’ broader research program, which includes comparative studies of exploration-exploitation balance, initialization methods, chaos-enhanced metaheuristics, and pruning strategies for convolutional networks, positions this paper as the culmination of a systematic effort to understand how evolutionary search behaves on machine learning design problems.

Several caveats temper the enthusiasm. The benchmark datasets, while diverse, are mostly small to medium in scale, and the computational cost of evaluating a population of candidate networks, each of which must be trained and tested, remains substantial compared with a single gradient descent run. The study also reports no external funding, and the authors declare no competing interests, which speaks to its independence but also to the limits of the experimental scale. Whether the multi-operator machinery retains its advantage on very high-dimensional problems, or on architectures with many more layers than typical feedforward networks, is a question for future work. The authors suggest that the method paves the way for broader applications across artificial intelligence and machine learning, and the generality of the encoding scheme gives that claim some plausibility.

Still, the headline result is hard to dismiss. A single algorithm that decides how deep a network should be, how wide, which nonlinearities its neurons should apply, and what every weight and bias should be, and that does so well enough to outperform established training methods on most of two dozen datasets, is a meaningful step toward neural networks that design themselves. For decades, the separation between architecture design and parameter training has been treated as a natural division of labor between human engineers and learning algorithms. NN-IMODE suggests that the division is not a necessity but a habit, and that evolution, given the right representation and enough operators to work with, can handle the whole job at once. As automated machine learning continues its advance, methods of this kind may become the default way that everyday predictive models are built, with human designers specifying only the goal and the data.

Subject of Research: Simultaneous optimization of feedforward neural network architectures and parameters using an improved multi-operator differential evolution algorithm

Article Title: NN-IMODE: a novel approach for simultaneous optimization of architecture and parameters in feedforward neural networks

Article References: Ferhat, A., Zitouni, F., Sallam, K. M., Harous, S., Lakbichi, R., & Limane, A. (2026). NN-IMODE: a novel approach for simultaneous optimization of architecture and parameters in feedforward neural networks. Cluster Computing, 29(13), Article 756. https://doi.org/10.1007/s10586-026-06551-6

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06551-6

Keywords: feedforward neural networks, differential evolution, neuroevolution, metaheuristics, neural network optimization, evolutionary computation, classification accuracy, multi-operator algorithms, UCI benchmark datasets, automated machine learning, backpropagation, architecture search