Tuesday, November 17, 2026
Science
No Result
View All Result
  • Login
  • HOME
  • SCIENCE NEWS
  • CONTACT US
  • HOME
  • SCIENCE NEWS
  • CONTACT US
No Result
View All Result
Scienmag
No Result
View All Result
Home Science News

Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability

November 17, 2026
in Science News
Gavin Prescott
By Gavin Prescott Scienmag Editorial Profile - Ecology and Ecosystem Dynamics
Reading Time: 6 mins read
0
Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability

Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability

65
SHARES
587
VIEWS
Share on FacebookShare on Twitter
ADVERTISEMENT

A comparative benchmark evaluating gradient-free reinforcement learning has revealed that neural network activation functions exert a decisive influence on the stability, convergence trajectories, and final reward outcomes of evolutionary algorithms. Conducted by researchers investigating NeuroEvolution on the Inverted Double Pendulum control task, the systematic study assessed twenty-four distinct algorithmic and structural configurations formed by crossing four evolutionary optimizers with six widely used activation functions. Across these controlled evaluations, the combination of Augmented Random Search paired with the Gaussian Error Linear Unit achieved the highest overall final test reward among all tested combinations. However, the experimental findings demonstrate that high final performance frequently coexists with severe training volatility, indicating that architectural non-linearities interact fundamentally with the exploration mechanics of evolutionary search rather than acting as interchangeable components.

NeuroEvolution offers an alternative optimization framework to traditional deep reinforcement learning methods that rely on backpropagation and gradient descent. In standard deep reinforcement learning, agent policies are parameterized by multi-layer artificial neural networks optimized through gradient-based estimators, which can be computationally demanding, highly sensitive to hyperparameter choices, difficult to parallelize across distributed systems, and vulnerable to numerical instability under non-linear system dynamics. By contrast, NeuroEvolution replaces gradient calculations entirely with population-based stochastic optimization algorithms inspired by biological evolution. Across successive generations, a population of candidate parameter vectors is evaluated directly within simulated environments based on cumulative episodic reward, bypassing backward passes and temporal-difference derivative propagation. While previous literature examined activation functions in gradient-based control or supervised neuroevolutionary architecture design, the researchers noted an absence of systematic studies examining how standard non-linearities shape evolutionary optimization dynamics in reinforcement learning tasks.

To rigorously isolate the interaction between evolutionary optimizers and network nonlinearities, the research team implemented a standardized experimental architecture using a fixed multilayer perceptron consisting of two hidden layers with 256 units each. The input layer mapped an eight-dimensional observation space describing the physical state of the benchmark system, while a single-neuron output layer governed by a hyperbolic tangent activation function ensured that continuous control commands remained constrained within normalized operational bounds. Across all twenty-four experimental configurations, the population size was fixed at 512 candidate policy networks evolved over 300 generations. Every generation involved evaluating all population members, while the best-performing policy was comprehensively re-evaluated every five generations using a parallelized evaluation batch of 128 simulated environments to monitor progress while dampening episode-level stochastic noise. Each individual configuration underwent five independent training runs to assess replicability under identical software and hardware conditions.

The benchmark chosen for all experiments was the continuous-control Inverted Double Pendulum task implemented via the Brax differentiable physics engine. The physical apparatus comprises a horizontally translating cart supporting two rigid links connected in series by rotational joints, requiring the control policy to maintain both pendulums in a vertical alignment against inherently unstable, highly non-linear coupled dynamics. The eight-dimensional observation vector provided to the policy network at each time step consists of the sine and cosine transformations of the first pole angle, the angular velocity of the first joint, the sine and cosine of the relative angle between the two poles, the angular velocity of the second joint, the cart position along the horizontal rail, and the linear cart velocity. Truncated trigonometric encodings avoid discontinuities at rotational boundary transitions, while initial states introduce uniform random perturbation noise between minus 0.01 and positive 0.01 to ensure non-trivial dynamic balance requirements from the onset of each simulation run.

System performance and policy fitness were quantified using a multi-component composite reward function designed to balance upright stability against excessive physical movement. At each simulation time step, the agent receives an alive bonus reward with a default magnitude of ten whenever the tip of the upper pendulum remains above a vertical threshold of one, below which the episode terminates immediately due to failure. Subtracted from this survival credit is a spatial distance penalty that penalizes horizontal cart displacement scaled by a factor of 0.01 alongside the squared vertical distance deviation of the upper pole tip relative to its fully upright target height of two. Additionally, a velocity penalty dampens rapid angular movement by penalizing the squared angular velocity of the lower joint weighted at 0.001 and the squared angular velocity of the upper joint weighted at 0.005, thereby rewarding precise stabilization while discouraging violent mechanical oscillations.

The study systematically investigated four evolutionary algorithms that incorporate fundamentally distinct search philosophies: NoiseReuseES, PersistentES, Policy Gradients with Parameter-Based Exploration, and Augmented Random Search. NoiseReuseES minimizes variance across individual fitness evaluations while maximizing execution efficiency by evaluating parameter perturbations across a shared set of random search directions reused among population members. PersistentES maintains long-lived, slowly evolving exploration trajectories over time to encourage sustained directional progress rather than reacting to volatile instantaneous disturbances. Policy Gradients with Parameter-Based Exploration establishes a probability distribution over the network parameter space, updating distribution means and variances using whole-episode return estimators to support coherent parameter exploration. Augmented Random Search operates as a lightweight, highly efficient optimizer that evaluates pairs of perturbations along random exploration directions and updates policy weights along vectors that yield superior reward performance.

These four search algorithms were paired across six representative activation functions displaying diverse mathematical properties: Rectified Linear Unit, Continuously Differentiable Exponential Linear Unit, Gaussian Error Linear Unit, Scaled Exponential Linear Unit, Sigmoid Linear Unit, and the hyperbolic tangent function. ReLU provides a standard piecewise linear baseline with zero output for negative inputs. CELU guarantees continuously differentiable transitions across positive and negative input domains. GELU softly gates values through probabilistic Gaussian scaling to provide enhanced non-linear expressiveness. SELU incorporates scaling parameters lambda of approximately 1.0507 and alpha of approximately 1.6733 to induce self-normalizing forward propagation dynamics that maintain stable intermediate means and variances. SiLU, also known as Swish, generates smooth, non-monotonic transformations via sigmoid multiplication, whereas tanh offers a traditional bounded and symmetric sigmoidal mapping. Entire computational pipelines were constructed in the JAX ecosystem using Flax, Evosax, Evojax, and Brax executed on standardized Kaggle GPU hardware.

The empirical results demonstrated pronounced algorithm-activation interaction effects that substantially dictated learning trajectories, convergence behavior, and terminal reward values. Augmented Random Search combined with GELU achieved the highest overall final test mean reward across the entire comparative suite, followed closely by PersistentES paired with GELU and PersistentES paired with ReLU. Competitive high-reward convergence was likewise observed in configurations utilizing Policy Gradients with Parameter-Based Exploration paired with SELU, CELU, and tanh, as well as Augmented Random Search paired with ReLU, and NoiseReuseES paired with GELU. However, strong terminal rewards were concentrated within a selective subset of pairings, whereas mismatched configurations suffered severe performance degradation. For example, PGPE combined with GELU dropped to a final mean reward of 5221, PGPE with SiLU achieved only 3147, PersistentES with SELU plummeted to 2947, ARS with CELU fell to 2394, and PersistentES with tanh converged to the lowest observed final reward of just 1863.

Numerical stability over the final 50 training iterations was evaluated using the coefficient of variation, computed as standard deviation divided by the mean multiplied by 100, where values beneath five percent indicated stable behavior, values between five and ten percent indicated moderate stability, and values exceeding ten percent denoted high instability. Under this metric and broader generational analysis, NoiseReuseES emerged as the most reliably stable optimizer across all tested non-linearities, exhibiting smooth, gradual upward trajectories free from catastrophic collapse, with SELU generating its smoothest and most consistent progression. While NoiseReuseES rarely captured the single highest peak score, its resilience prevented mid-training crashes across all conditions. Conversely, PersistentES displayed extreme volatility characterized by abrupt spikes and severe mid-training collapses that wiped out early progress under SELU, SiLU, tanh, and CELU, achieving partial stabilization exclusively when moderated by GELU.

Policy Gradients with Parameter-Based Exploration exhibited the highest sensitivity to the specific mathematical formulation of the activation function. When combined with CELU or tanh, PGPE followed smooth learning trajectories and achieved top-tier rewards, yet under GELU and particularly SiLU, it experienced severe oscillations, repeated surges, and degraded convergence. Augmented Random Search displayed a distinct, recurring generational pattern marked by rapid early improvement followed by substantial mid-training collapses across GELU, ReLU, tanh, SELU, and SiLU. In most cases, ARS demonstrated sufficient recovery capacity to achieve competitive terminal returns, with GELU yielding its least disruptive collapses and most orderly recovery, whereas under CELU it suffered an unrecoverable early collapse.

The authors emphasized that while these findings provide valuable empirical insight into structural compatibility in NeuroEvolution, the scope of the conclusions remains bounded by the specific experimental design. The comparative evaluations were conducted exclusively within a single benchmark environment, the Inverted Double Pendulum, utilizing a fixed two-hidden-layer proof-of-concept architecture without testing deeper or wider networks, alternative population scales, or additional tasks. Furthermore, the investigation deliberately excluded standard gradient-based deep reinforcement learning baselines such as Proximal Policy Optimization, Soft Actor-Critic, or Twin Delayed Deep Deterministic Policy Gradient, meaning the results do not establish superiority over conventional gradient-based frameworks. Future work highlighted by the researchers includes extending comparisons to broader continuous control benchmarks, evaluating modern activations such as Mish or PreLU, and testing advanced evolutionary algorithms like CMA-ES and OpenAI-ES across varying network depths.

Subject of Research: Comparative evaluation of evolutionary algorithms and activation functions in neuroevolutionary deep reinforcement learning

Article Title: A comparative study of evolutionary algorithms and activation functions in neuroevolution deep reinforcement learning

Article References: Magdy, H., Sarhan, A. M., Sallam, E., & Eita, M. A. (2026). A comparative study of evolutionary algorithms and activation functions in neuroevolution deep reinforcement learning. Journal of Electrical Systems and Information Technology, 13(1), Article 103. https://doi.org/10.1186/s43067-026-00403-4

Image Credits: AI Generated

DOI: 10.1186/s43067-026-00403-4

Keywords: NeuroEvolution, Deep Reinforcement Learning, Evolutionary Algorithms, Activation Functions, Inverted Double Pendulum, Augmented Random Search, NoiseReuseES, PersistentES, PGPE, GELU, SELU, CELU, Continuous Control, JAX, Brax, Learning Stability

Cite Scienmag News

Gavin Prescott. (November 17, 2026). Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability. Scienmag. https://scienmag.com/evolutionary-algorithm-study-shows-activation-functions-control-deep-reinforcement-learning-stability/

Gavin Prescott. "Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability." Scienmag, 17 November 2026, https://scienmag.com/evolutionary-algorithm-study-shows-activation-functions-control-deep-reinforcement-learning-stability/. Accessed 17 November 2026.

Gavin Prescott. "Evolutionary Algorithm Study Shows Activation Functions Control Deep Reinforcement Learning Stability." Scienmag. November 17, 2026. https://scienmag.com/evolutionary-algorithm-study-shows-activation-functions-control-deep-reinforcement-learning-stability/

Tags: activation functionsAugmented Random SearchBraxCELUcontinuous controldeep reinforcement learningevolutionary algorithmsGELUInverted double pendulumJAXLearning StabilityneuroevolutionNoiseReuseESPersistentESPGPESELU
Share26Tweet16
Previous Post

Adolescent Experienced Probable Tramadol Serotonergic Toxicity With Preserved Consciousness After Surgery

Next Post

Early Prognostic Markers Predict Bioprosthetic Mitral Valve Dysfunction in Younger Patients

Related Posts

Explainable AI Framework Improves Automated Alzheimer’s Detection in Brain MRI Scans
Science News

Explainable AI Framework Improves Automated Alzheimer’s Detection in Brain MRI Scans

November 17, 2026
Dynamic Magnetic Fields and Calcipotriol Alleviate Psoriasis-Like Skin Lesions in Mice
Science News

Dynamic Magnetic Fields and Calcipotriol Alleviate Psoriasis-Like Skin Lesions in Mice

November 17, 2026
Data Fragmentation Across Sectors Impedes Progress on Health Inequalities from Violence
Social Science

Data Fragmentation Across Sectors Impedes Progress on Health Inequalities from Violence

November 17, 2026
Biliopancreatic Limb Lengths Showed No Significant 36-Month Weight-Loss Differences in Cohort
Medicine

Biliopancreatic Limb Lengths Showed No Significant 36-Month Weight-Loss Differences in Cohort

November 17, 2026
Autonomous Ships Face Operational Limits at Sea and Ports, Seafarers Report
Science News

Autonomous Ships Face Operational Limits at Sea and Ports, Seafarers Report

November 17, 2026
Professional Footballers Experience Sustained Career Declines After ACL Reconstruction
Medicine

Professional Footballers Experience Sustained Career Declines After ACL Reconstruction

November 17, 2026
Next Post
Early Prognostic Markers Predict Bioprosthetic Mitral Valve Dysfunction in Younger Patients

Early Prognostic Markers Predict Bioprosthetic Mitral Valve Dysfunction in Younger Patients

  • Mothers who receive childcare support from maternal grandparents show more optimized

    Mothers who receive childcare support from maternal grandparents show more parental warmth, finds NTU Singapore study

    27656 shares
    Share 11059 Tweet 6912
  • University of Seville Breaks 120-Year-Old Mystery, Revises a Key Einstein Concept

    1061 shares
    Share 424 Tweet 265
  • Bee body mass, pathogens and local climate influence heat tolerance

    682 shares
    Share 273 Tweet 171
  • Researchers record first-ever images and data of a shark experiencing a boat strike

    546 shares
    Share 218 Tweet 137
  • Groundbreaking Clinical Trial Reveals Lubiprostone Enhances Kidney Function

    531 shares
    Share 212 Tweet 133
Science

Embark on a thrilling journey of discovery with Scienmag.com—your ultimate source for cutting-edge breakthroughs. Immerse yourself in a world where curiosity knows no limits and tomorrow’s possibilities become today’s reality!

RECENT NEWS

  • Explainable AI Framework Improves Automated Alzheimer’s Detection in Brain MRI Scans
  • Dynamic Magnetic Fields and Calcipotriol Alleviate Psoriasis-Like Skin Lesions in Mice
  • Data Fragmentation Across Sectors Impedes Progress on Health Inequalities from Violence
  • Biliopancreatic Limb Lengths Showed No Significant 36-Month Weight-Loss Differences in Cohort

Categories

  • Agriculture
  • Anthropology
  • Archaeology
  • Athmospheric
  • Biology
  • Biotechnology
  • Blog
  • Bussines
  • Cancer
  • Chemistry
  • Climate
  • Earth Science
  • Editorial Policy
  • Marine
  • Mathematics
  • Medicine
  • Pediatry
  • Policy
  • Psychology & Psychiatry
  • Science Education
  • Science News
  • Social Science
  • Space
  • Technology and Engineering

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 5,151 other subscribers

© 2026 Scienmag - Science Magazine

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • HOME
  • SCIENCE NEWS
  • CONTACT US

© 2026 Scienmag - Science Magazine

Discover more from Science

Subscribe now to keep reading and get access to the full archive.

Continue reading