Brain-Inspired Wiring Rules Reshape the Dynamics of Recurrent Neural Networks

Recurrent neural networks (RNNs) occupy a curious position in modern machine learning. Unlike the feedforward architectures that dominate large language models, RNNs contain feedback loops that allow them to sustain activity even when no external input arrives, behaving less like static calculators and more like self-running dynamical systems. In a new study published in Frontiers in Complex Systems, researchers at Friedrich-Alexander-University Erlangen-Nürnberg systematically asked a question that most prior work has glossed over: what happens to the dynamics and computational power of an RNN when its connection matrix is no longer statistically random, but instead carries the organizational fingerprints seen in real neural circuits?

The team, led by Claus Metzner, Achim Schilling, Andreas Maier, and Patrick Krauss, focused on three structural regularities that are hallmarks of biological networks. The first is Dale’s principle, the observation that a biological neuron is either excitatory or inhibitory, but never both, so that all of its outgoing connections share a single polarity. The second is reciprocity: if neuron A projects to neuron B, then B is more likely than chance to project back to A, a bias well documented in cortical circuits and central to Hopfield networks. The third is modularity, the tendency of the brain to organize neurons into densely connected groups embedded in a sparser background of weaker connectivity, a pattern that recurs from microcircuits up to whole brain areas.

Earlier work by the same group had shown that the dynamical regime of an RNN can be steered by purely statistical properties of its weight matrix: the width of the weight distribution, the connection density, and the balance between excitatory and inhibitory weights together determine whether the network settles into fixed points, oscillates globally, or descends into chaos. But those studies assumed statistically independent weights, an analytically convenient simplification that real brains violate. The new study therefore built weight matrices in which each of the three regularities could be dialed up continuously through its own control parameter, allowing the researchers to isolate the effect of each regularity while holding the fundamental statistics constant.

To characterize the resulting dynamics, the team used three compact measures. The fluctuation measure captures the average temporal variability of neural activations, distinguishing quiescent networks from strongly oscillating ones. The covariance measure, deliberately defined without mean subtraction or normalization, distinguishes global oscillations, where successive activations flip sign, from fixed points, where activations stay constant, and from chaos, where the average product hovers near zero. The nonlinearity measure indicates whether neurons operate in the linear middle of their sigmoidal activation function or are driven into saturated, digital-like states. Validation scans confirmed that each control parameter reliably produced its intended structural feature without substantially distorting the others.

The baseline phase diagrams, mapped across the plane spanned by coupling strength and excitatory-inhibitory balance, revealed four familiar regimes: a quiescent region at weak coupling, an oscillatory wing dominated by strong inhibitory connections, a fixed-point wing dominated by strong excitatory connections, and a chaotic regime near the balanced center. When the network was then embedded in a reservoir computing framework and asked to generate predefined output sequences after being primed by class-specific inputs, accuracy stayed high in the quiescent regime, collapsed in the chaotic regime, and also degraded in the deeply saturated parts of the oscillatory and fixed-point regions. Only narrow bands at the edges of chaos retained near-perfect performance, and those bands shrank as coupling strength increased.

The surprises came when the regularities were switched on. Strong Hopfield reciprocity, which in classical Hopfield networks stabilizes stored patterns as fixed-point attractors, turned out to be harmful here. It shrank the quiescent regime, pushed neurons toward saturated, quasi-digital operation, and drove accuracy in the chaotic regime even lower than in the standard random network. Correlation analysis explained why: reciprocal coupling produced strongly correlated pairs and blocks of neurons, some locked into period-two oscillations and others into fixed points, with near-perfect Pearson correlations that persisted even under added noise. The authors note that their use of synchronous updates, rather than the sequential updates of the original Hopfield model, explains why reciprocity produced oscillatory rather than memory-like attractors.

Dale homogeneity, by contrast, proved beneficial. When every sending neuron was constrained to a single output polarity, the initially weak correlations of the chaotic state gave way to stronger, predominantly positive correlations, and the activations settled into quasi-periodic collective fluctuations with noticeable phase shifts between neurons. Crucially, task accuracy improved across all tested phase points, and the reservoir could tolerate significantly higher coupling strengths within the chaotic regime before performance broke down. The structured, directional signal flow that Dale’s principle imposes appears to tame erratic fluctuations without sacrificing the dynamical richness that reservoir computing needs.

The most dramatic effects belonged to modularity. When the weight matrix was partitioned into square blocks, some assigned strong weights and others weak ones while the overall distribution width was kept constant, the oscillatory and fixed-point regimes shrank to narrow slivers of phase space, and the chaotic regime was replaced by a broad region of moderate, irregular but non-repetitive dynamics. Accuracy then rose to near-maximal levels across almost the entire phase diagram, meaning that even previously unproductive dynamical regimes became computationally useful. The correlation matrices showed internally coherent neuron clusters with weaker, mixed-sign coupling between modules, a balanced structure the authors identify as particularly favorable for reservoir computing. The gain was confirmed in a larger 100-neuron reservoir and in an independent patch-classification task, where accuracy climbed monotonically from near chance level as modularity increased.

Interestingly, the benefits of modularity did not require true block structure. When the block size was reduced so that no distinct modules remained, increasing the modularity parameter merely transformed the weight distribution from a single Gaussian into a heavy-tailed mixture of two Gaussians, and this alone still improved accuracy at one phase point. The authors draw a parallel to the human cortex, where synaptic strengths follow approximately log-normal, heavy-tailed distributions, a heterogeneity thought to combine stable high-impact pathways with a flexible background of weak connections. Dale homogeneity and modularity thus echo design principles that evolution appears to have already discovered.

The study has clear boundaries: it examined a single benchmark task, used fixed activation functions and untrained recurrent weights, and varied each regularity in isolation. Whether Dale homogeneity and modularity act synergistically, or whether reciprocity counteracts their benefits, remains open, as does the question of how these effects scale to large, hierarchical architectures with plastic synapses. The authors also suggest applying the same dynamical metrics to empirically derived connectomes of organisms such as C. elegans, Drosophila, or zebrafish larvae, and propose moving beyond eigenvalue-based criteria toward measures of trajectory reproducibility and state-space geometry. If the findings generalize, the message is striking: the structural quirks of biological wiring are not idiosyncrasies to be smoothed away, but functional ingredients that artificial networks may be missing.

Subject of Research: Effects of Dale's principle, reciprocity, and modularity on the dynamics and reservoir computing performance of recurrent neural networks

Article Title: Organizational regularities in recurrent neural networks

Article References: Metzner, C., Schilling, A., Maier, A., & Krauss, P. (2026). Organizational regularities in recurrent neural networks. Frontiers in Complex Systems, 3, Article 1636222. https://doi.org/10.3389/fcpxs.2025.1636222

Image Credits: AI Generated

DOI: 10.3389/fcpxs.2025.1636222

Keywords: recurrent neural networks, reservoir computing, Dale's principle, Hopfield networks, modularity, neural dynamics, chaos, edge of chaos, connectomics, computational neuroscience, weight matrices, machine learning