The heart of the matter lies in a technique called recursive least squares, or RLS, a workhorse of online system identification. Engineers use RLS to build a mathematical model of a physical process in real time, feeding in measurements of inputs and outputs and continuously refining the model’s parameters so that its predictions match reality. In the aerospace case study, the process variable is pressure inside a hydraulic circuit driven by a gear pump, standing in for the high-pressure fuel pump that supplies an aircraft’s fuel metering unit and injectors. The RLS algorithm updates its parameter estimates using a gain vector computed from a covariance matrix, conventionally written as P, which tracks the uncertainty of the estimates. As fresh data arrive, P shrinks, signaling growing confidence in the identified model. The catch is that everything depends on how P is initialized at time zero.

Set the initial covariance matrix to zero and the algorithm is paralyzed: the update gain stays at zero forever and the parameters never move. Set it too large and convergence drags on for a long time, while any disturbance triggers abrupt, violent parameter updates that can prolong identification or break it entirely. Traditionally, engineers have chosen this initial value by trial and error, relying on deep intuition about the specific system at hand. The research team, led by Daniel Eduardo Palacios-Lopez and Mario Cesar Maya-Rodriguez, argues that this ad hoc practice is a genuine unsolved challenge in the field, and their experiments prove just how costly guesswork can be. In a sensitivity study of forty runs with randomly assigned initialization values, performance varied wildly, with the worst cases showing settling times above seventy-five seconds and error integrals hundreds of times larger than the best runs.

To replace guesswork with systematic search, the team first needed a way to score candidate initializations. Conventional metrics such as root mean square error, integral square error, integral absolute error, and their time-weighted variants each capture only part of the picture, typically the magnitude of the estimation error without regard to how that error behaves over time. For aircraft fuel systems, both properties matter: the error must be small, and it must die down smoothly, without oscillations that could destabilize a downstream controller. The researchers therefore proposed a new composite metric, the Integral of the Absolute Error and its Derivative, or IAED. It sums the absolute error plus a weighted term proportional to the absolute rate of change of the error, with a weighting factor set to 0.1 in this study. The derivative term acts as a buffer against erratic convergence, penalizing parameter trajectories that jitter or swing wildly even if their average error looks acceptable.

With the metric in hand, the team formulated an optimization problem: find the pair of numbers that define the initial covariance matrix so as to minimize the IAED, subject to a constraint that the identified parameters settle within a two percent error band before a maximum allowed time. They encoded the covariance matrix cleverly, writing it as a mantissa times ten raised to an exponent, much the way computers store large numbers in finite bits. This encoding lets the search algorithm jump across orders of magnitude, which matters because preliminary trials showed that solutions of similar quality could differ by three orders of magnitude in their parameter values. The problem is also multimodal, meaning infinitely many pairs map to the same matrix, but any pair that minimizes the cost function yields the desired initialization.

Because each candidate solution had to be evaluated on the real plant, taking roughly five minutes per test, population-based heuristics demanding hundreds or thousands of evaluations were impractical. The team instead adapted Random Mutation Hill Climbing, a single-solution iterative method that proposes small random perturbations to the current solution and keeps any perturbation that improves the score. If no direction helps, the step size is shrunk by a constant factor, allowing the search to zoom in on promising regions. Infeasible solutions violating the settling time constraint were handled with a penalty that assigns them a very large cost. Twenty-one heuristic runs, each limited to twenty update steps, were enough to beat the best of forty random trials, a striking demonstration of efficiency under severe experimental constraints.

The numbers tell a compelling story. The best random initialization achieved an IAED score of 19.49, a root mean square error of 0.356, and a settling time of 45.33 seconds. The heuristic search found an initialization with an IAED of 10.32, nearly halving the cost, while cutting the root mean square error to 0.192 and the settling time to 43.26 seconds, an improvement of 2.06 seconds. Every standard indicator improved: integral absolute error fell from 18.89 to 9.99, integral square error from 15.23 to 4.41, and integral time absolute error from 356.78 to 181.58. The winning parameters corresponded to a covariance matrix of five times ten raised to the 4.3 power. A four-dimensional heat map comparing error, error derivative, cost, and settling time visually confirmed the heuristic solution’s dominance over the entire random search space.

Beyond the experiments, the authors built a formal mathematical framework to explain when and why their approach works. Using Lyapunov stability analysis, they showed that under persistent excitation of the input signal, the parameter estimation error of RLS is exponentially asymptotically stable, and they proved that an optimal initialization tuple exists on the compact search space by appealing to the Extreme Value Theorem. Two corollaries distinguish ideal from real conditions: in a noise-free environment the estimation error converges to zero, while with measurement noise and unmodeled dynamics the algorithm achieves weak convergence, entering and remaining within a bounded neighborhood whose radius depends on the disturbance power. A further theorem guarantees that the heuristic search itself converges, provided the cost functional satisfies a Lipschitz condition that rules out abrupt discontinuities in the search landscape.

The practical stakes extend well beyond the laboratory bench. The testbed, a gear pump feeding a hydraulic circuit through a regulating valve held at 64.2 percent opening with the motor at 40 percent speed, is explicitly a scaled proof-of-concept rather than certified aerospace hardware, and the authors are candid about that limitation. Yet the underlying trend scales. In a real aircraft, faster and smoother identification convergence prevents prolonged transient states during which a model-based controller, such as model predictive control, might govern poorly or even destabilize the plant. Shorter transients mean less thermal and mechanical stress on actuators and better energy efficiency, directly supporting the more-electric aircraft agenda and Sustainable Development Goal 7 on clean energy. The team notes that future work will impose explicit finite-time convergence bounds and explore other heuristic and metaheuristic search strategies, potentially turning what was once a designer’s black art into a routine, automated step in deploying adaptive control on next-generation aircraft.

Subject of Research: Heuristic initialization of the covariance matrix in recursive least squares system identification for aerospace fuel pressure control

Article Title: Heuristic-based covariance initialization for RLS identification: A new IAED metric applied to aerospace pressure control systems

Article References: Palacios-Lopez, D. E., Maya-Rodriguez, M. C., Lopez-Muñoz, R., Lopez-Yañez, I., & Lozano-Hernandez, Y. (2026). Heuristic-based covariance initialization for RLS identification: A new IAED metric applied to aerospace pressure control systems. Results in Engineering, 32, Article 113378. https://doi.org/10.1016/j.rineng.2026.113378

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113378

Keywords: recursive least squares, system identification, more-electric aircraft, fuel injection, pressure control, covariance matrix initialization, heuristic optimization, hill climbing, IAED metric, model predictive control, aerospace engineering, energy efficiency

The core insight behind EPR-C3 is that the search for the best subset should not be blind. Traditional approaches to the problem fall into two broad camps, each with well-known weaknesses. Stepwise procedures, which add or remove predictors one at a time based on statistical criteria, are fast but notoriously unstable, often missing the globally best model and producing results that shift with small perturbations of the data. Penalized regression methods such as the lasso, ridge, and elastic net shrink coefficients to perform implicit selection, but they transform the problem itself, replacing the ordinary least squares formulation with a modified objective and yielding biased estimates that complicate interpretation. Genetic algorithms and other stochastic search techniques can explore large spaces, but their reliance on random initialization and random operators means that two runs of the same algorithm on the same data can produce different answers, undermining the reproducibility that scientific work demands.

EPR-C3 takes a different path. The method is a deterministic multi-start neighborhood-search heuristic, meaning that it systematically explores multiple regions of the subset space using structured moves rather than random jumps, and that repeated runs on identical data produce identical results. The name encodes its workflow: expansion, perturbation, and reduction form the exploratory core, while C3 refers to a three-part constraint refinement stage comprising correlation cleanup, replacement recovery, and variance inflation factor pruning. Each stage addresses a specific pathology of high-dimensional regression. Expansion grows a candidate subset by adding predictors that promise improvement. Perturbation shakes up the current solution to escape local optima, the traps where a greedy search would otherwise stall. Reduction trims away predictors that no longer earn their place.

The C3 refinement stage is where the method embeds statistical admissibility directly into the search. Correlation cleanup removes predictors that are so strongly correlated with others already in the model that they add redundancy rather than information. Replacement recovery attempts to swap problematic predictors for alternatives that preserve predictive power while restoring stability. Variance inflation factor pruning, the final safeguard, quantifies how much the variance of an estimated coefficient is inflated by multicollinearity and eliminates predictors whose presence destabilizes the model. By enforcing these constraints during the search rather than checking them afterward, EPR-C3 ensures that every candidate model it evaluates is statistically admissible, avoiding the embarrassing situation in which a search returns a model with excellent fit but coefficients so unstable that they are meaningless.

A crucial design decision sets EPR-C3 apart from penalized approaches: it preserves the native ordinary least squares formulation throughout. The final output is not a shrunken or regularized set of coefficients but an explicit regression equation of the familiar form, with interpretable coefficients estimated by classical least squares. This matters enormously for applied researchers in fields such as chemistry, medicine, and the social sciences, where the goal is often not just prediction but understanding, and where an explicit equation with transparent coefficients is the deliverable that matters. The author’s own background in chemical modeling, including prior work on predicting acid dissociation constants in nitrogen compounds, reflects this applied orientation toward interpretable equations.

The benchmark results are striking. In analyses comparing EPR-C3 against exhaustive all-subsets regression, the heuristic recovered a high proportion of the Top-K admissible subsets, the best-ranked models that exhaustive search identifies, while dramatically reducing computational cost in highly combinatorial problems. In other words, when the exhaustive answer is computable, EPR-C3 usually finds it; when exhaustive search is impossible, EPR-C3 still delivers high-quality admissible models in a fraction of the time. The study goes further and proposes an empirical utility threshold, a practical guideline that tells analysts when the heuristic search becomes preferable to exhaustive enumeration, giving practitioners a principled rule for choosing between the two strategies rather than guessing.

The comparative evaluation is unusually thorough for a methods paper. EPR-C3 was benchmarked against stepwise procedures, penalized-regression preselection, filter and wrapper screening approaches, genetic algorithms, and branch-and-bound subset selection, the latter being the classical exact method that prunes the search space using bounds on the objective. Across these comparisons, a consistent pattern emerged: embedding admissibility constraints within the search workflow improves the tradeoff between recovery of the best models and computational efficiency. Competing methods that search first and check statistical validity later, or that abandon the least squares framework entirely, tend to either waste effort on inadmissible candidates or return models that require post-hoc repair. EPR-C3’s constraint-aware architecture avoids both failure modes.

Reproducibility and robustness received dedicated attention. Bootstrap analyses, in which the method was run on resampled versions of the data, showed close agreement with exhaustive selection in best-subset recovery, out-of-bag performance, and predictor-inclusion concentration, the degree to which the same predictors are selected consistently across resamples. This concentration measure is particularly important because instability of variable selection is one of the most cited criticisms of heuristic and stepwise methods. A seed-package analysis further confirmed that the deterministic nature of the algorithm holds in practice, with repeated runs converging on the same solutions. An applied case study demonstrated that EPR-C3 could reproduce a previously published multiple linear regression equation, recovering the same model with reduced runtime, providing a concrete demonstration that the method delivers on real problems rather than only on synthetic benchmarks.

The significance of this work extends beyond the specific algorithm. Variable selection remains one of the most contested practices in applied statistics, with methodologists repeatedly warning that automated selection can produce misleading inference, inflated significance, and models that fail to replicate. By making the search deterministic, constraint-aware, and anchored in the classical least squares framework, EPR-C3 offers a middle path between the computational impossibility of exhaustive search and the statistical compromises of penalization and greedy stepwise selection. The method is implemented in the author-maintained MLR-X project and publicly available, lowering the barrier for adoption. As datasets in genomics, chemometrics, economics, and the behavioral sciences continue to swell in dimensionality, tools that combine computational tractability with statistical admissibility and exact reproducibility are likely to find eager audiences. The study, published as volume 22, article 311 of the journal and supported by ANID-FONDECYT grant funding, suggests that the future of high-dimensional model selection may lie not in abandoning classical regression but in searching its vast model space more intelligently.

Subject of Research: Deterministic constraint-aware subset selection in high-dimensional multiple linear regression

Article Title: Epr-c3: a deterministic constraint-aware heuristic for high-dimensional subset selection in multiple linear regression

Article References: Epr-c3: a deterministic constraint-aware heuristic for high-dimensional subset selection in multiple linear regression. (n.d.). https://doi.org/10.1007/s41060-026-01298-0

Image Credits: AI Generated

DOI: 10.1007/s41060-026-01298-0

Keywords: subset selection, multiple linear regression, ordinary least squares, multicollinearity, variance inflation factor, heuristic optimization, best-subset regression, statistical computing, high-dimensional data, model selection, deterministic algorithms, bootstrap validation