The core insight of the work is that most of the records and most of the columns in a typical banking dataset are redundant. When a bank profiles a customer, it may hold hundreds of variables representing that individual, from transaction counts and account ages to demographic markers, and many of these variables overlap heavily. At the same time, thousands or millions of customer records often cluster into patterns that a handful of representative examples can capture. The computational cost of many classification algorithms grows with both the number of instances and the number of features, so every redundant record and every superfluous variable inflates training time, memory usage, and storage requirements. For an industry that runs classification tasks continuously across enormous customer bases, the savings from aggressive but careful reduction can be substantial.

To compress the number of instances, the researchers turned to a self-organizing map, an unsupervised neural network architecture introduced by Teuvo Kohonen in the 1980s. A SOM maps high-dimensional input data onto a usually two-dimensional grid of neurons, preserving the topology of the original data space: similar customers end up near each other on the map. Each input record activates a best matching unit, the neuron whose weight vector most closely resembles the record. By training the map and then using the neuron weight vectors as prototypes, the method replaces thousands of raw records with a far smaller set of synthetic representatives that summarize the structure of the data. In their experiments, this prototype generation step reduced the Census Income benchmark dataset by 99.3 percent and the proprietary banking dataset by 99.8 percent, meaning that fewer than one in a hundred original records needed to be retained to preserve the information the classifier requires.

Compressing instances is only half the battle. The second axis of reduction concerns the features themselves, and here the team proposed five weighting procedures that assign each variable a degree of relevance before the classification step. The most novel of these is a feature weighting technique based directly on the self-organizing map, dubbed FWSOM. Because the SOM has already learned a low-dimensional representation of the data, its internal structure encodes information about which variables drive the organization of the map. FWSOM exploits this by analyzing how strongly each feature contributes to the positioning of records on the trained map, producing weights that can then be folded into a weighted Euclidean distance metric used by the k-nearest neighbors classifier. Variables judged irrelevant receive low weights, effectively muting their influence without requiring a separate feature selection pass.

The remaining four procedures adapt permutation importance, a model-agnostic technique popularized by random forests and widely used in interpretability tools, to the peculiarities of SOM output. Permutation importance works by shuffling the values of one feature at a time and measuring how much the model’s performance degrades; a feature whose shuffling wrecks the predictions is clearly important. The catch is that the standard formulation assumes a conventional supervised model, not a topology-preserving map. The researchers therefore designed four tailored criteria: performance-based PI, which tracks changes in overall accuracy; error-based PI, which monitors shifts in classification error; prediction changes-based PI, which counts how often predictions flip when a feature is permuted; and F1-score-based PI, which focuses on the harmonic mean of precision and recall, a metric better suited to imbalanced datasets where one class, such as defaulters or fraud cases, is rare. Each variant offers a different lens on feature relevance, and the team evaluated all of them empirically.

The experimental design paired a public benchmark with real-world data. The Census Income dataset, drawn from the UCI Machine Learning Repository, is a classic classification problem in which the goal is to predict whether an individual earns above a certain threshold, a task structurally similar to many banking applications such as creditworthiness assessment. Alongside it, the researchers used a proprietary dataset from GAMCO covering real bank customers, giving the evaluation a dose of industrial reality that benchmark studies often lack. Performance was measured using standard metrics including accuracy, balanced accuracy, sensitivity, and false positive rate, ensuring that the compressed models were judged not merely on how fast they ran but on whether they still classified correctly, including on the minority classes that matter most in risk management.

The results were striking on both fronts. Permutation importance proved the more aggressive feature pruner, cutting 82.40 percent of features from the bank dataset and 21.49 percent from the income dataset. FWSOM, while more conservative, still removed 6.4 percent of features from the banking data and 14.29 percent from the income data, with the advantage that it integrates seamlessly into the SOM-based pipeline rather than requiring a separate model. Combined with the near-total prototype reduction, the full methodology delivered datasets that were orders of magnitude smaller than the originals. The practical consequences cascade: lower storage requirements, reduced memory footprints during training, faster classification of new customers, and simpler data analysis overall. For banks operating under tight latency constraints and rising cloud computing costs, a classifier that needs a fraction of the data to reach comparable decisions translates directly into money saved.

What makes the approach particularly appealing for the financial sector is its interpretability. Permutation importance produces an explicit ranking of which variables drive predictions, which aligns with regulatory expectations that automated decisions be explainable. Related research has applied permutation-based methods to default prediction and to identifying biomarkers in medicine, underscoring the technique’s versatility. By fusing that interpretability with the topology-preserving compression of the SOM, the Spanish team has built a pipeline in which the same structure that shrinks the data also reveals which customer attributes actually matter. The work builds on the group’s earlier prototype generation method using a growing self-organizing map, published in the same journal in 2023, extending it from instance reduction alone to a joint treatment of rows and columns.

The methodology also fits into a broader research landscape. Feature selection has been tackled with genetic algorithms, particle swarm optimization, Harris hawks optimization, sine-cosine algorithms, and variational autoencoders, each bringing computational overhead of its own. Prototype selection, meanwhile, has been explored through multi-armed bandits, geometric medians, and multilabel instance-based methods. The SOM-based approach distinguishes itself by addressing both dimensions of reduction within one unsupervised framework, and by being validated on proprietary banking data rather than benchmarks alone. The authors acknowledge limitations: the banking dataset and the code are proprietary and not publicly available, which restricts independent replication, and the balance between reduction and accuracy must be tuned per application. Still, the reported figures suggest that in data-heavy industries, the smartest model may be the one trained on far less. As machine learning spreads further into finance, healthcare, and beyond, techniques that strip away redundancy while preserving signal are likely to become as important as the algorithms they feed.

Subject of Research: A data reduction methodology combining self-organizing maps and permutation importance for k-nearest neighbors classification in the banking sector

Article Title: New data reduction methodology for classification applied to the banking sector

Article References: Ruiz-Moreno, S., Núñez-Reyes, A., García-Cantalapiedra, A., & Pavón, F. (2026). New data reduction methodology for classification applied to the banking sector. Neural Computing and Applications, 38(19), Article 757. https://doi.org/10.1007/s00521-026-12471-8

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12471-8

Keywords: machine learning, self-organizing map, k-nearest neighbors, data reduction, permutation importance, feature selection, banking, prototype generation, dimensionality reduction, Neural Computing and Applications, data, reduction

The research, led by Sureerat Makmuang and Kanet Wongravee of the Sensor Research Unit at Chulalongkorn University in Thailand, together with Simon Maher of the University of Liverpool and Sanong Ekgasit of Chulalongkorn University, combines near-infrared hyperspectral imaging (NIR-HSI) with a self-organizing map, or SOM, a type of artificial neural network that learns to organize complex data without being told what to look for. The result is a workflow that transforms the invisible chemical signatures of plastics into vivid color-coded maps, in which each polymer class—PET, polyethylene, polypropylene, or polystyrene—appears as its own distinct hue painted across the sandy terrain.

The physics behind the technique is elegant. When near-infrared light strikes a plastic fragment, the chemical bonds within the polymer absorb specific wavelengths in patterns as unique as fingerprints. A hyperspectral camera captures not just a conventional image but a full spectrum at every pixel, effectively recording hundreds of narrow wavelength bands simultaneously. Each pixel therefore carries a chemical identity waiting to be decoded. The challenge has always been interpretation: a sandy surface littered with fragments of different shapes, sizes, colors, and orientations produces spectra that are noisy, mixed, and difficult to separate with conventional statistical tools.

That is where the self-organizing map enters. An SOM is trained on the spectral data by repeatedly adjusting an internal grid of artificial neurons so that similar spectra cluster together, forming a topology that mirrors the chemical relationships in the data. In this study, the researchers enhanced the standard approach with a modified SOM and a novel percent-based expansion tolerance, or PBET, scheme that allows the model to estimate semi-quantitatively how much of a scanned surface is covered by each polymer type. The output is doubly informative: qualitative maps that show exactly where each class of microplastic sits, and quantitative coverage estimates expressed as percentages of the imaged area.

To test the system, the team prepared fragments of polyethylene terephthalate, polyethylene, polypropylene, and polystyrene from everyday household plastic materials, cut them into particles ranging from one to five millimeters, and distributed them on sand surfaces at controlled coverage levels spanning roughly 0.78 to 12.5 percent. After preprocessing the hyperspectral data to sharpen spectral quality, the modified SOMs classified the plastics with remarkable fidelity. Visually, individual fragments and mixed-polymer scenes alike were rendered as clean, color-separated maps. Quantitatively, the model’s predictions of surface coverage achieved coefficients of determination reaching as high as 1.00, with very low root-mean-square errors—performance figures that suggest the approach can rival labor-intensive reference methods.

Critically, the researchers did not stop at idealized laboratory conditions. They deliberately stressed the model with sources of real-world variability that plague field measurements: variations in particle size, differences in pigment color, and overlapping particles that stack atop one another and produce mixed spectra. The SOM workflow remained robust under these challenges, holding its classification accuracy where simpler methods would falter. The team then pushed the test further by imaging microplastic particles collected from natural beach samples—plastics the model had never seen before, weathered and coated by the environment. It identified and classified them correctly, a demonstration that the laboratory-trained system generalizes to the messy chemistry of the real world.

Perhaps the most striking finding concerns size. Conventional visual surveys of microplastic contamination rely on human eyes or standard photography, both of which routinely miss particles below about one millimeter. The hyperspectral approach proved capable of detecting and correctly classifying particles smaller than that threshold, underscoring a sensitivity that could close one of the largest blind spots in microplastic monitoring. Because smaller fragments are generally more bioavailable to organisms—and more likely to carry adsorbed toxins—this capability matters not just for counting pollution but for assessing its ecological risk.

The non-destructive nature of the method is its other defining advantage. Traditional microplastic analysis typically requires collecting sediment, transporting it to a lab, digesting organic matter, and running samples through spectroscopic instruments such as Fourier-transform infrared or Raman spectrometers—accurate techniques, but slow, costly, and destructive to the sample. Hyperspectral imaging flips that model: the sand stays in place, the measurement takes the form of a scan, and the same patch of ground can be revisited over time to track how contamination evolves. That opens the door to genuine longitudinal monitoring of beaches, dunes, and remediation sites, where repeated sampling has historically been impractical.

The researchers are careful to frame the achievement within the boundaries of their experiments. The workflow was developed and evaluated on sandy substrates with the four most common commodity polymers, under controlled illumination and geometry, and the coverage estimates are semi-quantitative rather than exhaustive particle counts. Wet sediments, dark soils, biofilms, and polymers beyond the tested four remain open challenges, and translating laboratory performance to drones or handheld field scanners will require further engineering. Yet the analytical foundation the study establishes—polymer-class mapping paired with surface-coverage estimation in a single rapid scan—is precisely the kind of groundwork needed before such instruments can be built.

If the approach matures as the results suggest, the implications reach far beyond sandy shores. Agricultural soils amended with plastic mulch fragments, construction sites receiving recycled aggregates, and coastal zones awaiting cleanup all demand the same basic information: which plastics are present, where, and in what abundance. By fusing hyperspectral imaging with self-organizing maps, this study demonstrates that answer can be rendered almost photographically—a colored chemical portrait of pollution that regulators, remediation engineers, and the public can read at a glance. In a world drowning in plastic fragments too small to see, a camera that makes them visible may prove one of the most consequential environmental tools of the decade.

Subject of Research: Non-destructive detection and mapping of microplastic contamination in sandy substrates using hyperspectral imaging and self-organizing maps

Article Title: Hyperspectral imaging and self-organizing map approach for non-destructive monitoring of microplastic contamination in sandy substrates

Article References: Makmuang, S., Maher, S., Ekgasit, S., & Wongravee, K. (2026). Hyperspectral imaging and self-organizing map approach for non-destructive monitoring of microplastic contamination in sandy substrates. Microplastics and Nanoplastics. https://doi.org/10.1186/s43591-026-00225-1

Image Credits: AI Generated

DOI: 10.1186/s43591-026-00225-1

Keywords: microplastics, hyperspectral imaging, self-organizing map, machine learning, sand substrates, polyethylene, polypropylene, polystyrene, PET, environmental monitoring, non-destructive analysis, beach pollution