Intelligent Undersampling Algorithm Enhances Machine Learning Accuracy on Imbalanced Data

In real-world data mining and machine learning applications, classification algorithms regularly encounter severely skewed sample distributions where instances of one category drastically outnumber those of another. Such class imbalances arise frequently across high-stakes domains, including industrial fault diagnosis, credit card fraud detection, and rare disease screening, where the cost of failing to identify a rare minority instance is significantly higher than misclassifying a common majority instance. Conventional classification algorithms designed to optimize overall predictive accuracy tend to be overwhelmed by majority-class patterns, frequently leading to poor recognition rates for minority-class samples. To mitigate this pervasive challenge, computer scientists Di Wang, Siyao An, Haoyu Gu, and Jiamin Li have introduced an intelligent data classification framework combining an enhanced k-nearest neighbor undersampling technique with a weighted ensemble of support vector machines, systematically addressing the information loss and decision boundary distortion that have long undermined traditional resampling methods.

Resampling strategies designed to restore balance between classes are generally divided into oversampling the minority instances and undersampling the majority instances. Undersampling offers distinct practical advantages in large-scale applications with extreme majority ratios due to its computational efficiency and operational simplicity, yet conventional undersampling approaches present substantial structural drawbacks. Random undersampling blindly discards majority instances, frequently stripping away vital discriminative features and eroding the true geometric structure of the majority distribution. Meanwhile, classical distance-based filtering techniques, such as standard k-nearest neighbors undersampling, rely solely on nearest-neighbor labels without evaluating underlying local density or boundary proximity. As a consequence, existing techniques frequently misidentify crucial boundary points, eliminate valid informational features, or fail to discard dense clusters of interior redundant data, ultimately hampering the generalization capability of downstream machine learning models.

The research team addressed these fundamental weaknesses by developing a density-aware and boundary-recognition undersampling model that assigns individualized retention weights to majority-class samples. Rather than applying coarse binary deletion rules, the proposed mechanism estimates the local spatial density of each majority sample using an improved k-nearest neighbor calculation based on Euclidean distance and an adaptive bandwidth parameter derived from the mean distance among majority neighbors. Simultaneously, the framework calculates a boundary coefficient that measures the proportion of minority instances present within each candidate’s global k-nearest neighborhood. By combining normalized local density with this boundary coefficient through an adjustable weighting parameter, the algorithm selectively prioritizes the preservation of majority instances located near complex classification borders while systematically flagging uninformative interior instances for elimination.

To prevent noisy or irregular samples from corrupting the reconstructed distribution, the proposed methodology integrates an automated outlier screening mechanism prior to final instance selection. The system calculates the mean local density across all majority candidates and marks any sample falling below thirty percent of this average threshold as an abnormal outlier unsuitable for classifier training. After excluding detected outliers, the remaining majority instances are ranked in descending order according to their composite retention weights. The algorithm then retains an exact number of top-ranking majority samples equal to the total count of available minority samples, directly achieving a balanced one-to-one ratio between classes. This targeted retention strategy ensures that the reconstructed training dataset preserves the principal structural boundary characteristics of the majority class while completely maintaining the integrity of the minority class.

To rigorously evaluate the fidelity of the balanced distributions generated by this process, the investigators established a mathematical reconstruction quality metric denoted as Q. This objective quality index quantitatively balances the preservation of information entropy before and after undersampling with the achieved degree of class equilibrium, assigning a seventy percent weight to entropy retention and a thirty percent weight to class balance. Through extensive grid-search experiments evaluating hyperparameter sensitivity, the researchers discovered that setting the weighting adjustment coefficient to 0.6 achieved the optimal balance between boundary sensitivity and internal redundancy elimination. Pearson correlation analysis demonstrated a robust correlation coefficient of 0.86 between the reconstruction quality metric and downstream area under the receiver operating characteristic curve scores, confirming that higher reconstruction quality directly translates into superior empirical classification performance.

Recognizing that even highly refined undersampling inevitably discards some amount of distributional information, the researchers combined their data reconstruction method with a modified Bagging ensemble architecture powered by support vector machines. While support vector machines equipped with radial basis function kernels excel at capturing complex decision boundaries within high-dimensional feature spaces, individual models remain vulnerable to sample perturbations and localized noise. The team utilized bootstrap sampling to generate multiple balanced training subsets from the reconstructed dataset, training independent support vector machine base classifiers in parallel. This ensemble structure effectively curbs model variance, alleviates the information bias introduced by data reduction, and significantly enhances overall predictive robustness across challenging classification tasks.

Unlike conventional Bagging architectures that combine model predictions through equal-weight majority voting, the researchers implemented a performance-oriented weighted voting mechanism to govern ensemble decision-making. Following the training of multiple base classifiers, each model is evaluated against the unsampled validation instances left out during bootstrap generation to calculate its individual classification accuracy. These validation accuracies are subsequently normalized into proportional voting weights, ensuring that base classifiers demonstrating superior discrimination exert a stronger influence on final aggregate predictions while weak or unstable classifiers are effectively discounted. Systematic tuning revealed that model accuracy increased rapidly before plateauing at ten base classifiers, achieving a peak validation accuracy of 89.71 percent and establishing an optimal trade-off between predictive power and computational overhead.

The experimental validation of the proposed framework was carried out in a standardized Python 3.9 environment leveraging the Scikit-learn and Imblearn libraries on an Intel Core i7 processor with 16 gigabytes of memory. The researchers conducted rigorous comparative evaluations across public benchmark datasets obtained from the UCI Machine Learning Repository, OpenML, and KEEL databases, spanning low, medium, and high class imbalance ratios. Performance outcomes were systematically contrasted against established sampling approaches, including random undersampling, Edited Nearest Neighbors, Tomek Links, One-Sided Selection, MUTE, Under-bagging KNN, and standard equal-voting Bagging-SVM models. To guarantee that observed performance gains were genuine rather than stochastic artifacts, Wilcoxon signed-rank tests were performed across ten repeated experimental iterations per dataset, yielding a p-value of 0.023 against the second-best performing benchmark and confirming statistical significance at the 95 percent confidence level.

Detailed distributional analysis on a representative imbalanced benchmark dataset demonstrated the precise filtering capability of the density-aware undersampling algorithm. In an experimental scenario where competing undersampling techniques were each tasked with retaining thirty-four majority instances to achieve class parity, random undersampling retained only 44.68 percent of key boundary samples due to blind removal, leading to severe structural degradation and blurred margins. Edited Nearest Neighbors achieved a boundary retention rate of 61.70 percent while over-eliminating valid instances. In contrast, the proposed k-nearest neighbor intelligent undersampling algorithm achieved a boundary retention rate of 95.74 percent, an internal redundant sample elimination rate of 92.36 percent, and an overall data information retention rate of 0.94, validating its capacity to cleanly separate critical boundary geometry from redundant interior clusters.

The downstream classification results summarized across comprehensive evaluation indicators demonstrated the decisive superiority of the integrated pipeline over all comparative baselines. Baseline configurations combining random undersampling with a single support vector machine suffered pronounced performance drops, yielding an overall accuracy of only 78.31 percent and a minority-class sensitivity of 65.38 percent due to the inadvertent loss of majority boundary features. The proposed hybrid framework achieved an overall accuracy of 89.70 percent, elevated minority sensitivity to 87.19 percent, attained a geometric mean of 88.8 percent, and reached an area under the receiver operating characteristic curve of 0.92. This represents a 16.05 percent improvement in area under the curve compared to random undersampling, along with a 5.75 percent advantage over mainstream ensemble classification models.

Receiver operating characteristic curve evaluations further illuminated the discriminatory power of the weighted voting Bagging-SVM architecture relative to individual and unweighted classifiers. While an isolated support vector machine model produced an area under the curve of 0.812 and standard equal-voting Bagging achieved 0.875, the proposed weighted fusion strategy attained an area under the curve of 0.938. Under equivalent false positive rate constraints, the true positive rate of the proposed model proved consistently superior, particularly within low false-alarm operational thresholds essential for practical monitoring systems. Parameter sensitivity analysis of the neighbor count parameter confirmed that setting k to 5 yielded a peak reconstruction quality of 0.943, whereas insufficient neighbor coverage at lower values or noise accumulation at higher values noticeably degraded structural boundary clarity.

Despite these demonstrable performance advantages, the authors highlighted several methodological limitations that provide clear avenues for future technological refinement. The current framework relies on cross-validation and grid searches to optimize neighbor parameters and outlier filtering thresholds, which introduces computational overhead and may exhibit reduced sampling efficiency when applied to ultra-high-dimensional datasets with extreme imbalance ratios. Furthermore, the pipeline currently operates without integrated deep representation learning modules capable of automated feature extraction from unstructured data. The authors noted that future research will focus on developing adaptive parameter updating strategies, incorporating feature dimensionality reduction modules, and exploring hybrid sampling architectures that couple intelligent majority undersampling with synthetic minority expansion to broaden practical utility across complex financial risk control and automated industrial inspection platforms.

Subject of Research: Machine learning classification algorithms for imbalanced datasets

Article Title: Intelligent under-sampled data classification algorithm based on machine learning and KNN

Article References: Wang, D., An, S., Gu, H., & Li, J. (2026). Intelligent under-sampled data classification algorithm based on machine learning and KNN. Discover Artificial Intelligence, 6(1), Article 1621. https://doi.org/10.1007/s44163-026-02422-6

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02422-6

Keywords: Machine Learning, Undersampling, K-Nearest Neighbors, Support Vector Machines, Bagging Ensemble, Class Imbalance, Data Reconstruction, Pattern Recognition, Intelligent, under-sampled, data, classification