Entropy-weighted feature selection lifts heart disease prediction to 94 percent accuracy

Heart disease remains the world’s single biggest killer, and a research team now argues that the smartest way to predict it may be to stop feeding machine learning models everything at once. In a study published in Discover Artificial Intelligence, Rais F. Shaikh and Avinash Sharma introduce an Entropy Weighted Hybrid Feature Selector, or EWHFS, a framework that fuses three complementary statistical criteria into a single entropy-normalized score. Tested on the widely used UCI Heart Disease dataset, the method distilled 13 clinical attributes down to just 8 highly discriminative features, cutting dimensionality by 38.5 percent while pushing classification accuracy to roughly 94 percent, with an area under the ROC curve of 0.95. The gains were statistically significant across validation folds, with p-values below 0.05 on paired t-tests and Wilcoxon signed-rank tests.

The scale of the problem the researchers are targeting is hard to overstate. According to the World Health Organization, cardiovascular disease, which spans coronary heart disease, stroke and heart failure, accounted for an estimated 19.8 million deaths globally in 2022, approximately 32 percent of all deaths worldwide. Evidence from multiple studies indicates that around 85 percent of those deaths result from myocardial infarction and stroke. The burden is not confined to wealthy nations; low- and middle-income countries often experience the most severe impact because their healthcare programs lack the diagnostic infrastructure to catch risk early. Early and accurate risk assessment, the authors argue, is therefore one of the most vital resources for reducing cardiovascular illness and death.

Traditional risk tools have well-documented limits. The Framingham Risk Score and QRISK models depend on linear predictions and were not designed to detect the complex interactions among multiple clinical risk factors that unfold in real patient records. Machine learning models can analyze nonlinear patterns in extensive medical data and flag high-risk patients earlier than classical statistical methods, and AI-driven approaches have shown promise in forecasting both in-hospital cardiac arrests and future cardiovascular events by detecting subtle signals within electronic health records. Yet the field faces persistent obstacles: many studies rely on small, homogeneous datasets that invite overfitting and limit generalizability to different patient groups, while deep learning models, however accurate, offer explanations that are difficult to trust in medical settings where transparency is essential.

Conventional feature selection carries its own baggage. Univariate filter-based techniques such as Mutual Information and Chi-Square score each feature independently, without accounting for duplicated attributes or the interactive influence that often appears in clinical datasets. That can lead models to pick redundant measurements with little incremental clinical relevance, which degrades both performance and interpretability. Hybrid and ensemble learning approaches can raise accuracy, but they typically demand more computing resources and produce harder-to-explain results, slowing adoption in healthcare environments that run on tight budgets and staffing limits. EWHFS was designed explicitly to attack this trade-off, squeezing maximum diagnostic information from a compact, non-redundant feature subset that clinicians can actually inspect.

The mathematics behind the framework rests on three pillars. First, Information Gain, derived from entropy calculations, quantifies how much uncertainty about heart disease status is reduced by observing each feature; larger values indicate higher diagnostic relevance. Second, a Fisher Score evaluates class separability by comparing the squared difference between the feature’s mean values in the diseased and non-diseased classes against the sum of the corresponding variances, penalizing features whose class distributions overlap. Third, an Anti-Correlation Index computes the average absolute Pearson correlation of each feature with all others, directly penalizing co-dependent clinical variables so that highly correlated attributes are not selected together. Each metric is entropy-normalized so that scores are bounded, probabilistically interpretable and scale-invariant across heterogeneous clinical attributes measured on wildly different numeric ranges.

The hybrid fusion score then combines these components with weighting coefficients that sum to one, maximizing feature relevance and class divergence while minimizing redundancy in a single optimization objective. Because each component can be computed in linear time with respect to the number of samples and features, the total complexity stays at O(n·m), making the approach suitable for large-scale clinical datasets and real-time systems. Notably, the entire framework ran on CPU-only hardware, an Intel Core i7 11th-generation processor with 16 GB of RAM, with no GPU required, a deliberate contrast with deep learning pipelines that depend on high-performance graphics processors.

The experimental protocol was built to prevent data leakage, a subtle but critical concern in biomedical machine learning. The dataset of 303 patient records was partitioned using stratified 10-fold cross-validation, with nine folds reserved for training and one fold held out as unseen test data in each iteration. All preprocessing, including median imputation for numerical variables like cholesterol, resting blood pressure and maximum heart rate, mode imputation for categorical variables, outlier handling using interquartile range and Z-score methods, min-max normalization, and class balancing with SMOTE and ADASYN oversampling, was applied exclusively to the training folds. The test fold remained completely untouched. Out-of-fold predictions from all ten iterations were then aggregated, so each sample was evaluated exactly once as unseen data, yielding an unbiased estimate of performance.

The results were striking. EWHFS reached about 94 percent accuracy, 93 percent precision, 94 percent recall and an F1-score of 93.5 percent, outperforming conventional feature selection strategies including Information Gain, Chi-Square, Mutual Information and Recursive Feature Elimination. Compared with the strongest baseline, the method lifted classification accuracy by roughly 3 to 6 percent while maintaining steady behavior across validation folds. On the ROC analysis, EWHFS achieved an AUC of 0.95, ahead of Recursive Feature Elimination at 0.94, Mutual Information at 0.92, Chi-Square at 0.90 and Information Gain at 0.89, with the proposed method’s curve remaining above the others across decision thresholds. The aggregated confusion matrix showed 155 diseased and 127 non-diseased cases classified correctly, with only 10 false negatives and 11 false positives out of 303 out-of-fold predictions.

That low false-negative count matters enormously in clinical practice, because missing a case of heart disease can delay intervention when time is most critical. The high recall of 94 percent indicates strong sensitivity, the model’s capacity to correctly identify patients who actually have the disease, while the redundancy suppression mechanism directly blocks the transmission of noisy and correlated features that typically generate false positives in standard selection methods. The F1-score, the harmonic mean of precision and recall, confirmed balanced performance on an imbalanced dataset, and the very small performance variance across cross-validation folds suggested a low risk of overfitting. Fold-wise accuracy and F1-score values analyzed with paired t-tests and Wilcoxon signed-rank tests verified that the gains over baseline methods were statistically significant at p below 0.05.

The authors emphasize deployment potential in resource-constrained healthcare environments. The smaller feature subset reduces memory usage and computational overhead, minimizing inference latency for real-time clinical monitoring systems and edge healthcare devices. Because the feature ranking remains interpretable, the framework could be integrated with electronic health record systems, wearable cardiovascular monitoring devices and IoT-enabled clinical infrastructures to support early risk assessment and automated decision support. The researchers acknowledge real-world obstacles, including heterogeneous clinical data formats, missing sensor values, hardware limitations, network latency and data privacy constraints, and note that model performance may vary across hospital populations and demographic distributions. Future work will focus on validating the framework with large-scale multi-center clinical datasets, optimizing it for edge AI deployment and exploring privacy-preserving federated learning approaches for secure healthcare integration.

The study arrives amid a wave of entropy-based feature engineering research in cardiovascular prediction, with prior hybrid approaches combining entropy weighting with SHAP explanations, genetic wrappers, ensemble bagging and deep neural networks, some reporting accuracies as high as 97.5 percent on other benchmarks. What distinguishes EWHFS, the authors argue, is its explicit, mathematically defined redundancy penalty and its simultaneous optimization of relevance, separability and diversity within one interpretable scoring system, rather than heuristic mixing of feature relevance measures. As machine learning moves from research labs into clinical decision support, the study suggests that the path to trustworthy cardiac prediction may run not through ever-larger models, but through asking, with entropy as the arbiter, exactly which few measurements carry the most diagnostic weight.

Subject of Research: An entropy weighted hybrid feature selection approach for heart disease prediction using machine learning

Article Title: An entropy weighted hybrid feature selection approach for heart disease prediction using machine learning

Article References: Shaikh, R. F., & Sharma, A. (2026). An entropy weighted hybrid feature selection approach for heart disease prediction using machine learning. Discover Artificial Intelligence, 6(1), Article 1630. https://doi.org/10.1007/s44163-026-02367-w

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02367-w

Keywords: entropy, weighted, hybrid, feature, selection, approach, heart, disease, prediction, machine, learning, scientific research