QualiGuard was developed by Batın Mert Kocakaplan, Emin Borandağ, Yusuf Özçevik, Fatih Yücalar, and Osman Altay, and is released under an MIT license with its trained model artifacts and derived metric dataset available under Creative Commons CC BY 4.0. The tool is built in Python 3.10 and integrates a remarkable range of established components: the Radon library for static complexity metrics, PyDriller for mining Git commit history, scikit-learn, LightGBM, XGBoost, TensorFlow, AutoGluon, and H2O AutoML for predictive modeling, and the GitHub REST API for large-scale repository collection. Rather than delivering disconnected fragments of analysis, the system assembles the entire chain, from raw repository to actionable verdict, into a single deployable package.

The premise underlying the tool is that software quality risk can be modeled as a supervised learning problem, in which measurable properties of source code and its development history are associated with observed quality outcomes. For every source file, QualiGuard extracts static size and complexity metrics, cognitive-complexity measurements derived from the abstract syntax tree, and process metrics from the Git history, including code churn, author entropy, and commit cadence. Defect labels are generated through the Śliwerski–Zimmermann–Zeller (SZZ) algorithm, a classic repository-mining technique that identifies lines deleted by bug-fixing commits and traces them back to the changes that introduced them. This combination of structural and historical evidence allows the assessment to account not only for how a file looks today but also for how it has evolved.

Beyond defect prediction, the tool detects seven classical code smells, including long methods, large classes, long parameter lists, deep nesting, god functions, high complexity, and low maintainability, using deterministic abstract-syntax-tree analysis. A second, predictive model complements this detector: using 28 code metrics, it estimates the probability that a file belongs to the high-smell-burden group, defined as the files at or above the project-specific 80th percentile of detected smells. This project-relative labeling avoids applying the same absolute smell-count threshold to repositories of very different sizes and structures. The deterministic detector identifies specific known flaws, while the predictive model provides a broader risk signal that can prioritize files for investigation, and the two components deliberately serve distinct, complementary roles.

To select its predictive models, the team benchmarked nine standalone candidates, spanning conventional machine learning, deep learning, and automated machine learning, plus one hybrid stacking configuration, on a dataset of 62,207 files mined from 1,000 public Python repositories. The evaluation used a cross-project five-fold GroupKFold protocol in which repository identity serves as the grouping variable, preventing files from the same project from leaking between training and test folds. This design matters because inflated accuracy from project-level leakage is a well-known pitfall in defect-prediction research. For defect prediction, the winning configuration was a hybrid stacking ensemble combining LightGBM with AutoGluon through an isotonic-calibrated logistic-regression meta-learner, achieving a mean F1 score of 0.64 while reducing fold-to-fold performance variability by roughly 38 percent compared with the strongest single model.

For code smell prediction, the story was different. Stacking yielded no consistent benefit, so the researchers retained a threshold-optimized LightGBM model, which attained a mean F1 score of 0.69. The lesson, the authors note, is that the optimal predictive configuration differs between tasks: defect labels and smell labels carry different class distributions, and assuming one configuration fits both would be unjustified. Operating thresholds were tuned per task, with files flagged as defect-prone at a calibrated probability of 0.42 and as smelly at 0.34. The calibrated probabilities are then mapped onto a three-tier decision policy: files below 0.30 defect probability pass automatically, those between 0.30 and 0.70 are routed to human review, and those at or above 0.70 are blocked from progressing without further inspection.

The user-facing side of QualiGuard is a Flask web application designed for local, single-user execution. A practitioner can paste the URL of a public GitHub repository or upload a compressed project archive of up to 100 megabytes, provided the archive retains its .git directory, since the process metrics and SZZ labeling depend on commit history. The analysis runs asynchronously in a background thread while a progress bar reports each milestone, from cloning through metric extraction, smell detection, and model inference. The resulting dashboard presents project-level health indicators such as defect density per thousand lines of code, refactor ratio, and recent activity, visualizes detected DevOps practices like GitHub Actions workflows, Dockerfiles, and Jenkins pipelines, and lists every analyzed file with its complexity, maintainability index, calibrated probabilities, and color-coded quality-gate tier.

Security received explicit attention, because the tool parses untrusted third-party code. Uploaded archives are validated against their central directory before extraction, with limits on entry count, total uncompressed size, and compression ratio to mitigate decompression bombs, and absolute paths or path-traversal components are rejected. Repository source code is parsed as text only and is never imported or executed. Before any Git command runs, a sanitization step removes repository-local hooks and rewrites the repository configuration using an allowlist, eliminating configuration options that could cause Git to execute external programs such as hooks, diff filters, or credential helpers. GitHub tokens, which are optional and used only to raise API rate limits, are verified before storage, masked in the interface, and excluded from logs. The software is accompanied by 266 automated tests covering metric extraction, labeling, prediction workflows, and web-tier security.

The authors are candid about the limits of the current evaluation. The practical assessment covers predictive performance and execution on a single workstation, and continuous deployment in production environments has not yet been tested. CI/CD integration, execution time across repositories of varying sizes, false-alarm burden, developer acceptance, and the impact of automated gate decisions on real merge outcomes all remain to be empirically assessed. The quality-gate tiers themselves use fixed probability boundaries, making QualiGuard better characterized as a learned, risk-based gate than a fully adaptive one, and the authors flag project-specific threshold adaptation and temporal recalibration as future work. Platform independence is a design characteristic rather than an experimentally validated result, since development and testing took place on Windows 11.

Even with those caveats, QualiGuard represents a meaningful step toward evidence-driven software quality assessment. It bridges two research threads, defect prediction and code smell detection, that are usually studied in isolation but speak to the same underlying notion of file-level risk, and it wraps both in a reproducible pipeline that researchers can reuse to build language-specific metric datasets. By replacing arbitrary fixed thresholds with calibrated, learned estimates of risk, and by making every verdict explainable down to the individual detected smell, the tool offers a glimpse of what continuous quality control in DevOps pipelines could look like when the gate itself has been taught, rather than merely configured. The code, dataset, and trained models are publicly available, inviting both practitioners and researchers to put the approach to the test.

Subject of Research: AI-driven risk-based quality gates for software defect prediction and code smell detection in DevOps pipelines

Article Title: QualiGuard: An AI-driven risk-based quality gate for defect prediction and code smell detection in DevOps pipelines

Article References: Kocakaplan, B. M., Borandağ, E., Özçevik, Y., Yücalar, F., & Altay, O. (2026). QualiGuard: An AI-driven risk-based quality gate for defect prediction and code smell detection in DevOps pipelines. SoftwareX, 36, Article 103090. https://doi.org/10.1016/j.softx.2026.103090

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103090

Keywords: QualiGuard, DevOps, defect prediction, code smells, machine learning, quality gates, SZZ algorithm, LightGBM, AutoGluon, software quality, repository mining, cross-project validation

The core challenge the team tackled is one that haunts deep learning across domains: neural networks are insatiably hungry for large, diverse, well-labeled datasets. In software engineering, that hunger is hard to satisfy. Annotated code smell datasets are scarce, expensive to produce, and heavily skewed—some smell types dominate the labeled examples while others appear only in tiny numbers. Class imbalance of this kind pushes a neural network toward the majority categories, leaving it nearly blind to the minority ones, which are often the most damaging defects. Data augmentation, the technique of programmatically expanding a training set without collecting new real-world samples, has transformed fields like image recognition and natural language processing, but translating it to source code has proven far trickier, because code must remain syntactically valid and semantically faithful after any transformation.

The researchers’ solution leans on a practice every programmer already knows: refactoring, the discipline-preserving rearrangement of code that changes its internal structure without altering its external behavior. Because a refactor is guaranteed to keep a program’s functionality intact while reshaping its form, it provides a principled way to generate legitimate new training variants of the same code. The team compared two augmentation engines: a rule-based approach that applies classical refactoring operations directly, and a large language model-based approach that uses generative models to produce refactored variants of source code. Crucially, they paired these with a loss penalty, an algorithmic balancing scheme for neural network training that penalizes errors on underrepresented classes more heavily, aiming to counteract the dataset’s skew from the optimization side as well as the data side.

To represent code in a form a neural network can digest, the team turned to pre-trained transformer models from the BERT family: the original BERT, which treats code as plain text; CodeBERT, which was pre-trained jointly on programming and natural languages; and GraphCodeBERT, which additionally incorporates the data-flow structure of programs during pre-training. Each model converts snippets of source code into dense numerical embeddings—vector fingerprints of a program’s semantics—which then serve as inputs to the downstream classification network. This embedding stage matters because two pieces of code that differ only in formatting or naming should map to nearby points in the embedding space, and the more structurally aware the embedding model, the more stable that mapping should be under refactoring.

The experimental results delivered a clear verdict. Code refactoring-based augmentation powered by GraphCodeBERT embeddings and large language models measurably enhanced classifier performance, and its benefits were most pronounced precisely where deep learning models are weakest: the imbalanced, underrepresented smell classes. When the team analyzed how the volume of augmented data influenced outcomes, they discovered a pattern of diminishing returns with an important twist. The greatest improvement arrived with the first increments of augmentation applied to minority classes; beyond that point, adding more synthetic examples yielded progressively smaller gains. The impact of augmentation volume also varied depending on the size of the class being boosted, a finding the researchers argue should inform how practitioners budget their augmentation efforts rather than blanket-duplicating synthetic data across all categories.

Equally instructive were the negative results. The loss penalty mechanism, which was expected to amplify the benefit of augmentation by rebalancing training dynamics, had a negligible effect on overall performance—and in some configurations proved actively detrimental. Augmentation based on vanilla BERT and CodeBERT embeddings likewise failed to move the needle or even hurt accuracy in certain settings. The implication is that not all code-aware embeddings are created equal: models that ignore program structure, such as plain BERT, appear unable to fully exploit refactored variants, while GraphCodeBERT’s data-flow-informed pre-training gives it the structural sensitivity needed to recognize that a refactored snippet and its original are two faces of the same entity.

Beyond the headline results, the study makes a methodological contribution of its own. Evaluating augmentation strategies requires repeatedly testing how performance responds to different volumes of synthetic data across multiple classes, a combinatorially expensive process. To tame this, the team developed a novel weighting strategy that optimizes the evaluation of augmentation volume distribution, cutting the computational overhead of the analysis while preserving its analytical precision. This kind of efficiency matters in practice, because organizations auditing large industrial codebases cannot afford exhaustive, compute-hungry tuning cycles for every detection model they deploy. The weighting scheme, the authors suggest, could become a reusable tool for anyone designing augmentation pipelines for code intelligence systems.

The broader significance of the work lies in what it says about the mechanics of learning from code. The findings reinforce a growing consensus that source code should not be treated as ordinary text. Programs carry rich graph-structured semantics—call graphs, data dependencies, inheritance hierarchies—and augmentation strategies that respect and exploit those structures deliver gains that text-level tricks cannot match. The results align with a wave of recent research on code-specific augmentation, from clone-aware contrastive learning to mixup-based transformations, but they are among the first to systematically quantify the relationship between augmentation volume, class size, and model architecture in the code smell domain, and to do so while comparing rule-based and LLM-based generation head to head.

For software engineering teams, the practical takeaway is immediately actionable. Code smell detection models trained with refactoring-based augmentation and structure-aware embeddings should be more reliable at flagging underrepresented defect patterns—precisely the smells that human reviewers most often miss. Because refactoring transformations are behavior-preserving by construction, the synthetic data they generate avoids the hallucination risks that come with purely generative augmentation, offering a safer middle ground between hand-crafted rules and free-form LLM generation. The researchers have also made their work reproducible, releasing all related source code and data through public repositories on GitHub and Zenodo, lowering the barrier for other teams to adopt and extend the technique.

Looking ahead, the authors suggest that code refactoring-based augmentation could drive the development of more efficient data augmentation strategies not only for code smell classification but for deep learning applications across software engineering more broadly—vulnerability detection, code search, clone detection, and automated repair among them. The lesson generalizes: when real-world labeled data is scarce, the discipline of software engineering itself, with its guarantees of behavioral equivalence, can become the engine that manufactures trustworthy training data. In a field racing to replace human code reviewers with machines, teaching models to see the same code through many refactored lenses may prove one of the most cost-effective ways to give them the experience they need—without a single additional line of manually annotated code.

Subject of Research: Data augmentation via code refactoring to improve deep learning-based code smell classification.

Article Title: Enhancing code smell classification with code refactoring-based data augmentation

Article References: Nizam, A., Aydin, M., Islamoglu, E., Sahmoud, S., & Ozaydin, S. B. (2026). Enhancing code smell classification with code refactoring-based data augmentation. Knowledge and Information Systems, 68(1), Article 254. https://doi.org/10.1007/s10115-026-02859-2

Image Credits: AI Generated

DOI: 10.1007/s10115-026-02859-2

Keywords: code smell, data augmentation, code refactoring, code embedding, GraphCodeBERT, CodeBERT, BERT, large language models, class imbalance, deep learning, software quality, Enhancing