TraceRank rethinks requirements traceability as configurable review support

Every large software project depends on a web of invisible connections. Requirements must be tied to the code that implements them, the tests that validate them, and the design documents that justify them. This web, known as requirements traceability, underpins change impact analysis, compliance review, maintenance, and assurance work. Yet in practice, trace matrices are incomplete, outdated, or exist only for fragments of a system, because creating and maintaining trace links is costly and time-consuming. A research team led by Hamad Alsawalqah at the University of Jordan, together with colleagues from Mutah University and the University of Tabuk, argues that the field has been solving the wrong problem all along. In a study published in Frontiers in Computer Science, they present TraceRank, a configurable review-support framework that treats traceability recovery not as a single ranking task but as a family of distinct review settings, each with its own evidence assumptions, inspection budget, and possibility of analyst feedback.

The core insight is deceptively simple: a method that is valid in one review setting may be invalid in another. A fully cold-start project with no validated trace links, a project with a partially validated trace matrix, a review constrained by a fixed effort budget, and an analyst-in-the-loop session do not expose the same information. They also do not require the same operating assumption. A technique that exploits validated historical links should never be compared as if it were a cold-start method. Likewise, a procedure that updates its rankings after analyst feedback is fundamentally different from a static ranking produced before review begins. TraceRank makes these assumptions explicit through a deterministic configuration rule: each recovery mode is enabled only when the evidence it requires is actually available under the declared task setting.

Technically, TraceRank organizes its evidence through a trace evidence hypergraph, a representation that connects requirements, requirement facets, candidate artifacts, artifact roles, evidence sources, validation states, and evidence-status descriptors. Requirement facets capture local parts of a requirement, such as actions, entities, conditions, constraints, or validation concerns, while artifact-role nodes describe implementation responsibilities like controller, interface, database, validator, or graphical user interface components. Evidence-source nodes record the type of support behind each candidate link, whether lexical retrieval, semantic matching, grouped-ranker support, dependency evidence, validated-link evidence, or analyst feedback. A trace-set hyperedge connects each requirement to its facets, candidate artifacts, and evidence bundle. This structure keeps heterogeneous support signals visible without forcing them all into one uninterpreted ranking score, which the authors argue is essential for review support rather than mere retrieval.

The cold-start foundation relies on a 42-dimensional candidate feature vector built from eight normalized retrieval and fusion scores, including TF-IDF, BM25, LSI, segment-maximum TF-IDF, specificity-adjusted TF-IDF, simple average, and reciprocal-rank fusion, together with ten scalar lexical, identifier, length, specificity, and score-dispersion features and rank-based features for each score family. Three training-only supervised models are fitted on these inputs: a class-balanced Logistic learning-to-rank model, a grouped LightGBM ranker, and a grouped XGBoost ranker. A conservative recovery-policy gate then accepts an objective-specific fusion only when its calibration utility improves by at least one percent and its critical recall measure does not decrease. On the frozen seed-42 split, this yields XGBoost for LibEST, LightGBM for SMOS, ranker-aware fusion for eANCI and iTrust, and an accepted objective-specific fusion for eTour. The best cold-start extension is a semantic-structural rescue that blends word-level and character-level semantic signals with structural cues such as direct artifact-name mentions, interface and implementation cues, package proximity, and artifact-name overlap, with calibrated weights selected per dataset.

The algorithmic centerpiece is the Validated Trace Responsibility Graph, or VTRG, enabled only when a project contains a partially validated trace matrix. The idea is that validated links reveal trace responsibility neighborhoods: requirements similar to a new requirement may share related artifacts, and artifacts that co-occur in validated trace sets may participate in the same implementation responsibility. VTRG constructs requirement-neighbor scores using label-free text similarity, builds artifact co-trace responsibility weights from validated trace sets, selects weak anchor artifacts from the cold-start base score, and optionally fits a latent trace-neighborhood representation through truncated singular value decomposition with at most eight components. The final score is a calibrated blend of cold-start and partial-trace evidence, with the globally selected Top-30 configuration using neighbor depth K equal to 10, anchor depth M equal to 10, and evidence weights of 0.45, 0.25, 0.20, and 0.10 for the base, requirement-neighbor, co-trace, and latent components respectively. Test labels are never used for weight selection, graph construction, or scoring.

The empirical evaluation spans five established traceability benchmark datasets: eTour, iTrust, SMOS, eANCI, and LibEST, totaling 447 requirements, 511 artifacts, 2,409 true links, and 51,407 candidate pairs. The primary point estimates use one frozen requirement-level 60/20/20 train/calibration/test split with random seed 42, ensuring that candidate pairs from the same requirement always remain in the same partition. In the cold-start setting, the best retained configuration recovers 404 of 501 true links at Top-30, achieving a LinkRecall@30 of 80.6 percent and completing 33 of 60 held-out requirements, compared with 381 links and 27 complete requirements for the base policy. Classical baselines fare considerably worse: TF-IDF recovers 293 links, BM25 only 252, and LSI 301, while the strongest learned ranker, LightGBM, recovers 380 links and is statistically indistinguishable from the base policy under paired requirement-level analysis.

The partial-trace results are striking. Under the partial-trace assumption, VTRG recovers 465 of 501 links, a LinkRecall@30 of 92.8 percent, completes 43 of 60 requirements, and reduces severe misses to just 12, where a severe miss is a true link ranked beyond the review depth plus a frozen boundary window of 10. Across the same five requirement-level split seeds used for the split-robustness analysis, VTRG achieves a mean LinkRecall@30 of 91.3 percent with a standard deviation of 4.0 percentage points and a mean CompleteRate@30 of 76.5 percent with a standard deviation of 8.1 percentage points. Dataset-level heterogeneity is substantial: LibEST is essentially saturated, SMOS gains most from VTRG, recovering 220 of 233 links with only one severe miss, eTour benefits from both rescue and VTRG, eANCI remains harder for complete recovery, and iTrust shows a smaller gain. The authors are careful to note that this gain is interpreted only under the partial-trace assumption; VTRG is not a cold-start ranker.

Selective validation makes the partial-trace setting more practical when analyst time is limited. The study evaluates validation budgets from zero to 100 percent of calibration requirements, using selection strategies including random, uncertainty-based, density-proxy, centrality-based, diversity-based, and VTRG-active selection. Training links alone already provide strong partial-trace evidence, recovering 458 of 501 links, and selected validation can approach the full-calibration result: 40 percent diversity-based selection reaches 465 links with a LinkRecall@30 of 92.8 percent, and 60 percent VTRG-active selection reaches 466 links at 93.0 percent. Paired hierarchical requirement-cluster bootstrap intervals with 10,000 replicates quantify uncertainty, treating the 60 positive requirements rather than the 501 nested links as the independent review units. The supported conclusion is that selected validation can approach full calibration, not that every strategy or budget is superior.

Zero-shot neural representation baselines strengthen the evidence that the reported results are not merely an artifact of comparing against classical lexical retrieval. On the identical seed-42 test requirements, CodeBERT recovers 192 of 501 links, GraphCodeBERT 203, UniXcoder 241, and SBERT-MPNet 265, all scored by cosine similarity of unit-normalized embeddings without trace labels or task-specific fine-tuning. The reproducible base recovery policy on the same requirements recovers 381 links, and the semantic-structural rescue recovers 404. The authors explicitly bound this comparison: TraceBERT itself was not executed, and these frozen representation baselines do not establish a universal ceiling for fine-tuned neural traceability models or LLM-based recovery. Archived diagnostic families, including multi-view neural semantic closure, fielded passage retrieval, pseudo-relevance feedback, coverage diversification, entity-role graph evidence, and a zero-shot cross-encoder reranker, produce only modest changes relative to the reproducible cold-start configurations.

Perhaps the most consequential contribution is the honest treatment of evidence status. Archived fixed-budget and simulated-feedback outputs record 50 of 60 complete requirements with 483 of 501 links and 48 of 60 with 482 of 501 respectively, but they are retained only as exploratory review-process sensitivity evidence because the originating row-level priority queue and sequential update history are not preserved in the clean archive. They are therefore not treated as confirmatory evidence in the primary conclusions. The results support a configuration-aware interpretation of trace recovery: cold-start, partial-trace, fixed-budget, and feedback-based recovery are different operating settings and should not be interpreted as one universal static ranking problem. For practitioners, the implication is that trace recovery should be treated as review support rather than automatic completion of the trace matrix, with analysts needing to know which evidence supports each candidate link and which requirements deserve deeper inspection. TraceRank supports analyst review, but it does not certify complete recovery or replace analyst judgment.

Subject of Research: TraceRank: configurable review support for requirements traceability recovery

Article Title: TraceRank: configurable review support for requirements traceability recovery

Article References: Alsawalqah, H., Abadleh, A., Alshamaileh, Y., Al-Tawil, M., Aljarah, I., & Alaydaa, M. S. (2026). TraceRank: configurable review support for requirements traceability recovery. Frontiers in Computer Science, 8, Article 1960635. https://doi.org/10.3389/fcomp.2026.1960635

Image Credits: AI Generated

DOI: 10.3389/fcomp.2026.1960635

Keywords: TraceRank, configurable, review, support, requirements, traceability, recovery, scientific research