A team led by Linh Do Nhat, Dzi Lam Tran Tuan and Vinh Truong Hoang has now published EmbedKD, an open-source Python toolkit designed specifically for distilling metric embeddings, in the journal SoftwareX. The software, released under the MIT license and available on GitHub, deliberately contributes no new distillation objective. Instead, its authors argue, the real gap in the field is the lifecycle around the methods: the diagnostics that reveal whether a teacher–student pair is compatible before training, the retrieval protocols that measure whether distillation actually helped, and the deployment checks that verify the model leaving the pipeline still behaves like the one that was trained. Every component, they note, could be rebuilt inside existing frameworks with moderate effort; what EmbedKD provides is that workflow, verified stage against stage.

The distinction matters because existing tools were built for a different problem. Frameworks such as torchdistill, mdistiller and KD-Lib offer config-driven pipelines, official implementations and combinations with pruning and quantization, but none ships a retrieval protocol with mean average precision and recall metrics, a gallery-query evaluation, pre- and post-training diagnostic reports, or a deployment benchmark with a parity check. Retrieval-oriented distillation methods themselves are well developed, from multi-view distillation for re-identification to pairwise-difference relational distillation, but each is released as code accompanying a single paper’s experiments. EmbedKD instead wraps the process in a six-command lifecycle: diagnose, fit, eval, extract, deploy and reproduce, each of which can be run from the command line or through a shared Python API that guarantees the two paths produce identical numbers.

Reproducibility is treated as a hard requirement rather than an aspiration. Every execution records its full state, including configuration, random seed, software versions, code commit and hardware, and all published results trace back to these logs. The reproduction command grades its output against tolerance-checked expected values and exits with a distinct error code on any metric drift, allowing continuous-integration systems to gate on it. In one demonstration, retraining the reference experiment returned results identical to the original run in every digit rather than merely within tolerance. The released version is archived on Zenodo with checksums for every checkpoint, and the reference environment, down to driver and package versions, ships with the repository.

Architecturally, the toolkit rests on three decisions. A registry system lets users add distillation objectives, task losses and dataset adapters as first-class citizens of the configuration file without touching the core. A single declarative configuration schema merges defaults, YAML files and command-line overrides into one validated, fail-fast configuration that is dumped verbatim into the run fingerprint. And an explicit backbone policy distinguishes five validated teacher and student architectures tested in continuous integration, nineteen models that pass smoke tests, and any other timm backbone, which runs only under an explicit experimental flag with a warning rather than a silent claim of support.

The toolkit also surfaces numerical subtleties that are easy to get wrong. Because the supported objectives operate on L2-normalized embeddings, mean squared error and cosine distance are the same function up to a constant, but the reduction over the embedding dimension means that at a dimensionality of 512, the same nominal weight makes the MSE distillation term 256 times weaker than the cosine term. Experiments on the CUB-200-2011 bird dataset confirmed the algebra: at the nominal weight the MSE objective changed nothing, while at the scale-matched weight it matched cosine distillation almost exactly. The toolkit prints the effective weight of each objective when a run starts, computes all losses in fp32 outside the autocast region because geometric losses underflow in fp16 and silently freeze training, and skips and counts non-finite loss components rather than poisoning a training step.

Perhaps the most striking findings come from the diagnostics themselves, which the authors report with unusual candour. In a controlled series where only teacher quality varied, a teacher weaker than the standalone student cost the student 0.023 mAP while its representation became more aligned with the teacher, a failure mode the team calls aligned-but-worse. The sign of the distillation effect changed near a teacher–student gap of 0.02 to 0.03 and saturated after roughly 0.05. But across different student architectures the gap failed to predict the magnitude of the gain, and a stronger ConvNeXt-Tiny teacher yielded smaller gains than ResNet50 for all three students tested. Most sobering, the pre-distillation risk label, intended to warn of harmful pairs, read LOW on every rung of the controlled series including the harmful one, and its underlying similarity score shifted enough between probe splits to cross its own threshold.

The cross-domain results were equally cautionary. When students trained on one dataset were evaluated on another without fine-tuning, distillation hurt in both directions tested: a CUB-trained ResNet18 lost 0.0074 mAP on Cars196, and a Cars-trained MobileNetV3 lost 0.0320 on CUB-200, both far outside the seed spread, even though both pairs had gained in domain. The distillation term, the authors suggest, pulls the student toward a teacher fitted to the source domain, and the pull that helps within a domain costs generality when the domain changes. For the deployment scenario, however, the news was better: a distilled MobileNetV3 kept 72.4 percent of its teacher’s Recall@1 while being 5.7 times smaller and eight times faster on CPU, with ONNX export verified by a mandatory parity check that recomputes retrieval metrics through the exported graph.

The authors are explicit about the limits of their evidence base: four demonstrations, one controlled series and one six-pair grid, all on a single machine, with several comparisons resting on single runs. The toolkit targets single-GPU image workloads, and the shipped training schedule trains past the best checkpoint for every backbone measured, requiring validation-based checkpoint selection. Yet the broader message is one the field is increasingly willing to hear. Whether a given embedding pair is worth distilling is today settled by trial and error, one configuration at a time; EmbedKD makes it cheap to settle by measurement instead, turning questions usually answered by folklore into routine, reproducible experiments, and, just as importantly, documenting where its own instruments cannot see.

Subject of Research: A reproducibility-first open-source toolkit for knowledge distillation of metric embeddings in image retrieval.

Article Title: EmbedKD: a reproducibility-first toolkit for knowledge distillation of metric embeddings

Article References: Do Nhat, L., Lam Tran Tuan, D., & Truong Hoang, V. (2026). EmbedKD: a reproducibility-first toolkit for knowledge distillation of metric embeddings. SoftwareX, 36, Article 103029. https://doi.org/10.1016/j.softx.2026.103029

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103029

Keywords: knowledge distillation, metric embeddings, image retrieval, reproducibility, model compression, open-source software, deep metric learning, teacher-student networks, ONNX deployment, negative transfer, representation similarity, fine-grained recognition

The field of evolutionary multitasking grew out of the recognition that many real-world optimization tasks are interrelated. Rather than running a separate evolutionary algorithm for every problem, multitasking algorithms solve several tasks simultaneously within a single framework, allowing candidate solutions to migrate between task-specific populations. When the tasks are similar, this exchange can dramatically accelerate convergence, because a solution that performs well on one problem may already be halfway to a good solution on another. The foundational work on multifactorial evolution by Gupta, Ong, and Feng in 2016 demonstrated the promise of this approach, and a growing body of research has since refined how knowledge moves between tasks.

The catch is negative transfer. When two tasks differ substantially in the structure of their search spaces or the shape of their objective functions, blindly importing solutions from a source task can pull a target population away from its own promising regions. The result is wasted computational effort and, in the worst cases, worse final solutions than a task would achieve on its own. Researchers have proposed a variety of remedies, from adaptive transfer probabilities based on population distribution statistics to explicit mapping techniques that translate solutions between task domains. The new algorithm, called MOMFEA-MTL, combines two complementary mechanisms to attack the problem from both directions: deciding when to transfer, and deciding how to transfer.

The first mechanism is an adaptive knowledge transfer strategy that continuously monitors the historical evolution of each population. Instead of fixing the probability that a solution crosses from one task to another, the algorithm adjusts that probability dynamically based on how well past transfers have served the target task. If incoming solutions have recently improved the target population’s performance, the transfer probability rises; if they have degraded it, the probability falls. This feedback loop suppresses negative transfer without requiring any prior knowledge of how similar the tasks are, which is precisely the information that is hardest to obtain in practical settings where the geometry of the search landscape is unknown.

The second mechanism tackles the question of how solutions should be transformed when they do cross between tasks. Naive approaches simply copy decision variables from one task’s representation to another, an operation that only makes sense when the tasks share a common search space structure. MOMFEA-MTL instead constructs a mapping matrix using a manifold transfer method, an approach rooted in the observation that high-dimensional data often lie on or near a low-dimensional manifold. By learning a transformation that preserves the intrinsic geometric structure of the source population while aligning it with the target task’s distribution, the algorithm can move solutions across task boundaries in a way that respects the underlying shape of each problem’s search space.

Selecting which solutions deserve the cost of this transformation is handled by a K-means solution selection strategy. Clustering the population into groups and choosing representative solutions from those clusters ensures that the transferred knowledge captures the diversity of the source task rather than a narrow sample of its best-performing region. This matters because multi-objective optimization does not seek a single best solution but an entire Pareto front of trade-off solutions, and preserving spread across the front is as important as pushing toward it. Once selected, the solutions are mapped from the source task to the target task, where they join the target population and accelerate its evolution.

The combination is designed for multi-objective multitasking problems, where each task involves optimizing several conflicting objectives simultaneously. This setting compounds the difficulty of knowledge transfer: not only must solutions be useful, but the distribution of trade-offs must also remain balanced. The authors position their approach within the broader lineage of multiobjective multifactorial optimization, building on the original MO-MFEA framework and its successors such as MO-MFEA-II, which introduced cognizant multitasking, and on explicit transfer methods like those based on autoencoding and transfer component analysis. Manifold transfer learning itself has precedent in dynamic multiobjective optimization, where it was used to predict how Pareto sets shift as problems change over time; the new work adapts the idea to the multitasking setting, where the shift is between tasks rather than between time steps.

To evaluate the method, the team ran experiments on nine classical multi-objective multitasking test functions, comparing MOMFEA-MTL against established baselines from the literature. The results showed that the proposed algorithm achieved competitive performance across the benchmark suite, with the adaptive transfer strategy and manifold-based mapping working together to deliver gains where task relatedness could be exploited while limiting damage where it could not. The authors report that the algorithm demonstrates good competitiveness on the test problems, supporting the central claim that combining adaptive transfer control with structure-preserving mapping is an effective recipe for multitasking optimization.

The practical implications extend beyond benchmarks. Evolutionary multitasking has already been applied to problems such as vehicle routing with occasional drivers, sparse reconstruction, multi-task learning for modular learning machines, and, notably by members of the same group, the optimization of steel rolling schedules in industrial production. In each of these domains, multiple related optimization problems arise naturally, and the cost of solving them one at a time is substantial. An algorithm that can reliably harvest the similarities between tasks while shielding itself from their differences could translate into measurable savings in computation time and solution quality, particularly for expensive simulations where each function evaluation carries real cost.

The research also contributes to a conceptual shift in how the field thinks about transfer itself. Early multitasking algorithms treated knowledge transfer as a fixed structural feature, with a constant probability of inter-task mating or a static mapping between search spaces. The trend, exemplified by self-regulated multitasking, adaptive transfer based on population distributions, and transfer rank methods, is toward algorithms that learn from their own experience which transfers help. MOMFEA-MTL fits squarely in this tradition, adding the geometric perspective of manifold learning to the toolkit. As optimization problems in industry and science grow larger and more entangled, the ability to solve many tasks at once, safely and efficiently, may prove to be one of evolutionary computation’s most valuable exports, and this work offers a carefully engineered step in that direction.

Subject of Research: A multi-objective multitasking evolutionary optimization algorithm using adaptive knowledge transfer and manifold transfer learning to mitigate negative transfer.

Article Title: Multi-objective multitasking optimization based on manifold transfer learning

Article References: Zhang, K., Cheng, Y., Wang, S., Sun, H., Wei, L., & Hu, Z. (2026). Multi-objective multitasking optimization based on manifold transfer learning. International Journal of Machine Learning and Cybernetics, 17(9), Article 458. https://doi.org/10.1007/s13042-026-03300-4

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03300-4

Keywords: evolutionary computation, multitasking optimization, multi-objective optimization, knowledge transfer, manifold learning, negative transfer, Pareto front, K-means clustering, evolutionary algorithms, transfer learning, optimization algorithms, machine learning