A comprehensive open-access survey published in the journal Vicinagearth by Yong Luo, Yonggang Wen, Han Hu, Bo Du, Ling-Yu Duan, and Dacheng Tao now offers the most systematic map to date of a rapidly growing answer to that weakness: transfer metric learning, or TML. The core idea is deceptively simple. If a machine cannot learn a good distance metric in a new domain because labeled examples are few, why not borrow the knowledge embedded in metrics already learned from related domains where data was plentiful? Just as a student who has mastered distinguishing horses from cats can more easily learn to tell zebras from tigers from only a handful of examples, a machine that has internalized a metric separating horses from cats can adapt that metric to separate zebras from tigers with far less supervision. The survey, published on 25 June 2024, traces more than a decade of work in this field, organizes dozens of algorithms into a coherent taxonomy, catalogs their applications, and lays out the open problems that will define the next decade of research.

The mathematical foundations of the field are worth understanding, because they explain both the power and the fragility of the approach. In the classical formulation, a linear distance metric is represented by a positive semi-definite matrix, often denoted A, which transforms the familiar Euclidean distance into a Mahalanobis distance that can weight different feature dimensions differently and capture correlations between them. With enough labeled data, or enough weakly supervised constraints in the form of similar and dissimilar pairs, or triplets stating that one sample is more similar to a second than to a third, optimization algorithms can carve out a geometry of the data space that makes even simple classifiers, such as k-nearest neighbors or k-means clustering, perform remarkably well. But when the labeled data in a new target domain is insufficient, the learned matrix overfits, and the resulting geometry is unreliable. Transfer metric learning intervenes precisely at this point, injecting information from source domains where supervision was abundant.

The survey’s central contribution is a clean taxonomy along three axes. First, methods are divided by feature setting into homogeneous TML, where source and target domains share the same feature space and differ only in data distribution, and heterogeneous TML, where the feature spaces themselves differ, sometimes with a substantial semantic gap between them. Second, methods are classified by label availability into inductive TML, where a few labeled target samples exist; transductive TML, where none do but the unlabeled target data is accessible during training; and unsupervised TML, where no labels exist anywhere. Third, and most technically revealing, methods are grouped by transfer strategy into four families: metric approximation, distribution approximation, subspace approximation, and distance approximation. Each family embodies a different philosophical bet about what, exactly, should travel between domains.

The metric approximation family, which includes some of the earliest TML work dating to around 2009, makes the most direct bet: it forces the target metric to stay close to pre-trained source metrics. In one influential formulation, the target Mahalanobis matrix is regularized toward multiple source matrices using the LogDet divergence, a scale-invariant measure of the difference between positive definite matrices, with adaptive weights learned to reflect how much each source metric should contribute. A cleverer variant sidesteps the burden of learning a full matrix altogether by decomposing the target metric as a weighted combination of base metrics derived from the eigenvectors of the source metrics. This transforms metric learning into a far smaller coefficient-learning problem, dramatically reducing the number of parameters to estimate when target labels are scarce, and it automatically satisfies the positive semi-definiteness constraint that would otherwise be computationally costly to enforce. Related multi-task approaches, such as the multi-task extension of the well-known large margin nearest neighbor algorithm, assume every domain metric decomposes into a shared common metric plus a task-specific correction, allowing all domains to help each other simultaneously.

Subspace and distance approximation strategies take a different route, motivated by a practical problem: when feature dimensions are high, directly learning a full metric matrix invites overfitting and expensive computation. Instead, these methods factorize the metric as a product of a low-rank transformation matrix with its own transpose, effectively projecting all domains into a shared lower-dimensional subspace where knowledge transfer happens. One notable scalable method, CP-mtML, assumes the squared distance function itself splits into a common part and a task-specific part, and it can be optimized efficiently with stochastic gradient descent, making it viable for large-scale face retrieval over high-dimensional features. The trade-off is mathematical: these formulations are typically non-convex, so only local optima can be guaranteed, a limitation that later work partially addressed with convex alternatives and efficient online algorithms.

Distribution approximation, by contrast, attacks the root cause of the domain gap directly. Since the main difference between homogeneous domains is that their data distributions diverge, these methods learn a nonlinear feature mapping that simultaneously satisfies the source domain’s similarity constraints and minimizes a statistical measure of the distribution difference between source and target samples in the transformed space. The classic instantiation uses maximum mean discrepancy, computed in a reproducing kernel Hilbert space, while its deep learning successor, deep transfer metric learning, replaces the kernel mapping with a multi-layer neural network and conducts the alignment at every hidden layer. Later refinements went further, reducing not just the marginal distribution difference but also the conditional distribution divergence by assigning pseudo-labels to target samples and aligning class-wise distributions, and recent adversarial methods have introduced techniques from generative adversarial networks to align domains even when source and target label spaces do not overlap. A complementary line of work imports importance sampling from covariate shift theory, weighting each source training pair by how likely it would be to appear in the target domain, so that source samples resembling target data contribute most to the target metric.

Heterogeneous TML, where the feature spaces themselves differ, is younger and smaller, but arguably more consequential for modern applications. Its flagship scenario is cross-lingual sentiment analysis: labeled English reviews are plentiful, labeled Spanish reviews are scarce, and the vocabularies of the two languages make their representations fundamentally incompatible, so no source metric can be applied directly. The dominant strategy is subspace approximation, finding a common representation into which all domains project, often exploiting large amounts of unlabeled data that has representations in every domain, with tensor-based regularizers capturing high-order correlations across domains. A particularly elegant framework extracts knowledge fragments, linear or nonlinear mappings, from a pre-trained source metric and forces the target metric to agree with those fragments on unlabeled corresponding data, a strategy the authors connect to Vladimir Vapnik’s learning using privileged information. Even more striking is the unsupervised variant known as metric imitation, which uses sophisticated, expensive features, such as deep convolutional representations, to teach a metric for cheap, fast features, with applications ranging from image super-resolution to efficient retrieval.

The breadth of applications documented in the survey is a testament to how general the problem is. Beyond face verification and person re-identification across camera networks, where illumination, background, and camera settings shift distributions constantly, TML has powered speech recognition across speaker groups, citation and social circle prediction in networks, customer behavior prediction in insurance, device fingerprinting for wireless security, protein function prediction, time series forecasting for air pollution and stock markets, remaining-useful-life estimation for bearings under new operating conditions, galaxy morphology characterization, EEG-based emotion recognition, and drug discovery. Benchmark evaluations on standard datasets such as Office-Caltech, which spans Amazon product images, webcam photos, digital SLR images, and Caltech categories, and the cross-lingual sentiment dataset spanning English, French, German, and Japanese reviews, confirm the survey’s empirical conclusions: metric learning helps even with scarce labels, transfer helps more, and intriguingly, transductive methods with zero target labels can sometimes beat inductive ones because they exploit the structure of the actual test data.

Perhaps the most valuable part of the survey is its candid account of what the field still cannot do. Selective transfer remains unsolved: most algorithms assume source knowledge is beneficial, yet poorly related sources cause negative transfer, and the authors argue that future systems should exploit rather than merely avoid it, possibly through transferability metrics or hypothesis-space-level selection. Theory lags practice, with generalization bounds existing only for narrow cases, leaving open the fundamental question of when and how source knowledge helps. Most methods still learn linear metrics, inadequate for the nonlinear structure of visual data, and the field has barely begun to address streaming data, lifelong adaptation, binary and hashed metrics for big-data retrieval, one-shot and zero-shot extremes, and open-world settings. The authors close with a provocative observation: large pre-trained models such as CLIP, which learn generalizable visual representations from vast numbers of unlabeled image-text pairs, are in spirit doing unsupervised heterogeneous transfer metric learning at planetary scale, suggesting that the future of the field may lie in fusing its principled taxonomy with the raw power of multi-modal foundation models. For a technique that is, at its heart, about teaching machines to borrow what they cannot learn alone, that convergence may be the most exciting transfer of all.

Subject of Research: Transfer metric learning methods for improving distance metric learning across domains with scarce labeled data

Article Title: Transfer metric learning: algorithms, applications and outlooks

Article References: Luo, Y., Wen, Y., Hu, H., Du, B., Duan, L.-Y., & Tao, D. (2024). Transfer metric learning: algorithms, applications and outlooks. Vicinagearth, 1(1), Article 2. https://doi.org/10.1007/s44336-024-00003-8

Image Credits: AI Generated

DOI: 10.1007/s44336-024-00003-8

Keywords: transfer metric learning, distance metric learning, transfer learning, machine learning, domain adaptation, Mahalanobis distance, deep learning, computer vision, person re-identification, cross-lingual classification, maximum mean discrepancy, survey