The research, led by Vu-Linh Nguyen of the Université de technologie de Compiègne in France and Xuan-Truong Hoang of the Japan Advanced Institute of Science and Technology, together with Sébastien Destercke, Cassio de Campos and Van-Nam Huynh, recasts multi-label classification as a problem of learning order structures. Instead of treating each label as an independent yes-or-no question, the authors model the relationships between labels as preferences: one label may be strictly preferred to another, the two may be considered equivalent, or they may be incomparable. The result is a richer representation that, the authors argue, can be exploited to make multi-label learning both more accurate and more robust.

The core idea is a formal transformation. For any pair of labels, the possible joint outcomes form four categories: the first label is relevant and the second is not, the reverse, both are relevant, or neither is. A standard multi-label classifier predicts a binary vector, but the new framework maps that vector onto a relation between the two labels. A label vector in which one label is active and the other inactive becomes a strict preference between them; two active labels become an equivalence; and two inactive labels become another form of indifference or incomparability, depending on the chosen structure. Learning to classify instances then becomes learning to predict the most probable order structure, a problem the authors call Bayes-optimal order structure prediction.

This reframing is not merely cosmetic. Because order structures obey transitivity, they carry constraints that ordinary independent binary predictions lack. The authors show that finding the most probable preorder, an order structure that allows both strict preferences and equivalences, can be formulated as an integer linear programming problem. The transitivity conditions translate naturally into linear constraints, and the logarithm of the probability of an order structure decomposes into a sum over label pairs, making the optimization tractable with standard solvers. The framework also accommodates partial orders, which are more parsimonious but, as the authors demonstrate through careful examples, can lose crucial information by lumping together the cases where both labels are relevant and where neither is.

The distinction matters in high-stakes applications. The authors illustrate the point with a medical scenario involving two diseases encoded as two labels. A patient whose predicted joint outcome places both diseases as almost certainly present should clearly be sent for further diagnosis, while a patient in whom both are almost certainly absent should not. A framework based on partial orders treats these two situations as the same relation and would issue identical recommendations for both patients. The preorder representation, by contrast, distinguishes them, preserving exactly the kind of information that separates a useful clinical decision support system from a dangerous one.

The new approach also exposes weaknesses in an established preference-learning method for multi-label classification known as calibrated label ranking. That earlier technique ranks labels by pairwise comparison and uses a threshold, estimated from the expected number of irrelevant labels, to separate relevant from irrelevant ones. Through worked examples, the authors show that calibrated label ranking can misclassify the most probable preference relation in cases where the two labels are most likely equivalent, because its scoring scheme collapses the four joint outcome probabilities into a single pairwise score. The richer representation, which keeps all four probabilities, avoids these failures by construction.

On the theoretical side, the paper establishes which order structures are Bayes-optimal under different evaluation metrics. Under subset accuracy, the optimal prediction is simply the most probable order structure. Under Hamming-style accuracies, which score each pairwise relation independently, the optimal structure can be obtained pair by pair, a considerable computational simplification. The authors further extend their results to cost-sensitive variants, where different errors carry different penalties, and sketch how confusion-matrix-derived metrics such as the F-measure and the Jaccard index can be generalized to the order-structure setting, while noting that efficient optimization for those generalized metrics remains an open problem.

The empirical study covers ten publicly available multi-label datasets, with random forests and LightGBM as base learners to estimate the pairwise probabilities. The experiments focus on two regimes where order structures should shine. The first is robustness: the researchers artificially corrupted training labels at varying noise levels and with imbalanced classes, then measured how well each method predicted the true binary vectors. The second is partial abstention, where a classifier is allowed to withhold predictions on individual labels it is unsure about rather than guessing. In both regimes, the preorder-based classifiers consistently outperformed established baselines including binary relevance, classifier chains, calibrated label ranking and ensembles of classifier chains, particularly on the F-measure, a balanced metric that rewards both precision and recall.

The abstention results are especially striking. Allowing predictions with abstention led to impressive improvements in F-measure, and the preorder-based classifiers achieved competitive gains with significantly lower abstention rates than their partial-order counterparts. In other words, the richer structure not only made better predictions when it did predict, but also needed to hedge less often. Runtime experiments on the Enron email dataset, which has 53 labels, showed that the integer linear programming approach exhibits polynomial-like scaling in the number of labels, with per-instance prediction times of at most a few seconds when implemented with open-source solvers, a practical footprint attributable in part to the sparse constraint matrices of the underlying optimization problems.

The work, supported by funding from the French National Research Agency, the Office of Naval Research Global, the European Defence Fund and the Dutch Research Council, points toward a broader lesson for machine learning: sometimes the path to robustness lies not in bigger models or more data, but in choosing a representation whose structure matches the problem. By treating labels not as isolated switches but as elements of an ordered system, the researchers have shown that the rich mathematics of preferences and orders can absorb noise, tolerate imbalance and signal uncertainty in ways that conventional multi-label pipelines cannot. As datasets grow noisier and applications demand more honest predictions, that shift in perspective may prove increasingly hard to ignore.

Subject of Research: Transforming multi-label classification into order structure learning via preference learning to improve robustness against noisy and imbalanced labels and to enable partial abstention

Article Title: Robust Multi-label Classification via Preference Learning

Article References: Nguyen, V.-L., Hoang, X.-T., Destercke, S., de Campos, C., & Huynh, V.-N. (2026). Robust Multi-label Classification via Preference Learning. Machine Learning, 115(9), Article 211. https://doi.org/10.1007/s10994-026-07147-2

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07147-2

Keywords: multi-label classification, preference learning, order structures, robustness, noisy labels, imbalanced labels, partial abstention, Bayes-optimal prediction, integer linear programming, calibrated label ranking, machine learning, label ranking