The labeling problem sits at the heart of why streaming machine learning is so difficult. In classical supervised learning, algorithms are trained on datasets in which every example already carries a label, a process that is expensive and time-consuming but at least finite. In a data stream, new examples arrive continuously and without end, and if every one of them had to be sent to a human expert for annotation, the cost would quickly become prohibitive. Active learning offers a way out: instead of labeling everything, the system itself decides which incoming examples are worth the expense of an expert query, and which can be safely ignored or inferred. The art lies in choosing well. Query too often and the budget evaporates; query too rarely and the model drifts out of date, silently degrading as the world moves on.

Concept drift sharpens this dilemma considerably. When the underlying relationship between features and classes changes, a model trained on yesterday’s data may be actively misleading today. Drift can arrive suddenly, as when a network intrusion technique appears out of nowhere, or gradually, as customer behavior evolves season by season. Either way, the moments immediately after a drift begins are precisely when fresh labels are most valuable, and precisely when a passive system is least likely to seek them. Existing active learning methods for streams have developed various heuristics for detecting uncertainty and triggering queries, but many of them depend on parameters that must be tuned in advance, assumptions about the data that the designers of MLIDSC argue are rarely justified in real deployments where prior knowledge is scarce.

The second challenge, class imbalance, compounds the difficulty in a way that is particularly insidious in multiclass settings. In a binary problem with a rare class, resampling techniques and cost-sensitive learning have well-understood analogues. But when a stream contains many classes with wildly different frequencies, and when the identity of the rare classes can itself change over time, the problem becomes far more tangled. Rare classes are often the ones that matter most: the fraudulent transaction among millions of legitimate ones, the failing component signal buried in routine telemetry, the emerging disease pattern in a stream of ordinary diagnoses. A labeling strategy that samples uniformly across the stream will spend most of its budget on the abundant classes, leaving the rare ones underrepresented and the model blind to exactly the events it was built to catch.

MLIDSC, whose name abbreviates Multiclass Imbalanced Data Stream with Concept Drift, addresses these intertwined challenges with what the authors describe as a self-adaptive online labeling strategy. The key idea is that the decision to query an expert should be made based on the current context of the stream, rather than on fixed thresholds or pre-set parameters. The framework introduces an automated labeling mechanism that eliminates the need for prior parameter assumptions, allowing the system to calibrate its own behavior against the data as it actually arrives. This matters in practice because streaming deployments are often set up once and left to run for months or years, with little opportunity for the kind of manual retuning that offline machine learning pipelines take for granted.

The most distinctive technical contribution of the framework is a novel weighting scheme designed to prioritize informative minority-class data in nonstationary environments. The scheme combines two signals: the imbalance ratio, which captures how underrepresented a given class is at a given moment, and the importance of individual instances at that point in time. By fusing these two measures, MLIDSC directs its limited labeling budget toward the data points that are simultaneously rare and critical, rather than spreading attention evenly or relying on static class priors. In a stream where the balance of classes shifts as drift proceeds, this dynamic weighting allows the system to notice when a formerly abundant class is fading and a formerly rare one is rising, and to reallocate expert attention accordingly.

Another important design choice distinguishes MLIDSC from much of the prior literature: it processes data online, example by example, rather than in chunks. Many existing methods for drifting and imbalanced streams operate on fixed-size batches, accumulating a buffer of examples before making labeling and model-update decisions. Chunk-based processing simplifies some algorithmic questions, but the authors argue it is impractical for scenarios that demand continuous processing, where waiting for a buffer to fill introduces latency and can delay the detection of abrupt changes. By working in a truly online fashion, MLIDSC can react to each new example as it arrives, which is essential in applications such as network security or industrial monitoring where a delayed response is often as bad as no response at all.

The experimental evaluation behind the framework is notably comprehensive. The researchers tested MLIDSC on both real and synthetic data streams, with synthetic datasets generated using the Scikit-multiflow Python package and real datasets drawn from the Massive Online Analysis repository and the UCI machine learning repository, including the Statlog project data. Crucially, the experiments varied both the degree of concept drift and the imbalance ratio, allowing the team to probe how the method behaves across the full spectrum of difficulty that streaming environments present. The results reported in the paper show that MLIDSC achieves high accuracy while substantially reducing labeling costs compared to existing methods, a combination that is the central promise of active learning and the standard against which such frameworks are judged.

The implications reach well beyond the machine learning research community. Any organization that deploys predictive models on live data faces the twin pressures that MLIDSC targets: expert annotation is expensive, and the world refuses to hold still. Network traffic classification, one of the motivating applications in this line of research, is a vivid example. The mix of applications and protocols traversing a network changes constantly, and traffic classes are wildly imbalanced, with a handful of dominant services dwarfing the rare but security-critical flows that analysts most need to identify. A framework that keeps a classifier accurate in such an environment while asking human experts to label only a small, well-chosen fraction of the stream translates directly into operational savings and faster detection of emerging threats.

The work also fits into a broader and rapidly evolving research conversation. The references anchoring the study trace a decade of progress in the field, from early active learning methods for drifting streams developed by Žliobaitė and colleagues, through ensemble approaches that pair drift detection with resampling strategies, to recent work on multiclass imbalance by Liu, Li, Han and others. The authors of MLIDSC build on their own prior contributions as well, including hybrid labeling strategies and drift detection techniques based on outlier computation. What sets the new framework apart in this crowded landscape is its insistence on self-adaptation: rather than asking practitioners to guess at parameters before deployment, it derives its labeling decisions from the stream itself. As machine learning systems are increasingly entrusted with continuous, high-stakes decisions in environments that never stop changing, that kind of autonomy may prove to be not just a convenience but a necessity, and MLIDSC offers a concrete, experimentally validated template for how to achieve it.

Subject of Research: Online active learning for multiclass imbalanced data streams with concept drift

Article Title: MLIDSC: A self-adaptive online active learning framework for multiclass imbalanced data stream with concept drift

Article References: Halder, B., Hasan, K. M. A., & Ahmed, M. M. (2026). MLIDSC: A self-adaptive online active learning framework for multiclass imbalanced data stream with concept drift. Applied Intelligence, 56(15), Article 431. https://doi.org/10.1007/s10489-026-07473-x

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07473-x

Keywords: machine learning, active learning, concept drift, data streams, class imbalance, online learning, labeling strategy, multiclass classification, adaptive algorithms, streaming data, Applied Intelligence, MLIDSC

The motivation for the work lies in a structural weakness of earlier approaches. Traditional hate speech detectors, built on support vector machines, logistic regression, convolutional and recurrent neural networks, and eventually transformer models such as BERT, were trained on datasets that collapsed all forms of hate into one category. That conflation forces models to rely on surface-level lexical cues rather than deeper contextual meaning, and the consequences are uneven: hate directed at minority or rarely referenced groups gets misclassified far more often than hate against well-represented targets. Existing multiclass datasets offered some improvement, but most were small, inconsistently annotated, or still failed to model the target of abuse explicitly, creating a persistent trade-off between scale, label granularity, and annotation quality.

To test what large language models could add, the team assembled a dataset of 12,957 English tweets, built by re-annotating the hate speech subset of the widely used TweetEval benchmark. Of these, 5,455 samples carry one of five target-specific hate labels and 7,502 are non-hateful. Three independent annotators labeled each instance, with a fourth resolving disagreements, achieving a moderate inter-annotator agreement of 0.6003 on Krippendorff’s alpha, a figure the authors note reflects the inherent subjectivity of judging hateful content. Because some original categories were too sparse to train on reliably, the researchers consolidated them: threats and violence, with only 106 instances, was merged into profanity and general abuse, while hate toward countries, with just 18 samples, was folded into racial and ethnic hate. The result is a six-class problem that mirrors the imbalance of real platforms, where non-hate content makes up nearly 58 percent of the data and religious hate speech barely 1.3 percent.

Five models with distinct architectures and training histories were put through their paces: DeepSeek-R1-Distill-Qwen-32B, Mistral-7B-Instruct, WizardLM-13B, Phi-4-mini, and OpenAI’s GPT-4o, alongside a fine-tuned BERT-base-uncased baseline. Each was evaluated three ways: zero-shot prompting, few-shot prompting with six labeled examples, and supervised fine-tuning. The open-source models were adapted using QLoRA, a parameter-efficient technique that freezes the pretrained weights and trains only small low-rank adapter matrices inserted into each transformer block, with the base model loaded in 4-bit precision to fit on a single NVIDIA A100 GPU. GPT-4o was fine-tuned through OpenAI’s API at a total training cost of 85.75 dollars.

The zero-shot results were sobering. Every model struggled, with macro F1 scores, which weight all classes equally and thus expose weakness on minority categories, remaining low across the board. GPT-4o led with a macro F1 of 0.4503, while WizardLM, which had performed well in an earlier benchmark, scored close to zero on this dataset. More troubling still, several models refused to stay within the label space: WizardLM produced 1,574 unclassified outputs, including refusals, reasoning artifacts, and requests for context, while Mistral managed only four. Few-shot prompting helped some models, improving Phi and Mistral, but actually degraded DeepSeek and GPT-4o, demonstrating that in-context examples alone do not guarantee better behavior on complex classification tasks.

Fine-tuning changed the picture dramatically. The fine-tuned DeepSeek model achieved the best multiclass performance of any system tested, with macro and weighted F1 scores of 59.91 and 76.71 percent respectively, and it eliminated unclassified outputs entirely. Mistral followed closely with a weighted F1 of 74.12 percent, while fine-tuned GPT-4o, despite its strong showing in prompting settings, reached only 61.54 percent. The replicated BERT baseline remained competitive at 70.89 percent weighted F1 but was clearly outperformed by the best LLM. Against a suite of replicated classical machine learning and deep learning baselines, including logistic regression, naive Bayes, CNN, and BiLSTM models, the fine-tuned DeepSeek improved weighted F1 by margins ranging from 2.59 to 29.5 percentage points. McNemar statistical tests confirmed the differences were significant against BERT, BiLSTM, and logistic regression.

Crucially, the team did not stop at same-dataset evaluation, a common weakness in prior work. They assessed the best DeepSeek model on a remapped subset of the external ETHOS benchmark, aligning its gender, religion, and race labels with the study’s taxonomy and yielding 789 evaluation instances. The model scored an F1 of 88.31 percent on this external test, with no unclassified outputs, evidence of genuine cross-dataset generalization. A control experiment sharpened the interpretation: when a DeepSeek model was fine-tuned directly on ETHOS, it performed well on ETHOS itself but dropped sharply when tested on the original multiclass dataset, suggesting that the multiclass-trained model had learned richer, more transferable linguistic patterns rather than dataset-specific shortcuts.

The gains came with caveats. Per-class analysis revealed that minority categories remained hard: racial and ethnic hate speech and religious hate speech, each under two percent of the data, achieved F1 scores of only 0.4211 and 0.5098 respectively, while immigration and xenophobic hate, the best-annotated class, reached 0.7624. Confusion matrices showed the dominant error was not mixing up hate categories but distinguishing hate from non-hate at all, particularly for implicit or context-dependent hostility that lacks overt slurs. Computational cost also matters: the DeepSeek model required 4.71 hours of training and 32.5 minutes to classify the 1,620-tweet test set, whereas logistic regression completed both in seconds. The authors frame this as a practical trade-off, recommending LLMs where fine-grained discrimination justifies the expense and simpler classifiers for high-throughput, resource-constrained settings.

The study also confronts the ethics of deployment. False positives can suppress counter-speech, satire, or educational discussion, and may disproportionately silence the very communities these systems claim to protect, while false negatives leave targeted groups exposed. The authors stress that model predictions should never serve as the sole basis for content removal or user sanctions, and they call for human oversight, transparent governance, and periodic auditing. Looking forward, they outline plans to extend the framework to multilingual settings, adopt multi-label formulations for posts targeting multiple groups, and incorporate community-centered evaluation. The fine-tuned model and dataset have been released publicly, inviting the research community to build on a benchmark that finally asks not just whether a post is hateful, but whom it harms.

Subject of Research: Fine-tuning large language models for target-specific multiclass hate speech detection on social media

Article Title: Fine-tuning large language models for binary and multiclass target-specific hate speech detection

Article References: Kaddoura, S., & Al-Kohlani, S. (2026). Fine-tuning large language models for binary and multiclass target-specific hate speech detection. Discover Artificial Intelligence, 6(1), Article 1296. https://doi.org/10.1007/s44163-026-02354-1

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02354-1

Keywords: large language models, hate speech detection, fine-tuning, QLoRA, DeepSeek, BERT, multiclass classification, content moderation, natural language processing, social media, ETHOS dataset, class imbalance