<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>QLoRA &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/qlora/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Thu, 08 Oct 2026 08:30:30 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>QLoRA &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>AI Learns to Diagnose Grapevine Diseases Like a Plant Pathologist</title>
		<link>https://scienmag.com/ai-learns-to-diagnose-grapevine-diseases-like-a-plant-pathologist/</link>
		
		<dc:creator><![CDATA[Alan Morgan]]></dc:creator>
		<pubDate>Thu, 08 Oct 2026 08:30:30 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[AI systems for foliar disease detection]]></category>
		<category><![CDATA[AI-powered grapevine disease identification]]></category>
		<category><![CDATA[artificial intelligence in plant pathology]]></category>
		<category><![CDATA[convolutional neural networks]]></category>
		<category><![CDATA[cross-attention]]></category>
		<category><![CDATA[domain shift]]></category>
		<category><![CDATA[explainable AI]]></category>
		<category><![CDATA[explainable AI for agriculture]]></category>
		<category><![CDATA[grapevine disease]]></category>
		<category><![CDATA[grapevine disease detection]]></category>
		<category><![CDATA[hybrid framework for plant disease diagnosis]]></category>
		<category><![CDATA[improving accuracy in plant disease diagnosis]]></category>
		<category><![CDATA[LLaVA]]></category>
		<category><![CDATA[overcoming CNN limitations in plant health]]></category>
		<category><![CDATA[plant disease classification using deep learning]]></category>
		<category><![CDATA[plant pathology]]></category>
		<category><![CDATA[PlantVillage]]></category>
		<category><![CDATA[precision agriculture]]></category>
		<category><![CDATA[QLoRA]]></category>
		<category><![CDATA[real-time vineyard disease monitoring]]></category>
		<category><![CDATA[saliency auto-crop]]></category>
		<category><![CDATA[transparent AI models for farmers]]></category>
		<category><![CDATA[vision-language models]]></category>
		<category><![CDATA[vision-language models for crop health]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=246834</guid>

					<description><![CDATA[Researchers in Vietnam have built a hybrid vision-language AI that diagnoses grapevine diseases with 98.7 percent accuracy while explaining its reasoning and resisting the background overfitting that cripples conventional CNNs in real fields.]]></description>
										<content:encoded><![CDATA[<p>Grapevines are among the most economically valuable crops on Earth, and they are under constant assault from foliar diseases such as Black Rot, ESCA, and Leaf Blight. For generations, detecting these threats has depended on the trained eye of agricultural experts, a process that is slow, expensive, and vulnerable to human subjectivity. Now, a pair of researchers from the Data Science Laboratory at the Industrial University of Ho Chi Minh City in Vietnam has unveiled an artificial intelligence system that not only diagnoses grapevine diseases with remarkable accuracy but also explains its reasoning in plain language. The study, published in Discover Artificial Intelligence, describes a Global–Local Hybrid Framework built on a vision-language model that mimics how a human pathologist actually works: first surveying the whole leaf, then zooming in on the lesion with a magnifying glass.</p>
<p>The team, Lang Hoang Son and Bui Thanh Hung, chose to move beyond the convolutional neural networks (CNNs) that have dominated plant disease classification for years. CNNs such as ResNet and EfficientNet can achieve near-perfect scores on curated laboratory datasets, but they suffer from two fundamental weaknesses. They extract raw pixel-level features without high-level semantic reasoning, and they operate as inscrutable black boxes, offering farmers a class label with no justification. Worse, researchers have documented the so-called Clever Hans phenomenon, in which CNNs quietly learn to recognize background artifacts—soil, lighting, uniform laboratory backdrops—rather than the biological morphology of the disease itself. When such models confront real-world field images, their performance can collapse.</p>
<p>Vision-language models (VLMs) such as LLaVA, which combine a CLIP image encoder with a large language model backbone, promised a way out by generating human-readable diagnostic rationales. But they brought their own problem: the attention sink phenomenon. When fed complex real-world photographs, these models squander computational attention on noisy background elements—rocks, shadows, variable illumination—instead of focusing on the microscopic necrotic lesions that actually matter. The Vietnamese team&#8217;s solution was architectural: give the model two views of every leaf at once, one global and one local, so that attention is anchored to genuine pathology.</p>
<p>The local view is produced by an unsupervised Saliency Auto-Crop algorithm that requires no bounding box annotations, a major practical advantage because manual annotation is prohibitively expensive at agricultural scale. The algorithm first converts the input image into the CIE-LAB color space, decoupling luminance from chrominance. This matters because the a* channel is especially sensitive to the green-red spectrum, allowing necrotic brown lesions to stand out against green leaf tissue regardless of lighting conditions. A weighted saliency score is computed across the image, then Otsu&#8217;s adaptive thresholding and mathematical morphological operations filter out background noise. Finally, image moments are used to calculate the centroid of the isolated lesion, and a 128-by-128-pixel patch is cropped around it. The researchers empirically tested the patch size: crops of 64 pixels risk truncating diagnostic transition zones such as chlorotic halos, while 224-pixel crops capture so much healthy tissue that the patch becomes redundant with the global view.</p>
<p>These two images—a 256-pixel global view and the 128-pixel lesion patch—are encoded by a pre-trained CLIP ViT-L/14 encoder into 576 visual tokens each, projected into the language model&#8217;s embedding space, and concatenated with a carefully engineered text prompt that explicitly tells the model which image is which. The prompt instructs the system to identify exactly one of four categories: Black Rot, ESCA, Healthy, or Leaf Blight. Fine-tuning the 7-billion-parameter LLaVA-1.5 model conventionally would demand enormous computational resources, so the team applied 4-bit QLoRA, a quantized low-rank adaptation technique. The pre-trained weights are frozen and quantized into a 4-bit NormalFloat format, and lightweight trainable adapter matrices are injected into the decoder layers. Only 7.68 percent of the total parameters were updated, allowing the entire training run to complete on free cloud GPUs—an NVIDIA Tesla T4 pair with 32 gigabytes of memory—while the model acquired specialized plant pathology knowledge.</p>
<p>The results on the PlantVillage grapevine dataset, comprising 9,027 images split into training, validation, and blind test sets, were striking. The hybrid framework achieved an overall accuracy of 98.7 percent on the blind test, with Leaf Blight recognized perfectly at 100 percent, Healthy leaves at 99.8 percent, ESCA at 98.1 percent, and Black Rot at 97.0 percent—the last slightly hampered by the visual resemblance between early black rot lesions and leaf blight. An ablation study confirmed that both image streams are indispensable: the global image alone reached 97.8 percent, the local patch alone just 90.3 percent, and the un-tuned zero-shot model a dismal 18.6 percent. The QLoRA rank mattered too, with accuracy climbing from 62.1 percent at rank 8 to 98.0 percent at rank 32 before the team settled on rank 64 for maximum stability.</p>
<p>But the most consequential experiment was the one designed to expose the Clever Hans effect. The team evaluated all models in a strictly zero-shot manner on PlantDoc, a dataset of grapevine images captured in uncontrolled field conditions with complex backgrounds and volatile lighting. Advanced CNNs that had scored near 100 percent in the laboratory plummeted: EfficientNet-B4 fell from 99.8 percent to 54.9 percent, a catastrophic domain gap of 44.9 percentage points. The hybrid VLM framework, by contrast, retained 80.5 percent accuracy overall, with healthy leaf recognition peaking at 98.6 percent, though Black Rot degraded to 60.9 percent. The researchers attribute this resilience to the saliency-cropped local patches acting as semantic anchors that prevent the model from being derailed by environmental noise. The framework also scored 98.55 percent on the external AgroMind benchmark and a perfect 100 percent on AgroCoT, both laboratory-style datasets with subtle domain shifts.</p>
<p>Interpretability was engineered into the system rather than bolted on afterward. Instead of relying on post-hoc explanation tools like Grad-CAM, which merely visualize local gradients, the team extracted cross-attention matrices directly from the 31st and final decoder layer of the language model—the layer of highest semantic abstraction, immediately preceding the output head. A forward hook captures the mean attention weights that text tokens assign to visual tokens, which are then upsampled by bilinear interpolation and overlaid on the original image as heatmaps. Layer-wise analysis showed that shallow layers produce sparse, texture-focused activations, while deeper layers concentrate attention on pathological regions. Quantitatively, the model allocated roughly 73 percent of its attention to the global image and 27 percent to the local patch, a ratio that held steady whether the leaf was diseased or healthy—a Welch&#8217;s t-test found no significant difference between the two conditions, with a negligible effect size—indicating a stable, balanced diagnostic strategy rather than arbitrary attention spikes.</p>
<p>The authors are candid about the limitations. Inference is slow: the fine-tuned LLaVA requires 2,231 milliseconds per image, or 0.448 frames per second, and consumes 149 joules per image on a Tesla T4, making real-time deployment on drones or mobile devices impractical for now. Black Rot&#8217;s scattered, multi-spot lesions sometimes defeat the single-patch cropping strategy, motivating future multi-scale, multi-crop attention mechanisms. And the residual 17-point domain gap on field imagery shows that laboratory overfitting has not been fully eliminated. The team&#8217;s roadmap includes knowledge distillation to compress the model&#8217;s reasoning into lighter architectures, deeper quantization and pruning for on-device diagnostics, and evolving the static CIE-LAB extraction module into an autonomous visual agent that can propose and refine its own lesion bounding boxes through iterative reasoning.</p>
<p>Even with those caveats, the study marks a meaningful shift in agricultural AI. Where previous systems chased marginal accuracy gains on sterile benchmarks, this framework trades a sliver of in-domain performance for transparency, natural-language reasoning, and genuine cross-domain robustness. For vineyard managers, the difference is tangible: instead of an opaque percentage, they receive a diagnosis, a rationale, and a heatmap showing exactly which lesion triggered the decision. As vision-language models continue to mature, the grapevine may become the proving ground for a new generation of AI systems that agriculture can actually trust.</p>
<p><strong>Subject of Research:</strong> Interpretable vision-language AI for grapevine disease diagnosis</p>
<p><strong>Article Title:</strong> A global-local hybrid vision-language framework for interpretable grapevine disease diagnosis</p>
<p><strong>Article References:</strong> A global-local hybrid vision-language framework for interpretable grapevine disease diagnosis. (n.d.). <a href="https://doi.org/10.1007/s44163-026-02463-x" rel="noopener noreferrer">https://doi.org/10.1007/s44163-026-02463-x</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44163-026-02463-x" rel="noopener noreferrer">10.1007/s44163-026-02463-x</a></p>
<p><strong>Keywords:</strong> vision-language models, grapevine disease, plant pathology, explainable AI, LLaVA, QLoRA, saliency auto-crop, precision agriculture, convolutional neural networks, domain shift, cross-attention, PlantVillage</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">246834</post-id>	</item>
		<item>
		<title>Fine-tuned AI models learn to name the target of online hate</title>
		<link>https://scienmag.com/fine-tuned-ai-models-learn-to-name-the-target-of-online-hate/</link>
		
		<dc:creator><![CDATA[Denise Maddox]]></dc:creator>
		<pubDate>Sun, 04 Oct 2026 02:31:08 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[AI bias in hate speech detection]]></category>
		<category><![CDATA[BERT]]></category>
		<category><![CDATA[challenges in automated hate speech moderation]]></category>
		<category><![CDATA[class imbalance]]></category>
		<category><![CDATA[content moderation]]></category>
		<category><![CDATA[DeepSeek]]></category>
		<category><![CDATA[differentiating hate speech targets]]></category>
		<category><![CDATA[ETHOS dataset]]></category>
		<category><![CDATA[fine-tuning]]></category>
		<category><![CDATA[fine-tuning AI for hate group detection]]></category>
		<category><![CDATA[Hate speech classification]]></category>
		<category><![CDATA[hate speech detection]]></category>
		<category><![CDATA[improving hate speech classifiers]]></category>
		<category><![CDATA[large language models]]></category>
		<category><![CDATA[machine learning for online hate]]></category>
		<category><![CDATA[multiclass classification]]></category>
		<category><![CDATA[natural language processing]]></category>
		<category><![CDATA[QLoRA]]></category>
		<category><![CDATA[social media]]></category>
		<category><![CDATA[social media moderation]]></category>
		<category><![CDATA[supervised learning in hate speech detection]]></category>
		<category><![CDATA[targeted hate speech identification]]></category>
		<category><![CDATA[transformer models in hate speech analysis]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=233130</guid>

					<description><![CDATA[A new study shows that fine-tuning large language models substantially outperforms prompting and classical baselines for detecting which group online hate speech targets, though minority categories remain difficult.]]></description>
										<content:encoded><![CDATA[<p>Hate speech on social media has long been treated by automated moderation systems as a single, undifferentiated problem: a post is either hateful or it is not. A new study argues that this binary framing is precisely why so many moderation tools fail the communities that need them most. Researchers Sanaa Kaddoura of Zayed University and Sumaia Al-Kohlani of United Arab Emirates University have systematically evaluated how well modern large language models can classify hateful tweets not just as hate, but by the specific group being targeted, spanning gender, race and ethnicity, immigration and xenophobia, religion, and general abuse. Their findings, published in Discover Artificial Intelligence, show that supervised fine-tuning transforms these models from unreliable label-generators into the strongest classifiers yet tested on this task.</p>
<p>The motivation for the work lies in a structural weakness of earlier approaches. Traditional hate speech detectors, built on support vector machines, logistic regression, convolutional and recurrent neural networks, and eventually transformer models such as BERT, were trained on datasets that collapsed all forms of hate into one category. That conflation forces models to rely on surface-level lexical cues rather than deeper contextual meaning, and the consequences are uneven: hate directed at minority or rarely referenced groups gets misclassified far more often than hate against well-represented targets. Existing multiclass datasets offered some improvement, but most were small, inconsistently annotated, or still failed to model the target of abuse explicitly, creating a persistent trade-off between scale, label granularity, and annotation quality.</p>
<p>To test what large language models could add, the team assembled a dataset of 12,957 English tweets, built by re-annotating the hate speech subset of the widely used TweetEval benchmark. Of these, 5,455 samples carry one of five target-specific hate labels and 7,502 are non-hateful. Three independent annotators labeled each instance, with a fourth resolving disagreements, achieving a moderate inter-annotator agreement of 0.6003 on Krippendorff&#8217;s alpha, a figure the authors note reflects the inherent subjectivity of judging hateful content. Because some original categories were too sparse to train on reliably, the researchers consolidated them: threats and violence, with only 106 instances, was merged into profanity and general abuse, while hate toward countries, with just 18 samples, was folded into racial and ethnic hate. The result is a six-class problem that mirrors the imbalance of real platforms, where non-hate content makes up nearly 58 percent of the data and religious hate speech barely 1.3 percent.</p>
<p>Five models with distinct architectures and training histories were put through their paces: DeepSeek-R1-Distill-Qwen-32B, Mistral-7B-Instruct, WizardLM-13B, Phi-4-mini, and OpenAI&#8217;s GPT-4o, alongside a fine-tuned BERT-base-uncased baseline. Each was evaluated three ways: zero-shot prompting, few-shot prompting with six labeled examples, and supervised fine-tuning. The open-source models were adapted using QLoRA, a parameter-efficient technique that freezes the pretrained weights and trains only small low-rank adapter matrices inserted into each transformer block, with the base model loaded in 4-bit precision to fit on a single NVIDIA A100 GPU. GPT-4o was fine-tuned through OpenAI&#8217;s API at a total training cost of 85.75 dollars.</p>
<p>The zero-shot results were sobering. Every model struggled, with macro F1 scores, which weight all classes equally and thus expose weakness on minority categories, remaining low across the board. GPT-4o led with a macro F1 of 0.4503, while WizardLM, which had performed well in an earlier benchmark, scored close to zero on this dataset. More troubling still, several models refused to stay within the label space: WizardLM produced 1,574 unclassified outputs, including refusals, reasoning artifacts, and requests for context, while Mistral managed only four. Few-shot prompting helped some models, improving Phi and Mistral, but actually degraded DeepSeek and GPT-4o, demonstrating that in-context examples alone do not guarantee better behavior on complex classification tasks.</p>
<p>Fine-tuning changed the picture dramatically. The fine-tuned DeepSeek model achieved the best multiclass performance of any system tested, with macro and weighted F1 scores of 59.91 and 76.71 percent respectively, and it eliminated unclassified outputs entirely. Mistral followed closely with a weighted F1 of 74.12 percent, while fine-tuned GPT-4o, despite its strong showing in prompting settings, reached only 61.54 percent. The replicated BERT baseline remained competitive at 70.89 percent weighted F1 but was clearly outperformed by the best LLM. Against a suite of replicated classical machine learning and deep learning baselines, including logistic regression, naive Bayes, CNN, and BiLSTM models, the fine-tuned DeepSeek improved weighted F1 by margins ranging from 2.59 to 29.5 percentage points. McNemar statistical tests confirmed the differences were significant against BERT, BiLSTM, and logistic regression.</p>
<p>Crucially, the team did not stop at same-dataset evaluation, a common weakness in prior work. They assessed the best DeepSeek model on a remapped subset of the external ETHOS benchmark, aligning its gender, religion, and race labels with the study&#8217;s taxonomy and yielding 789 evaluation instances. The model scored an F1 of 88.31 percent on this external test, with no unclassified outputs, evidence of genuine cross-dataset generalization. A control experiment sharpened the interpretation: when a DeepSeek model was fine-tuned directly on ETHOS, it performed well on ETHOS itself but dropped sharply when tested on the original multiclass dataset, suggesting that the multiclass-trained model had learned richer, more transferable linguistic patterns rather than dataset-specific shortcuts.</p>
<p>The gains came with caveats. Per-class analysis revealed that minority categories remained hard: racial and ethnic hate speech and religious hate speech, each under two percent of the data, achieved F1 scores of only 0.4211 and 0.5098 respectively, while immigration and xenophobic hate, the best-annotated class, reached 0.7624. Confusion matrices showed the dominant error was not mixing up hate categories but distinguishing hate from non-hate at all, particularly for implicit or context-dependent hostility that lacks overt slurs. Computational cost also matters: the DeepSeek model required 4.71 hours of training and 32.5 minutes to classify the 1,620-tweet test set, whereas logistic regression completed both in seconds. The authors frame this as a practical trade-off, recommending LLMs where fine-grained discrimination justifies the expense and simpler classifiers for high-throughput, resource-constrained settings.</p>
<p>The study also confronts the ethics of deployment. False positives can suppress counter-speech, satire, or educational discussion, and may disproportionately silence the very communities these systems claim to protect, while false negatives leave targeted groups exposed. The authors stress that model predictions should never serve as the sole basis for content removal or user sanctions, and they call for human oversight, transparent governance, and periodic auditing. Looking forward, they outline plans to extend the framework to multilingual settings, adopt multi-label formulations for posts targeting multiple groups, and incorporate community-centered evaluation. The fine-tuned model and dataset have been released publicly, inviting the research community to build on a benchmark that finally asks not just whether a post is hateful, but whom it harms.</p>
<p><strong>Subject of Research:</strong> Fine-tuning large language models for target-specific multiclass hate speech detection on social media</p>
<p><strong>Article Title:</strong> Fine-tuning large language models for binary and multiclass target-specific hate speech detection</p>
<p><strong>Article References:</strong> Kaddoura, S., &amp; Al-Kohlani, S. (2026). Fine-tuning large language models for binary and multiclass target-specific hate speech detection. <em>Discover Artificial Intelligence, 6</em>(1), Article 1296. <a href="https://doi.org/10.1007/s44163-026-02354-1" rel="noopener noreferrer">https://doi.org/10.1007/s44163-026-02354-1</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44163-026-02354-1" rel="noopener noreferrer">10.1007/s44163-026-02354-1</a></p>
<p><strong>Keywords:</strong> large language models, hate speech detection, fine-tuning, QLoRA, DeepSeek, BERT, multiclass classification, content moderation, natural language processing, social media, ETHOS dataset, class imbalance</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">233130</post-id>	</item>
		<item>
		<title>AI Learns to Read Kurdish News Stance with Just 2,174 Articles</title>
		<link>https://scienmag.com/ai-learns-to-read-kurdish-news-stance-with-just-2174-articles/</link>
		
		<dc:creator><![CDATA[Denise Maddox]]></dc:creator>
		<pubDate>Fri, 25 Sep 2026 01:29:43 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[automatic stance classification in Kurdish]]></category>
		<category><![CDATA[benchmarking]]></category>
		<category><![CDATA[class imbalance]]></category>
		<category><![CDATA[data augmentation]]></category>
		<category><![CDATA[data engineering for low-resource languages]]></category>
		<category><![CDATA[empirical benchmark for Kurdish NLP]]></category>
		<category><![CDATA[fine-tuning]]></category>
		<category><![CDATA[fine-tuning language models for Kurdish]]></category>
		<category><![CDATA[KuBERT]]></category>
		<category><![CDATA[Kurdish news bias detection]]></category>
		<category><![CDATA[Kurdish news sentiment analysis]]></category>
		<category><![CDATA[Kurdish news stance detection]]></category>
		<category><![CDATA[LoRa]]></category>
		<category><![CDATA[low-resource language NLP]]></category>
		<category><![CDATA[low-resource languages]]></category>
		<category><![CDATA[macro F1]]></category>
		<category><![CDATA[misinformation and polarized discourse analysis]]></category>
		<category><![CDATA[multilingual NLP development in Iraq]]></category>
		<category><![CDATA[natural language processing]]></category>
		<category><![CDATA[QLoRA]]></category>
		<category><![CDATA[Sorani Kurdish]]></category>
		<category><![CDATA[Sorani Kurdish natural language processing]]></category>
		<category><![CDATA[stance detection]]></category>
		<category><![CDATA[stance detection models for Kurdish]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=213859</guid>

					<description><![CDATA[Researchers report the first trained stance-detection models and benchmark for Sorani Kurdish, using data augmentation and QLoRA fine-tuning of KuBERT to reach a macro F1 of 0.58 on a leakage-free test protocol.]]></description>
										<content:encoded><![CDATA[<p>Researchers in Sulaimani, in the Kurdistan Region of Iraq, have built the first trained stance-detection models and empirical benchmark for Sorani Kurdish news, one of the most widely spoken varieties of Kurdish yet long absent from the map of modern natural language processing. Their study, published in the International Journal of Data Science and Analytics, shows how far careful data engineering and efficient fine-tuning of a language model can go when a language has almost no annotated data to learn from. The work was carried out by Hawar Hussein Yaba of the Kurdistan Technical Institute together with Rebwar M. Nabi and Rebaz M. Nabi of Sulaimani Polytechnic University and the Raparin Technical and Vocational Institute.</p>
<p>Stance detection is the task of automatically determining whether a piece of text is in favor of, against, or neutral toward a target such as a political claim, a public figure, or a news event. It is a cornerstone technology for studying misinformation, polarized discourse, and the shape of public debate online. For high-resource languages like English, mature models and large benchmark datasets exist, and recent research has pushed toward multimodal and zero-shot approaches. For Sorani Kurdish, however, the authors report that no trained stance-detection model or benchmark had previously been published at all, despite the recent release of a small annotated dataset known as the Bochun Kurdish Stance Detection Dataset. That mismatch between an available resource and any applied modeling is the gap the new study set out to close.</p>
<p>The starting point was a public dataset of 2,174 annotated Sorani news articles, hosted on Mendeley Data. That number is tiny by the standards of modern machine learning, where models routinely train on tens of thousands or millions of labeled examples. Worse, the dataset suffered from severe class imbalance, meaning the three stance categories were far from equally represented, a condition that notoriously causes classifiers to ignore minority classes and inflate their apparent accuracy. The team therefore framed their central question as how to build reliable stance detectors under both extreme data scarcity and skewed label distributions.</p>
<p>Their answer combined two lines of attack. The first was a contextual data augmentation pipeline that expanded the training corpus into a strictly filtered, class-balanced set. At its heart lies masked-token substitution powered by KuBERT, a BERT language model pre-trained specifically for central Kurdish. In this technique, words in training sentences are replaced with a mask token, and the language model predicts plausible substitutes that fit the surrounding context, generating new sentences that stay grammatically and semantically close to the originals. This is a low-resource adaptation of contextual augmentation, an approach introduced at NAACL in 2018, and the team paired it with controlled oversampling so that each stance class contributed a balanced share of training examples. Crucially, the augmentation was subjected to strict filtering, and human Kurdish-language annotators validated a sample of the generated text to confirm that the procedure preserved the original labels, an assumption the authors say underpins the entire approach.</p>
<p>The second line of attack was the fine-tuning strategy itself. Rather than training models from scratch, the researchers adapted KuBERT, the Kurdish BERT model released in 2024, under three distinct regimes. Full-parameter updating adjusts every weight in the network and typically demands the most compute and memory. Low-rank adaptation, or LoRA, freezes the base weights and instead learns small low-rank matrices injected into the transformer&#8217;s layers, cutting the number of trainable parameters dramatically. QLoRA goes a step further by quantizing the frozen base model to 4-bit precision before applying LoRA, a configuration popularized by the 2023 QLoRA paper for efficient fine-tuning of large language models. Comparing these three strategies on the same data provided a rare empirical head-to-head in a genuinely low-resource setting.</p>
<p>Evaluation methodology received as much attention as the models themselves. The team ran all experiments across five random seeds, a safeguard against the luck of any single training run, and scored everything on a held-out test set drawn exclusively from real, unaugmented articles. This distinction matters: evaluating on synthetic text can silently leak the very patterns augmentation introduces, so testing only on genuine journalism gives a more honest picture of real-world performance. Macro-averaged F1 served as the primary metric because it weights each stance class equally and thus exposes failure on minority classes, with accuracy, weighted F1, the Matthews Correlation Coefficient, and Cohen&#8217;s kappa reported alongside it. MCC, in particular, is prized for giving a truthful single number even when class distributions are unbalanced.</p>
<p>The results tell a clear story about why task-specific adaptation matters. A zero-shot KuBERT baseline, applied to stance detection without any fine-tuning, scored a macro F1 of roughly 0.20 with an MCC near negative 0.03, meaning it performed barely better than random guessing. After fine-tuning on the original, imbalanced dataset, models reached a macro F1 of around 0.48 with an MCC of about 0.24, more than doubling the baseline and confirming that domain adaptation is essential even when data is scarce. But the most striking gains appeared on the class-balanced augmented condition, evaluated under a corrected, leakage-free protocol. There, the best configuration, KuBERT fine-tuned with QLoRA, achieved a mean macro F1 of 0.58 and an MCC of 0.37 across seeds, with the strongest single run reaching a macro F1 of 0.60.</p>
<p>Perhaps the most instructive finding came from the per-class and ablation analysis, which disentangled where the improvement actually came from. The authors report that most of the gain was driven by correcting class imbalance rather than by the diversity of augmented examples alone. In other words, balancing the classes was the dominant lever, with contextual augmentation contributing as the mechanism that made balancing possible without exhausting the real data. This nuance carries a practical lesson for anyone building classifiers on small, skewed datasets in low-resource languages: the expensive machinery of augmentation and parameter-efficient fine-tuning pays off most when paired with a disciplined focus on label distribution and honest evaluation.</p>
<p>The choice of QLoRA as the winning configuration also has practical implications beyond accuracy. Because QLoRA trains only tiny adapter modules on top of a 4-bit quantized base model, it slashes memory requirements, putting fine-tuning of pretrained language models within reach of research groups without access to large-scale computing infrastructure. For language communities outside the technological mainstream, that accessibility may prove as consequential as the benchmark numbers themselves. The team&#8217;s augmentation pipeline, fine-tuning scripts, model configuration files, and per-seed evaluation results are available from the corresponding author on reasonable request, and the underlying Bochun dataset is public, lowering the barrier for follow-up work.</p>
<p>The broader significance of the study lies in what it demonstrates for the roughly tens of millions of Sorani speakers whose media landscape has been effectively invisible to computational text analysis. Reliable stance detection could enable systematic study of how misinformation spreads through Kurdish news and social media, inform fact-checking efforts, and support media-monitoring tools tuned to the region&#8217;s discourse. The authors are explicit that their contribution is a first competitive baseline rather than a solved problem: a macro F1 of 0.58, while a substantial step above chance, still leaves considerable room for improvement, and future work will likely explore larger corpora, additional architectures, and zero-shot techniques now emerging for stance detection more broadly. But as a proof of concept, the study shows that a small annotated dataset, a Kurdish language model, and a leakage-free, class-aware evaluation protocol can together close a meaningful portion of the gap between low-resource languages and the cutting edge of natural language understanding.</p>
<p><strong>Subject of Research:</strong> Stance detection for low-resource Sorani Kurdish news using data augmentation and parameter-efficient fine-tuning of a Kurdish BERT model</p>
<p><strong>Article Title:</strong> Towards robust stance detection: data augmentation, model fine-tuning, and empirical benchmarking</p>
<p><strong>Article References:</strong> Yaba, H. H., Nabi, R. M., &amp; Nabi, R. M. (2026). Towards robust stance detection: data augmentation, model fine-tuning, and empirical benchmarking. <em>International Journal of Data Science and Analytics, 22</em>(1), Article 310. <a href="https://doi.org/10.1007/s41060-026-01266-8" rel="noopener noreferrer">https://doi.org/10.1007/s41060-026-01266-8</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s41060-026-01266-8" rel="noopener noreferrer">10.1007/s41060-026-01266-8</a></p>
<p><strong>Keywords:</strong> stance detection, Sorani Kurdish, KuBERT, data augmentation, QLoRA, LoRA, fine-tuning, low-resource languages, natural language processing, class imbalance, macro F1, benchmarking</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">213859</post-id>	</item>
		<item>
		<title>Knowledge Graphs and Large Language Models Team Up to Diagnose Satellite Amplifier Faults</title>
		<link>https://scienmag.com/knowledge-graphs-and-large-language-models-team-up-to-diagnose-satellite-amplifier-faults/</link>
		
		<dc:creator><![CDATA[Grant Pearson]]></dc:creator>
		<pubDate>Tue, 22 Sep 2026 15:36:50 +0000</pubDate>
				<category><![CDATA[Space]]></category>
		<category><![CDATA[automated satellite fault reasoning]]></category>
		<category><![CDATA[decision support]]></category>
		<category><![CDATA[fault diagnosis]]></category>
		<category><![CDATA[fault event documentation analysis]]></category>
		<category><![CDATA[fault symptom and root cause correlation]]></category>
		<category><![CDATA[intelligent fault diagnosis in space systems]]></category>
		<category><![CDATA[knowledge graph]]></category>
		<category><![CDATA[knowledge graphs in satellite fault detection]]></category>
		<category><![CDATA[large language model]]></category>
		<category><![CDATA[large language models for satellite diagnostics]]></category>
		<category><![CDATA[multi-dimensional fault event reasoning]]></category>
		<category><![CDATA[named entity recognition]]></category>
		<category><![CDATA[on-orbit reliability]]></category>
		<category><![CDATA[on-orbit satellite health monitoring]]></category>
		<category><![CDATA[QLoRA]]></category>
		<category><![CDATA[Qwen2-7B-Instruct]]></category>
		<category><![CDATA[retrieval-augmented generation]]></category>
		<category><![CDATA[satellite amplifier fault diagnosis]]></category>
		<category><![CDATA[satellite communication system troubleshooting]]></category>
		<category><![CDATA[satellite communications]]></category>
		<category><![CDATA[space mission fault management]]></category>
		<category><![CDATA[space TWTAs]]></category>
		<category><![CDATA[space TWTAs failure analysis]]></category>
		<category><![CDATA[Space: Science & Technology]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=206455</guid>

					<description><![CDATA[Researchers in China combined a fine-tuned large language model with a TWTA fault knowledge graph to achieve a 94.56 percent expert adoption rate in satellite amplifier fault diagnosis.]]></description>
										<content:encoded><![CDATA[<p>Space traveling wave tube amplifiers, known as TWTAs, sit at the heart of nearly every satellite communications payload. As the final-stage amplification devices in satellite communication systems, they combine high power, high efficiency, and ultra-wideband performance, which has made them the dominant choice for boosting signals before transmission back to Earth. Because their operational status directly determines satellite signal transmission capability, any malfunction can degrade a mission or, in the worst case, interrupt service entirely. As the number of on-orbit applications continues to climb, so does the volume of text records describing fault events, fault phenomena, root causes, and corrective solutions. These records hold enormous diagnostic value, yet they have long been trapped in a format that resists automated analysis.</p>
<p>Traditional fault analysis for space TWTAs depends on manual statistics and empirical judgment. Engineers comb through accumulated fault documentation, match observed symptoms against remembered cases, and reason their way to a probable cause and remedy. This approach suffers from low efficiency, scales poorly as the fault knowledge base grows, and lacks any capacity for knowledge reasoning across the many dimensions of a fault event, such as the mission stage in which a failure occurred, the components involved, the phenomenon observed, the causal analysis, and the correction applied. Artificial intelligence techniques, particularly knowledge graphs and large language models, have shown substantial potential in fault diagnosis in other domains, but until now, research on effectively integrating the two and applying the combination to the specific fault diagnosis of space TWTAs remained unexplored.</p>
<p>Researchers from the School of Electronic Science and Engineering at the University of Electronic Science and Technology of China have addressed this gap. In a study published in Space: Science &amp; Technology, the team proposed a fault diagnosis framework for space TWTAs built on the integration of a knowledge graph and a large language model. The framework consists of three cooperating modules: a named entity recognition module, a knowledge graph module, and a graph retrieval module. Together, these modules implement retrieval-augmented generation functionality. A user submits a fault description in natural language; the NER module extracts the relevant entities; the graph retrieval module automatically generates graph query language to search the knowledge graph; the retrieved contextual information is combined with the original query to form a prompt; and a general-purpose large language model generates the final diagnostic response. Notably, requirements analysis, graph retrieval, information judgment, and answer generation are all accomplished by a single large language model, which enhances the coupling among modules and the overall stability of the framework.</p>
<p>The named entity recognition module is the framework&#8217;s first line of interpretation. The research team constructed three types of corpora for data alignment: a component corpus, a stage corpus, and a fault description corpus. Implementation proceeds in three steps: data augmentation, data pair preparation, and model training. Data augmentation enriches the diversity of training data by replacing component name aliases, rephrasing fault descriptions, and adjusting sentence structures, so that a single raw data entry yields multiple augmented variants. The component name, stage, and fault description information are extracted to form the ASSISTANT portion of each example, which is combined with the PROMPT portion to constitute training data pairs. Model training employs QLoRA technology to fine-tune the Qwen2-7B-Instruct large language model. QLoRA freezes the 4-bit quantized pre-trained model and updates only the parameters of low-rank adaptation modules, achieving efficient fine-tuning under limited computational resources. The fine-tuned model automatically extracts three categories of entities from user inputs: stage, component, and fault.</p>
<p>The knowledge graph module defines the structured representation of TWTA fault knowledge. Five entity types are specified: stage, component, fault, analysis, and correction. Five relationship types connect them: causal relationship, corrective relationship, occurrence relationship, involvement relationship, and composition relationship. This schema comprehensively restores the details of real fault events, including the time of occurrence, the components involved, descriptions of the phenomenon, the causal analysis performed, and the corrective solutions applied. By organizing heterogeneous text records into this graph, the framework gains an explicit, queryable model of fault causality that a language model alone, relying purely on parametric memory, could not reliably provide.</p>
<p>Bridging natural language and the graph is the job of the graph retrieval module. The team adopted a two-step self-instruction method to generate training data pairs that map natural language questions to graph query language, enabling automatic translation from a user&#8217;s free-text description to a formal query. This alignment between the large language model and the knowledge graph is what allows the full pipeline to run without handcrafted query templates. Once the graph returns relevant substructures, a retrieval-augmented generation mechanism feeds that evidence, alongside the original user query, into the language model, which then produces fault causes and treatment solutions grounded in the curated knowledge base rather than in speculation.</p>
<p>Experimental validation targeted both individual modules and the end-to-end framework. In tests of the named entity recognition module, the QLoRA-fine-tuned Qwen2-7B-Instruct model achieved F1-scores of 97.56 percent for stage entities, 79.99 percent for component entities, and 90.00 percent for fault entities, yielding a macro-average F1-score of 89.18 percent. These results significantly outperformed both the non-fine-tuned version of the same model and GPT-4o. The graph retrieval module attained a hit rate of 89 percent across 100 natural language queries, confirming that the self-instruction alignment reliably converts questions into queries that locate the intended graph content.</p>
<p>Real-scenario testing demonstrated the practical value of the integrated system. Out of 100 fault descriptions, the framework provided responses to 92, and domain experts adopted 87 of those, corresponding to an adoption rate of 94.56 percent. By contrast, the non-fine-tuned baseline model, although it generated responses to all 100 descriptions, achieved only 23 adoptions. The framework also maintained semantic consistency when answering identical questions repeatedly, while the baseline model exhibited persistent instability across trials. In interference experiments introducing three types of noise, the framework achieved accuracies of 97.22 percent under spelling errors, 95.00 percent under distracting sentences, and 100 percent under irrelevant data, and it effectively rejected irrelevant queries, demonstrating robust stability in messy, real-world conditions.</p>
<p>Scale and speed matter for operational decision support, and the larger experiments addressed both. Across a test set of 1,618 data entries, the framework still maintained an accuracy of 88.13 percent, and the average response time was 19 seconds per query. Those figures validate the system as a usable decision-support tool rather than a laboratory curiosity, giving engineers a fast second opinion grounded in a structured fault knowledge base within the time frame of an interactive troubleshooting session.</p>
<p>The research offers an efficient technical solution for intelligent fault diagnosis of space traveling wave tube amplifiers, and its significance extends beyond a single component type. It shows how a domain-specific knowledge graph, aligned with a fine-tuned, resource-efficient large language model through retrieval-augmented generation, can convert decades of unstructured maintenance text into actionable reasoning. For spacecraft operators, the payoff is enhanced on-orbit reliability and improved operational maintenance efficiency; for the broader aerospace community, it provides a template for bringing natural language understanding and knowledge reasoning together wherever scattered fault records currently resist systematic exploitation.</p>
<p><strong>Subject of Research:</strong> An intelligent fault diagnosis framework for space traveling wave tube amplifiers that integrates a knowledge graph with a fine-tuned large language model</p>
<p><strong>Article Title:</strong> Fault diagnosis framework for space TWTAs by combining the knowledge graph and large language mode</p>
<p><strong>Article References:</strong> Fault diagnosis framework for space TWTAs by combining the knowledge graph and large language mode. (n.d.). <a href="https://www.eurekalert.org/news-releases/1144833" rel="noopener noreferrer">Original publication</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> Not provided</p>
<p><strong>Keywords:</strong> space TWTAs, fault diagnosis, knowledge graph, large language model, Qwen2-7B-Instruct, QLoRA, named entity recognition, retrieval-augmented generation, satellite communications, Space: Science &amp; Technology, on-orbit reliability, decision support</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">206455</post-id>	</item>
	</channel>
</rss>
