<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>macro F1 &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/macro-f1/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Sat, 03 Oct 2026 19:19:02 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>macro F1 &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>AI Framework Reads Insect Clues Like an Expert to Identify Crop Pests</title>
		<link>https://scienmag.com/ai-framework-reads-insect-clues-like-an-expert-to-identify-crop-pests/</link>
		
		<dc:creator><![CDATA[Alan Morgan]]></dc:creator>
		<pubDate>Sat, 03 Oct 2026 19:19:02 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[agricultural AI]]></category>
		<category><![CDATA[AI for agriculture]]></category>
		<category><![CDATA[challenges in pest recognition]]></category>
		<category><![CDATA[CLARiF pest recognition]]></category>
		<category><![CDATA[crop pest detection]]></category>
		<category><![CDATA[crop protection]]></category>
		<category><![CDATA[deep learning]]></category>
		<category><![CDATA[entomology]]></category>
		<category><![CDATA[exemplar-informed re-ranking]]></category>
		<category><![CDATA[fine-grained classification]]></category>
		<category><![CDATA[fine-grained insect classification]]></category>
		<category><![CDATA[image segmentation]]></category>
		<category><![CDATA[Insect pest identification]]></category>
		<category><![CDATA[insect species differentiation]]></category>
		<category><![CDATA[IP102 benchmark]]></category>
		<category><![CDATA[macro F1]]></category>
		<category><![CDATA[mask-guided part extraction]]></category>
		<category><![CDATA[open-set recognition]]></category>
		<category><![CDATA[pest recognition]]></category>
		<category><![CDATA[retrieval-augmented learning]]></category>
		<category><![CDATA[structured diagnostic captioning]]></category>
		<category><![CDATA[tackling cluttered field images]]></category>
		<category><![CDATA[vision-language framework]]></category>
		<category><![CDATA[vision-language model]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=231594</guid>

					<description><![CDATA[A new vision–language framework called CLARiF combines diagnostic captioning, part-level segmentation, and retrieval-informed re-ranking to achieve state-of-the-art fine-grained pest recognition on the IP102 benchmark while reliably rejecting species it has never seen.]]></description>
										<content:encoded><![CDATA[<p>Fine-grained pest recognition has long been one of the most stubborn problems in agricultural artificial intelligence. A farmer or agronomist photographing an insect in the field is rarely rewarded with a clean, studio-style image. Instead, the camera captures the insect against tangled foliage, dappled light, soil, and other insects, all of which conspire to obscure the very features that distinguish one species from another. The challenge is compounded by the fact that many pest species look almost identical to the human eye, differing only in the pattern of wing veins, the shape of an antenna segment, or the spacing of spots along the abdomen. A new study published in Applied Intelligence introduces a vision–language framework called CLARiF, which tackles these difficulties by combining structured diagnostic captioning, mask-guided part extraction, bidirectional part–attribute fusion, and exemplar-informed re-ranking into a single recognition pipeline.</p>
<p>The research team, led by Qiuyu Li and Zhijie Xu of Xi&#8217;an Jiaotong-Liverpool University together with colleagues at the University of Liverpool and Zhengzhou University of Light Industry, framed the problem around five recurring obstacles: cluttered backgrounds, life-stage variation, subtle inter-class differences, long-tailed category distributions, and unsupported taxa that the model has never seen during training. Each of these obstacles can derail a conventional classifier. Life-stage variation means that the larva of one species may resemble the adult of another; long-tailed distributions mean that a handful of pest categories dominate the training data while dozens of rare species are represented by only a few images. Unsupported taxa pose an even deeper problem, because a closed-set classifier will confidently assign a label even when the insect in front of it belongs to a species entirely absent from its training set.</p>
<p>CLARiF&#8217;s first line of attack is structured diagnostic captioning. Rather than treating an image as an undifferentiated bundle of pixels, the framework generates textual descriptions that follow the logic an entomologist would use at a diagnostic key: body shape, coloration, markings, wing structure, and other morphological clues. This converts the recognition task into something closer to a cross-modal reasoning problem, in which the visual evidence and the verbal description of diagnostic traits must be brought into alignment. The approach draws on the recent wave of vision–language models, including architectures descended from CLIP, Flamingo, BLIP-2, LLaVA, and Qwen-VL, which have demonstrated that pairing images with natural language can dramatically improve generalization on tasks that require fine distinctions.</p>
<p>The second component, mask-guided part extraction, addresses the clutter problem directly. Using segmentation techniques in the lineage of the Segment Anything Model family, the framework isolates the insect from its surroundings before deeper analysis begins. This matters because a classifier that must simultaneously learn what the insect looks like and what the background looks like wastes capacity on irrelevant variation. By constraining attention to the segmented organism, and further to meaningful body parts within it, CLARiF ensures that the features driving a decision correspond to the anatomy of the pest rather than to the texture of a leaf or the angle of the sunlight.</p>
<p>The third element, bidirectional part–attribute fusion, is where the framework earns the alignment half of its name. Visual features extracted from specific body parts are fused with the attribute terms in the diagnostic captions in both directions: parts inform which attributes are present, and attributes guide which parts deserve scrutiny. This bidirectional flow allows the model to reason, in effect, that a particular spot pattern on the forewing supports one species hypothesis while the segment count on the antenna supports another. Such structured reasoning is precisely what separates fine-grained recognition from ordinary object classification, where a single holistic impression of the image is often sufficient.</p>
<p>The fourth component, exemplar-informed re-ranking, borrows an idea from retrieval-augmented systems. After the model produces an initial ranking of candidate species, the framework consults a gallery of reference exemplars and adjusts the ranking in light of the closest matches. This retrieval-informed fusion gives the system a form of case-based memory: even if the learned representations are imperfect, comparison against curated reference images can correct the final decision. The authors report that under comparable data-processing and fine-tuning settings, CLARiF outperformed independently reproduced vision–language baselines while retaining each model&#8217;s native language backbone, suggesting that the gains come from the framework rather than from any single underlying model.</p>
<p>The headline numbers are striking. On the official test split of IP102, the large-scale benchmark for insect pest recognition introduced by Wu and colleagues in 2019, CLARiF achieves an accuracy of 77.81 percent with a standard deviation of 0.18 across runs, and a macro-F1 score of 77.07 percent with a standard deviation of 0.21. The macro-F1 figure is particularly meaningful in this domain because it weights all classes equally, refusing to let abundant categories mask poor performance on rare ones. In a long-tailed setting where many species have few training examples, a high macro-F1 indicates that the framework is genuinely learning to distinguish the scarce and difficult categories, not merely riding the statistics of the common ones.</p>
<p>Perhaps the most consequential result concerns the open-set problem. Under a fixed held-out-species protocol, in which certain species were deliberately excluded from training and then presented to the model at test time, CLARiF achieved an area under the receiver operating characteristic curve of 88.62 percent for rejecting unsupported inputs. In practical terms, this means the system can often tell when it does not know, flagging an insect as outside its competence rather than fabricating a confident but wrong identification. For agricultural deployment, this property may matter more than raw accuracy: a misidentification of a quarantine pest as a harmless lookalike, or vice versa, can trigger unnecessary pesticide applications or allow an invasive species to spread undetected. A model that abstains when uncertain is a far safer instrument than one that always answers.</p>
<p>The evaluation was not confined to a single dataset. Alongside IP102, the study drew on the Forestry Pest Dataset and a curated control set referred to as IP102-Control, and the authors report that the data-processing pipeline, prompt templates, random seeds, held-out-species split files, gallery-construction scripts, and rejection-threshold configurations are documented in detail, with source code and processed exemplar metadata available from the corresponding author on reasonable request. This level of procedural transparency addresses a persistent weakness in the applied machine-learning literature, where comparisons between methods are often confounded by undisclosed differences in preprocessing, augmentation, or evaluation protocol. By reproducing the baselines themselves under identical settings, the team strengthened the claim that the observed improvements are attributable to the CLARiF design.</p>
<p>The broader significance of the work lies in what it suggests about the future of agricultural AI. Pest management is a cornerstone of global food security, and deep learning has already transformed tasks from disease detection on leaves to automated insect monitoring in greenhouses and pheromone traps. Yet most deployed systems remain brittle at the species level, where the decisions that matter are actually made. CLARiF demonstrates that the combination of language-grounded diagnostic reasoning, precise visual localization, and retrieval-based verification can push fine-grained recognition to a level of reliability that begins to approach expert practice, while the open-set rejection capability provides a guardrail against the overconfident errors that have historically undermined trust in automated identification. As vision–language models continue to improve, frameworks of this kind could become the analytical backbone of smartphone-based field diagnostics, extension services in regions without resident entomologists, and early-warning networks for invasive pests. The study, published in volume 56 of Applied Intelligence as article number 474, was supported by the SIP Leadership Talent Program, the Jiangsu Provincial Double Initiative Plan, and a Xi&#8217;an Jiaotong-Liverpool University research development grant, and the authors declare no competing interests.</p>
<p><strong>Subject of Research:</strong> Vision–language-based fine-grained recognition of agricultural insect pests</p>
<p><strong>Article Title:</strong> CLARiF: clue alignment and retrieval-informed fusion for fine-grained pest recognition</p>
<p><strong>Article References:</strong> Li, Q., Pan, Y., Xiang, N., Zhang, H., Li, Z., Huang, X., &amp; Xu, Z. (2026). CLARiF: clue alignment and retrieval-informed fusion for fine-grained pest recognition. <em>Applied Intelligence, 56</em>(15), Article 474. <a href="https://doi.org/10.1007/s10489-026-07522-5" rel="noopener noreferrer">https://doi.org/10.1007/s10489-026-07522-5</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s10489-026-07522-5" rel="noopener noreferrer">10.1007/s10489-026-07522-5</a></p>
<p><strong>Keywords:</strong> pest recognition, vision-language model, fine-grained classification, IP102 benchmark, agricultural AI, open-set recognition, retrieval-augmented learning, image segmentation, macro-F1, entomology, deep learning, crop protection</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">231594</post-id>	</item>
		<item>
		<title>AI Learns to Read Kurdish News Stance with Just 2,174 Articles</title>
		<link>https://scienmag.com/ai-learns-to-read-kurdish-news-stance-with-just-2174-articles/</link>
		
		<dc:creator><![CDATA[Denise Maddox]]></dc:creator>
		<pubDate>Fri, 25 Sep 2026 01:29:43 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[automatic stance classification in Kurdish]]></category>
		<category><![CDATA[benchmarking]]></category>
		<category><![CDATA[class imbalance]]></category>
		<category><![CDATA[data augmentation]]></category>
		<category><![CDATA[data engineering for low-resource languages]]></category>
		<category><![CDATA[empirical benchmark for Kurdish NLP]]></category>
		<category><![CDATA[fine-tuning]]></category>
		<category><![CDATA[fine-tuning language models for Kurdish]]></category>
		<category><![CDATA[KuBERT]]></category>
		<category><![CDATA[Kurdish news bias detection]]></category>
		<category><![CDATA[Kurdish news sentiment analysis]]></category>
		<category><![CDATA[Kurdish news stance detection]]></category>
		<category><![CDATA[LoRa]]></category>
		<category><![CDATA[low-resource language NLP]]></category>
		<category><![CDATA[low-resource languages]]></category>
		<category><![CDATA[macro F1]]></category>
		<category><![CDATA[misinformation and polarized discourse analysis]]></category>
		<category><![CDATA[multilingual NLP development in Iraq]]></category>
		<category><![CDATA[natural language processing]]></category>
		<category><![CDATA[QLoRA]]></category>
		<category><![CDATA[Sorani Kurdish]]></category>
		<category><![CDATA[Sorani Kurdish natural language processing]]></category>
		<category><![CDATA[stance detection]]></category>
		<category><![CDATA[stance detection models for Kurdish]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=213859</guid>

					<description><![CDATA[Researchers report the first trained stance-detection models and benchmark for Sorani Kurdish, using data augmentation and QLoRA fine-tuning of KuBERT to reach a macro F1 of 0.58 on a leakage-free test protocol.]]></description>
										<content:encoded><![CDATA[<p>Researchers in Sulaimani, in the Kurdistan Region of Iraq, have built the first trained stance-detection models and empirical benchmark for Sorani Kurdish news, one of the most widely spoken varieties of Kurdish yet long absent from the map of modern natural language processing. Their study, published in the International Journal of Data Science and Analytics, shows how far careful data engineering and efficient fine-tuning of a language model can go when a language has almost no annotated data to learn from. The work was carried out by Hawar Hussein Yaba of the Kurdistan Technical Institute together with Rebwar M. Nabi and Rebaz M. Nabi of Sulaimani Polytechnic University and the Raparin Technical and Vocational Institute.</p>
<p>Stance detection is the task of automatically determining whether a piece of text is in favor of, against, or neutral toward a target such as a political claim, a public figure, or a news event. It is a cornerstone technology for studying misinformation, polarized discourse, and the shape of public debate online. For high-resource languages like English, mature models and large benchmark datasets exist, and recent research has pushed toward multimodal and zero-shot approaches. For Sorani Kurdish, however, the authors report that no trained stance-detection model or benchmark had previously been published at all, despite the recent release of a small annotated dataset known as the Bochun Kurdish Stance Detection Dataset. That mismatch between an available resource and any applied modeling is the gap the new study set out to close.</p>
<p>The starting point was a public dataset of 2,174 annotated Sorani news articles, hosted on Mendeley Data. That number is tiny by the standards of modern machine learning, where models routinely train on tens of thousands or millions of labeled examples. Worse, the dataset suffered from severe class imbalance, meaning the three stance categories were far from equally represented, a condition that notoriously causes classifiers to ignore minority classes and inflate their apparent accuracy. The team therefore framed their central question as how to build reliable stance detectors under both extreme data scarcity and skewed label distributions.</p>
<p>Their answer combined two lines of attack. The first was a contextual data augmentation pipeline that expanded the training corpus into a strictly filtered, class-balanced set. At its heart lies masked-token substitution powered by KuBERT, a BERT language model pre-trained specifically for central Kurdish. In this technique, words in training sentences are replaced with a mask token, and the language model predicts plausible substitutes that fit the surrounding context, generating new sentences that stay grammatically and semantically close to the originals. This is a low-resource adaptation of contextual augmentation, an approach introduced at NAACL in 2018, and the team paired it with controlled oversampling so that each stance class contributed a balanced share of training examples. Crucially, the augmentation was subjected to strict filtering, and human Kurdish-language annotators validated a sample of the generated text to confirm that the procedure preserved the original labels, an assumption the authors say underpins the entire approach.</p>
<p>The second line of attack was the fine-tuning strategy itself. Rather than training models from scratch, the researchers adapted KuBERT, the Kurdish BERT model released in 2024, under three distinct regimes. Full-parameter updating adjusts every weight in the network and typically demands the most compute and memory. Low-rank adaptation, or LoRA, freezes the base weights and instead learns small low-rank matrices injected into the transformer&#8217;s layers, cutting the number of trainable parameters dramatically. QLoRA goes a step further by quantizing the frozen base model to 4-bit precision before applying LoRA, a configuration popularized by the 2023 QLoRA paper for efficient fine-tuning of large language models. Comparing these three strategies on the same data provided a rare empirical head-to-head in a genuinely low-resource setting.</p>
<p>Evaluation methodology received as much attention as the models themselves. The team ran all experiments across five random seeds, a safeguard against the luck of any single training run, and scored everything on a held-out test set drawn exclusively from real, unaugmented articles. This distinction matters: evaluating on synthetic text can silently leak the very patterns augmentation introduces, so testing only on genuine journalism gives a more honest picture of real-world performance. Macro-averaged F1 served as the primary metric because it weights each stance class equally and thus exposes failure on minority classes, with accuracy, weighted F1, the Matthews Correlation Coefficient, and Cohen&#8217;s kappa reported alongside it. MCC, in particular, is prized for giving a truthful single number even when class distributions are unbalanced.</p>
<p>The results tell a clear story about why task-specific adaptation matters. A zero-shot KuBERT baseline, applied to stance detection without any fine-tuning, scored a macro F1 of roughly 0.20 with an MCC near negative 0.03, meaning it performed barely better than random guessing. After fine-tuning on the original, imbalanced dataset, models reached a macro F1 of around 0.48 with an MCC of about 0.24, more than doubling the baseline and confirming that domain adaptation is essential even when data is scarce. But the most striking gains appeared on the class-balanced augmented condition, evaluated under a corrected, leakage-free protocol. There, the best configuration, KuBERT fine-tuned with QLoRA, achieved a mean macro F1 of 0.58 and an MCC of 0.37 across seeds, with the strongest single run reaching a macro F1 of 0.60.</p>
<p>Perhaps the most instructive finding came from the per-class and ablation analysis, which disentangled where the improvement actually came from. The authors report that most of the gain was driven by correcting class imbalance rather than by the diversity of augmented examples alone. In other words, balancing the classes was the dominant lever, with contextual augmentation contributing as the mechanism that made balancing possible without exhausting the real data. This nuance carries a practical lesson for anyone building classifiers on small, skewed datasets in low-resource languages: the expensive machinery of augmentation and parameter-efficient fine-tuning pays off most when paired with a disciplined focus on label distribution and honest evaluation.</p>
<p>The choice of QLoRA as the winning configuration also has practical implications beyond accuracy. Because QLoRA trains only tiny adapter modules on top of a 4-bit quantized base model, it slashes memory requirements, putting fine-tuning of pretrained language models within reach of research groups without access to large-scale computing infrastructure. For language communities outside the technological mainstream, that accessibility may prove as consequential as the benchmark numbers themselves. The team&#8217;s augmentation pipeline, fine-tuning scripts, model configuration files, and per-seed evaluation results are available from the corresponding author on reasonable request, and the underlying Bochun dataset is public, lowering the barrier for follow-up work.</p>
<p>The broader significance of the study lies in what it demonstrates for the roughly tens of millions of Sorani speakers whose media landscape has been effectively invisible to computational text analysis. Reliable stance detection could enable systematic study of how misinformation spreads through Kurdish news and social media, inform fact-checking efforts, and support media-monitoring tools tuned to the region&#8217;s discourse. The authors are explicit that their contribution is a first competitive baseline rather than a solved problem: a macro F1 of 0.58, while a substantial step above chance, still leaves considerable room for improvement, and future work will likely explore larger corpora, additional architectures, and zero-shot techniques now emerging for stance detection more broadly. But as a proof of concept, the study shows that a small annotated dataset, a Kurdish language model, and a leakage-free, class-aware evaluation protocol can together close a meaningful portion of the gap between low-resource languages and the cutting edge of natural language understanding.</p>
<p><strong>Subject of Research:</strong> Stance detection for low-resource Sorani Kurdish news using data augmentation and parameter-efficient fine-tuning of a Kurdish BERT model</p>
<p><strong>Article Title:</strong> Towards robust stance detection: data augmentation, model fine-tuning, and empirical benchmarking</p>
<p><strong>Article References:</strong> Yaba, H. H., Nabi, R. M., &amp; Nabi, R. M. (2026). Towards robust stance detection: data augmentation, model fine-tuning, and empirical benchmarking. <em>International Journal of Data Science and Analytics, 22</em>(1), Article 310. <a href="https://doi.org/10.1007/s41060-026-01266-8" rel="noopener noreferrer">https://doi.org/10.1007/s41060-026-01266-8</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s41060-026-01266-8" rel="noopener noreferrer">10.1007/s41060-026-01266-8</a></p>
<p><strong>Keywords:</strong> stance detection, Sorani Kurdish, KuBERT, data augmentation, QLoRA, LoRA, fine-tuning, low-resource languages, natural language processing, class imbalance, macro F1, benchmarking</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">213859</post-id>	</item>
	</channel>
</rss>
