The research, conducted by Jiwoo Choi and Chang Ouk Kim of Yonsei University’s Department of Industrial Engineering, builds on one of the most consequential developments in computer vision of the past several years: the rise of vision-language models, or VLMs. These models, chief among them CLIP, developed by OpenAI researchers, are trained on vast quantities of image-text pairs harvested from the web. Rather than learning only to classify images into fixed categories, they learn a shared space in which pictures and sentences live side by side. That structure gives them a remarkable ability to generalize. Show CLIP an image of an object it has never formally been taught to recognize, describe that object in plain English, and the model can often match the two without any task-specific training at all, a capability known as zero-shot transfer.

For anomaly detection, this generality is exactly what factories need. Traditional industrial anomaly detectors typically rely on convolutional neural networks pretrained on ImageNet, extracting features and flagging anything that deviates from a learned notion of normality. These systems work well when they can be trained on hundreds or thousands of images of a specific product, but they adapt poorly when the product line changes. A VLM-based approach promises something different: the ability to describe what a defect looks like in language, for instance, a photo of a flawless bottle cap paired with the phrase a photo of a damaged bottle cap, and let the model’s learned alignment between vision and language do the rest. The catch is that this alignment depends on how the description is written.

That dependency is the problem of prompt engineering. In CLIP-style systems, class labels are not fed to the model as bare words; they are wrapped in natural-language templates, such as a photo of a [category], which the text encoder converts into embeddings that guide the image encoder’s interpretation. Writing these templates by hand is something of a dark art. Small changes in wording can shift performance measurably, and crafting prompts that work for a specific industrial domain, with its particular vocabulary of defects and materials, requires knowledge that quality engineers may not have. Worse, hand-written prompts are static. A factory floor is a dynamic environment: lighting changes, new defect modes appear, product designs are revised. A prompt tuned for yesterday’s conditions may quietly degrade tomorrow.

Choi and Kim’s answer is to stop writing prompts by hand and instead let the model learn them. Their method, a few-shot anomaly prompt tuning approach, treats the prompt itself as a set of trainable parameters. Rather than fixed words, the prompt contains continuous embedding vectors that are optimized through gradient descent on a small handful of labeled examples, in this case as few as four images per category. This idea descends from a broader line of research on prompt tuning for vision-language models, which showed that optimizing soft prompts in the model’s embedding space can match or exceed the performance of painstakingly engineered text prompts. Applied to anomaly detection, the technique lets the model discover, from a tiny dataset, exactly how the language side of the system should describe normal and defective states for the product at hand.

But learning from four examples creates its own hazard, one familiar to anyone who has trained machine learning models on scarce data: overconfidence. When an anomaly map is generated from so little supervision, individual regions of an image can receive spuriously high anomaly scores, producing false positives, alarms raised on perfectly good parts. In a manufacturing setting, false positives are not merely an annoyance. Every false alarm triggers inspection, rework, or a halted line, and a detector that cries wolf too often will be ignored or switched off. The Yonsei team addresses this with a second contribution, a technique they call top-k anomaly score ensemble, or TASE. Instead of trusting a single model’s peak anomaly score, the method aggregates the top-scoring regions across multiple predictions, an ensembling strategy that smooths out the idiosyncratic errors any single pass can produce and tempers the model’s tendency to overreact to borderline cases.

The results, reported on the two most widely used benchmarks in industrial anomaly detection, are striking. On MVTec-AD, a comprehensive real-world dataset of manufacturing defects ranging from scratched metal to contaminated tiles, the proposed model achieves an area under the receiver operating characteristic curve, or AUROC, of 95.6 percent under a 4-shot setting, meaning it saw only four example images per product category before being tested. On VisA, a harder and more varied dataset of visual anomalies, it reaches 88.5 percent. AUROC is a standard measure of a detector’s ability to distinguish defective from normal samples across all possible decision thresholds, and scores in this range, achieved with so little training data, place the method among the competitive few-shot approaches in the field.

Efficiency is the study’s other headline claim. Adapting large pretrained models to new tasks is often computationally punishing, and prior prompt-based anomaly detection methods have carried significant inference-time costs. The proposed approach delivers a 92 percent improvement in inference speed over a prior method, a difference that matters enormously in practice. A production line inspects parts continuously, and a detector that runs slowly either inspects a sample of the output or forces expensive hardware investments. A detector that runs quickly can, in principle, examine every part in real time. The speed gain, combined with the minimal data requirement, points toward a deployment scenario in which a factory can adapt the system to a new product in minutes rather than days, using little more than a smartphone photo of a good part and a few defective ones.

The broader significance of the work lies in what it says about how specialized AI is being built. The old paradigm, collect a massive labeled dataset, train a bespoke model, deploy it, is giving way to a new one: take a general-purpose foundation model trained on the open internet, and nudge it toward a narrow domain with a handful of examples and a small number of trainable parameters. Prompt tuning sits at the heart of this shift because it avoids updating the enormous backbone of the model itself. Only the lightweight prompt vectors change, which keeps training cheap, reduces the risk of catastrophic forgetting of the model’s general knowledge, and makes adaptation feasible on modest hardware. The Yonsei study demonstrates that this recipe extends beyond classification and captioning into the quality-control trenches of manufacturing.

There are, of course, limits that the authors’ benchmarks do not erase. Four shots is a small but not negligible amount of supervision, and someone must still identify and label the defective examples. The reported AUROC figures, while strong, leave room for errors on subtle or novel defect types that differ from anything in the few-shot examples. And like all CLIP-based systems, the method inherits whatever blind spots and biases the underlying foundation model acquired during its internet-scale training. Still, the trajectory is clear. As vision-language models grow more capable, techniques like learned prompts and score ensembling are turning them from impressive generalists into practical specialists, ones that can walk onto a factory floor, look at four pictures, and start catching the defects that slip past human eyes. The work was supported by the National Research Foundation of Korea, and its code of results joins a fast-growing literature, including WinCLIP and related methods, that is rapidly redefining what is possible when inspection data is scarce and time is short.

Subject of Research: Few-shot visual anomaly detection and localization in manufacturing using prompt tuning of the CLIP vision-language model

Article Title: Efficient few-shot visual anomaly detection and localization via prompt tuning

Article References: Choi, J., & Kim, C. O. (2026). Efficient few-shot visual anomaly detection and localization via prompt tuning. Applied Intelligence, 56(15), Article 450. https://doi.org/10.1007/s10489-026-07497-3

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07497-3

Keywords: anomaly detection, prompt tuning, prompt engineering, vision-language model, CLIP, few-shot learning, industrial inspection, manufacturing, computer vision, MVTec-AD, VisA, machine learning