The problem the researchers tackle is what they describe as a form of error propagation, closely related to hallucination in language models. When a tool-augmented pipeline invokes an OCR engine, most existing frameworks simply fold the returned text into the reasoning process, regardless of whether that text is noisy, spatially inconsistent, or plainly wrong. This becomes dangerous under realistic document conditions—low resolution, blur, occlusion, stylized fonts, and complex layouts—where OCR and layout parsers degrade substantially. The authors draw on the established distinction between intrinsic hallucinations, which contradict source evidence, and extrinsic hallucinations, which introduce unsupported information. Tool over-trust, they argue, behaves like extrinsic hallucination propagation: the model incorporates tool-generated text that is weakly supported by, or directly inconsistent with, what the image actually shows.

Why do powerful models fall for bad tool output? The study points to the limited grounding of large language models. Because language models operate largely through statistical pattern matching over textual co-occurrences rather than grounded semantic understanding, multimodal systems may privilege textual tool outputs simply because they arrive in linguistic form—even when visual evidence contradicts them. Existing tool-learning frameworks such as Toolformer and ToolLLM demonstrate that models can learn to invoke APIs effectively, but they primarily evaluate whether a tool was used and whether final performance improved. They rarely ask the more fundamental question: is the tool’s answer actually trustworthy for this particular document region?

The proposed answer is a conflict-aware uncertainty alignment framework with three working parts. First, a lightweight visual uncertainty estimation head—two convolutional layers producing a dense map over the image—predicts spatial regions where the model’s internal perception is likely to be unreliable. The head is trained with binary cross-entropy against region-level error targets derived from whether predictions were actually correct. Second, a spatial alignment mechanism converts scalar tool confidences into spatially grounded maps: OCR character-level confidences are projected onto the image plane through their bounding boxes, then transformed into tool uncertainty by subtracting from one, so that visual and tool reliability live on the same semantic scale and in the same coordinate system. Third, a conflict module computes the pointwise absolute difference between the two uncertainty maps, yielding a localized disagreement signal and a global conflict score for each tool.

On top of this representation sits a decision policy trained with reinforcement learning. Given the image, the query, the visual uncertainty map, and the aligned tool uncertainty maps, the policy chooses among three actions: trust the model’s internal reasoning, invoke a specific external tool, or reject an unreliable tool output before it contaminates downstream reasoning. The reward function is deliberately rejection-aware—it grants a bonus for correctly rejecting incorrect tool outputs, rewards correct predictions, and penalizes each tool invocation by a configurable cost. The policy is optimized using Proximal Policy Optimization (PPO), though the authors are careful to note that PPO is a practical optimization mechanism here, not a methodological contribution in itself, and that supervised gating or contextual-bandit approaches could in principle learn similar arbitration policies.

The empirical results are striking. Across four document-understanding benchmarks—DocVQA, FUNSD, SROIE, and PubLayNet—the framework achieved absolute gains of 5.8 percent on DocVQA, 4.9 F1 points on FUNSD, and 4.9 percent on SROIE over an always-call baseline, while cutting average external tool invocation by more than half. On DocVQA, the full framework reached 81.7 percent accuracy. The efficiency finding is perhaps the most counterintuitive: calling tools more often did not produce better answers. Unconditional tool usage introduced substantial error propagation, because faulty OCR text flowed directly into reasoning. The conflict-aware system instead settled at roughly one tool call per sample, invoking parsers only for genuinely ambiguous structures like nested tables and dense multi-column layouts, and skipping them for clean headers and well-aligned text.

Rejection behavior proved to be the framework’s signature capability. Measured by the proportion of incorrect external tool outputs correctly suppressed, the learned policy outperformed both heuristic threshold gating and supervised gating across all four datasets. Qualitative case studies show the mechanism in action: when OCR misreads a degraded text region, the visual uncertainty map flags the area, the aligned tool uncertainty diverges, and the resulting conflict spike prompts the policy to discard the OCR output and rely on internal multimodal reasoning. When the document is clean and the tool agrees with the image, conflict stays low and the tool’s contribution is welcomed. The system learns context-dependent trust calibration rather than blanket acceptance or rejection.

Robustness testing added a further layer of rigor. The researchers built a controlled tool-failure benchmark using Gaussian blur, motion blur, occlusion, geometric distortion, compression artifacts, low resolution, and stylized-font substitution—and crucially held out some corruption types from training to test generalization. The framework retained its arbitration advantage under these unseen degradations, suggesting the policy learned general reliability cues rather than memorizing specific corruption patterns. Ablation studies confirmed that every component matters: replacing spatial uncertainty with a scalar signal dropped accuracy to 78.9 percent, removing explicit conflict modeling dropped it to 79.6 percent, removing the rejection-aware reward dropped it to 79.4 percent, and abandoning learned policy optimization altogether fell to 74.6 percent.

The authors are candid about limitations. The conflict measure captures spatial disagreement but not semantic contradiction—two sources can agree spatially while disagreeing at a higher conceptual level. The approach assumes tools expose spatially localizable confidence, which is natural for OCR and layout parsers but harder for legacy systems returning only global scores. And controlled perturbations cannot fully reproduce real-world failures such as heterogeneous illumination, handwritten annotations, or multilingual content. Still, the practical implications reach well beyond the lab. Document automation, receipt processing, healthcare documentation, financial workflows, and archival digitization all depend on OCR and layout tools that fail in localized, hard-to-predict ways. A system that knows when to doubt its own assistants—and pays only for the help it actually needs—represents a meaningful step toward multimodal AI that is not just capable, but reliably self-aware about the limits of its evidence.

Subject of Research: Uncertainty-guided tool arbitration for tool-augmented multimodal document understanding

Article Title: Conflict-aware uncertainty alignment for tool-augmented document-centric multimodal intelligence

Article References: Muthusami, R., & Saritha, K. (2026). Conflict-aware uncertainty alignment for tool-augmented document-centric multimodal intelligence. Discover Artificial Intelligence, 6(1), Article 1359. https://doi.org/10.1007/s44163-026-02409-3

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02409-3

Keywords: vision-language models, tool augmentation, uncertainty estimation, OCR, document understanding, conflict-aware arbitration, reinforcement learning, PPO, hallucination, multimodal AI, DocVQA, selective inference

The scale of the literature review itself tells a story about how fast this field is moving. The authors searched databases such as Google Scholar and IEEE Xplore using terms including document shadow removal, shadow detection and removal, and document image enhancement, collecting work published up to September 2024 from authors in more than ten countries. Of the 41 selected publications, 30, or 73.2 percent, appeared at conferences while 11, or 26.8 percent, appeared in journals, a distribution that reflects the field’s preference for rapid dissemination at venues such as the Conference on Computer Vision and Pattern Recognition, the International Conference on Computer Vision, and ICASSP. China accounts for the largest share of output, with six journal and thirteen conference papers representing 46.3 percent of the total. The temporal trend is equally striking: for journal papers, 72.7 percent of publications are concentrated in 2023 and 2024, and conference output has climbed steadily since 2022, signaling a research area that has only recently reached critical mass.

To understand why shadow removal from documents is harder than it looks, consider what a shadow actually does to a page. It attenuates illumination non-uniformly, often with soft penumbral edges, and it interacts with the underlying content: thin strokes of text, colored figures, paper texture, and even the curvature of a thick bound book. Early benchmark datasets captured this difficulty only partially. The Adobe dataset introduced by Bako and colleagues in 2016 contains just 81 pairs of shadowed and shadow-free images captured with a Canon 5D Mark II, drawn from 11 documents with 5 to 9 variations in shadow intensity and shape, and it includes only light shadows on plain text pages. Jung and colleagues added 87 pairs featuring multiple cast shadows created by deliberately obstructing the light source, while Shah and colleagues produced a dataset of 100 images with deliberately harder shadows and sharp transitions using obstacles at varying distances from a Moto G5 Plus camera. Each of these early collections was small and limited to narrow document types, which constrained how well models trained on them could generalize.

The dataset landscape has since matured considerably. Kligler and colleagues expanded diversity with 300 image pairs spanning handwritten documents, printed documents, posters, and fonts, each category containing 7 to 10 documents with 8 to 12 shadow variations. Wang and colleagues built the Optical Shadow Removal dataset of 237 manually captured images of books, newspapers, and brochures shot on an iPhone XR specifically to address the scarcity of strong shadows. The most significant recent contributions tackle the scale problem head-on. Lin and colleagues created the Synthetic Document Shadow Removal Dataset with 8,309 pairs of synthetic shadow images, shadow-free images, and shadow masks generated from 970 documents under varied lighting and occluders, alongside a real-world companion set of 540 captured images. Matsuo and colleagues pushed synthesis further with a fully synthetic dataset that generates documents from text, graphics, and textures and renders shadows with a graphics engine. Most ambitiously, Zhang and colleagues assembled the Real Document Dataset, 4,916 real image pairs split into 4,371 for training and 545 for testing, covering papers, books, and brochures across complex scenes. The survey evaluates all nine datasets on scale, authenticity, and diversity, noting that synthetic data solves the pairing problem, since true shadow-free and shadowed captures of the same page under identical conditions are nearly impossible to collect in the wild, but that synthesizing realistic multi-source lighting remains an open challenge.

On the algorithmic side, the survey draws a fundamental line between conventional methods, which rely on manually designed features and physical models, and neural network-based methods, which learn features directly from data. Conventional approaches split into two families. Shadow map-based methods first estimate where the shadow lies, often by comparing local background colors to global ones, then use that map to correct the image. Bako’s pioneering algorithm estimated local text and background colors and generated a shadow map from their ratio, achieving an average mean squared error of 22.26, though its assumption of a constant background limited performance on varied pages. Kligler’s group reinterpreted image points as 3D point clouds to separate foreground from background and reached a structural similarity score of 0.943 on printed documents. Later work by Wang and colleagues refined background estimation with iterative shadow scaling, and a joint water-filling algorithm with adaptive chroma adjustment maintained brightness and color consistency across shadow boundaries.

Illumination-based methods take the opposite tack: rather than detecting shadow pixels directly, they model the lighting itself. Rooted in the physics of light transport and often built on Retinex theory, which separates an image into reflectance and illumination components, these methods correct the uneven illumination field so that shadows disappear as a byproduct. Classic examples include Zhang’s restoration system for thick bound documents, which used vertical projection profiles and connected component analysis to find shadow boundaries and was later deployed in a text retrieval project for the National Archives of Singapore. Jung’s water-filling algorithm treated pixel brightness as a terrain surface, simulated water flooding the valleys to estimate shadow artifacts, and removed them using a Lambertian surface model, achieving an average peak signal-to-noise ratio of 23.38. Other illumination approaches have delivered measurable downstream benefits: a method designed for the Tesseract optical character recognition engine raised character recognition accuracy by an average of 8.1 percent, and a black top-hat transform approach reached 99.54 percent recognition accuracy in OCR experiments, demonstrating that cleaner images translate directly into better machine reading.

The deep learning era has reorganized the field around two architectural philosophies. Single-stage methods perform everything inside one end-to-end network, simplifying workflows and enabling real-time inference. Generative adversarial networks dominated early efforts: DE-GAN used an encoder-decoder generator with a fully convolutional discriminator for document enhancement, UDoc-GAN tackled illumination correction in unpaired images by predicting background light features and improved character error rate by 3 percent, and a lightweight Laplacian pyramid network with input and output attention achieved a 35 percent relative improvement in mean average error while running in real time on mobile hardware. More recent single-stage systems bring in transformers and frequency analysis. FSENet used transformer modules for global color correction and cascaded sparse convolutions to restore high-frequency detail, reaching a PSNR of 28.67 and SSIM of 0.96 on a dataset of 7,620 high-resolution pairs, while ShaDocFormer integrated Otsu thresholding with transformer attention to detect shadow masks and achieved a PSNR of 29.46 on the Real Document Dataset. Diffusion models have entered the arena as well, with frameworks that treat binarization as a reverse diffusion process and others that preserve text features through differentiable modules during deblurring.

Multi-stage methods, by contrast, decompose the problem into sequential subtasks, typically detecting the shadow first and then removing it, which allows each stage to specialize. BEDSR-Net exemplifies the strategy, using a background estimation module to extract global background color and spatial distribution information before removal, and achieving average PSNR and SSIM scores of 37.55 and 0.9534 on synthetic and real images. Zhang’s color-aware background extraction network paired with a background-guided removal network posted an RMSE of 2.219, PSNR of 37.585, and SSIM of 0.983 on the Real Document Dataset, the strongest combined figures reported in the survey. ShadocNet employed transformers for global shadow remapping followed by pixel-level refinement, and DocDeshadower decomposed images into low- and high-frequency components with Laplacian pyramids before processing each with dedicated transformer branches. Across the benchmark tables, BGShadowNet recorded the highest average SSIM at 0.9431, while DCShadow-Net led on PSNR at 27.9608 and RMSE at 11.3178, though the authors caution that no single method dominates every dataset and that fair comparison on a unified benchmark remains an unfinished task.

Perhaps the survey’s most valuable contribution is its forward agenda, distilled into nine concrete recommendations. The authors call for large-scale benchmark datasets that better reflect real-world lighting, including multiple light sources and complex occluder shapes, along with multilingual document coverage and fine-grained shadow classification so that removal strategies can be matched to shadow types. They advocate multimodal fusion that combines OCR signals and 3D depth information with image processing, domain adaptation techniques to transfer knowledge from abundant synthetic data to real scenarios, and self-supervised or unsupervised learning to reduce dependence on costly annotations. On the deployment side, they highlight lightweight models for edge devices using compression and clustering, user-guided interactive removal, generative formulations that treat shadow regions as noise for GANs and diffusion models, and explainable methods that reveal why a model decided a region was shadowed. For a problem that most people encounter daily without realizing it has a name, the message of this first comprehensive survey is clear: the humble shadow on a photographed page has become a serious test bed for how well modern computer vision can understand light, physics, and text all at once, and the tools it produces will quietly shape how reliably the world’s paper heritage enters the digital age.

Subject of Research: Shadow removal from document images using conventional and neural network-based computer vision methods

Article Title: A comprehensive survey on shadow removal from document images: datasets, methods, and opportunities

Article References: Wang, B., Li, C., Zou, W., Zhang, Y., Chen, X., & Chen, C. P. (2025). A comprehensive survey on shadow removal from document images: datasets, methods, and opportunities. Vicinagearth, 2(1), Article 1. https://doi.org/10.1007/s44336-024-00010-9

Image Credits: AI Generated

DOI: 10.1007/s44336-024-00010-9

Keywords: document image processing, shadow removal, computer vision, deep learning, datasets, transformers, generative adversarial networks, diffusion models, OCR, image enhancement, Retinex theory, benchmark datasets

That insight anchors a study published in the journal AI & Society by Yael S. Hacohen of Tel Aviv University, who introduces the concept of “material uncertainty”: the condition of fragile textual objects whose meaning cannot be separated from the damaged, improvised, or unstable material forms through which they survive. Her argument targets a quiet but consequential problem in modern archives. Handwritten-text recognition, optical character recognition, automated translation, metadata generation, and pattern detection now promise to make dispersed, multilingual collections searchable at scale. These tools are no longer peripheral experiments; they are part of the ordinary infrastructure of archival work. Yet the very features that make Holocaust last writings historically and ethically charged—stains, creases, improvised surfaces, uncertain addresses, broken chains of custody—are precisely the features least easily stabilized as data.

Hacohen reconstructs the standard computational pipeline in technical detail. Preprocessing begins after image acquisition with binarization, deskewing, noise reduction, and contrast enhancement, operations that establish the categories through which an image will be read: text, background, distortion, noise. In the humanities scholar Johanna Drucker’s terms, data are produced rather than discovered—”capta,” actively taken rather than given. Segmentation then divides the page into processable spatial units such as text regions, columns, lines, and baselines. Processing converts visual marks into machine-readable characters, with modern handwriting-recognition systems relying on deep-learning architectures, including recurrent neural networks and encoder–decoder models with attention mechanisms, to infer probable words despite ambiguous letterforms. Post-processing follows: outputs are cleaned and normalized, named entities are tagged into searchable metadata fields, neural machine translation trained on historical corpora mediates linguistic access, and classification models sort documents by subject, genre, period, and provenance.

Each stage is genuinely valuable, and the study does not pretend otherwise. Machine learning has enabled large-scale transcription of handwritten records that would otherwise remain unsearchable, and it can recover text through stains, tears, and fading. But the pipeline privileges what can be stabilized—names, dates, standardized categories, recurring linguistic patterns—while whatever cannot be confidently parsed becomes “error” and whatever cannot be standardized becomes “noise.” Those residual categories, as classification scholars Geoffrey Bowker and Susan Leigh Star showed, are where systems accumulate their exclusions. The greater danger is not error but displacement: once an object has been scanned, segmented, transcribed, and indexed, the clean derivative may begin to circulate as though it had exhausted the damaged artifact from which it was derived. To test where that displacement occurs, Hacohen executed a diagnostic workflow trace on three Holocaust last writings chosen as materially heterogeneous limit cases.

The first is Eva Schulzová’s illustrated poem “One Night in Terezín.” Deported to Theresienstadt at the age of ten in December 1941, Schulzová composed the pencil-on-paper work two years later, intertwining realistic drawings of the Dresden barracks with a tercet and seven couplets, moving from the armed gendarme outside to imprisoned women whispering tales of home, and finally to a faintly penciled sketch of a lavish feast—roasted turkey and cake—beside the question of how many more nights they will endure, an answer she notes only God knows. Less than a month later she was deported to Auschwitz-Birkenau, where she was murdered. The workflow often succeeded technically: it recovered much of the verbal sequence and produced a richer searchable record. But background normalization treated yellowing and fold shadow as interference, thresholding forced a page of continuous tonal variation into foreground and background, and segmentation could not determine whether pencil was functioning as script, image, interruption, or scene—a distinction the manuscript itself does not sustain.

The metadata stage made the loss vivid. The system categorized the document as a handwritten lyric poem in rhyming couplets, recorded “illustrations: true,” and described the drawings as a guard booth, clock, window, bed scene, and “food still-life.” It captured the presence of drawings but not a single relation between words and images, eliminating the object as multimodal entirely. It misidentified a guarded prisoner hut as a guard booth, overlooked the sunset entirely, and reduced a grieving child’s imagined feast to a still-life label. The handwriting model, meanwhile, recovered much of the Czech text but its errors clustered around the very words that fix the poem to Terezín—the ghetto, the Dresden Barracks, the term for Jews—resolving uncertain words into plausible Czech at precisely the points that anchor the poem historically, without marking those readings as inference rather than recovery. The faint final couplet, where the child’s address becomes most unstable, was also where the material trace was least secure; and the metadata recorded the poem’s archival location without registering the complete absence of knowledge about how it survived.

The second limit case, Eliezer Heiman’s two inscribed clay tablets, exposes a categorical failure. A Hebrew and Yiddish writer hidden with his family during the liquidation of the Kovno ghetto, Heiman engraved his final words on fired clay on September 22, 1943, deliberately recalling the Tablets of the Law: the right tablet records the names of those concealed in “the house of slaves and the house of doom,” while the left attests that he wrote the columns “with my own two hands” as “a memorial for generations to come.” The tablets were recovered after the hiding place was bombed and burned in July 1944. Computationally, the object defeated the pipeline almost immediately. Because Heiman’s letters are incised and made visible through depth, shadow, and ochre infill—now partially flaked—the flaked patches share the scale, color, and contrast of the engraved strokes. Adaptive thresholding classified roughly a third of the clay surface as foreground; line detection collapsed because flaking filled the interlinear spaces. The inscription was not hidden behind removable noise; it survives through a damaged surface materially entangled with the writing. The study distinguishes this as a capture-level loss, fixed at acquisition, and recommends reflectance transformation imaging or photometric stereo to record surface geometry rather than flattening relief into a single intensity image.

The third case, Sal Bramson’s cloth fragment, shows how partial success can be the most insidious outcome. Arrested with his brother in Amsterdam in 1942, Bramson produced drawings and written messages on white cotton cloth during nine months in an Amsterdam prison, smuggling them to his mother hidden in a shirt collar through the weekly laundry circuit. The surviving fragment holds four numbered drawings of his cell with handwritten Dutch captions of architectural precision—dimensions, sightlines, the peephole, the one hour of hot food. The workflow could mask the cloth, suppress the periodic weave, and binarize strokes with reasonable confidence, but suppressing the weave weakens the evidence of the very medium that made the message possible, since the cloth could travel only because it was foldable, concealable cloth. Pixel classification separated pictorial from textual components, yet caption and drawing are one documentary apparatus: the caption guides the mother’s eye while the drawing gives it spatial force. The workflow separated as layout what functions as address, and lexical correction risked converting fragmented handwriting into plausible Dutch that was grammatically smoother but historically wrong. The brothers were deported to Sobibor on April 27, 1943, and murdered three days later.

From these traces Hacohen draws a design imperative rather than a rejection of computation. The remedy is a relation-preserving workflow instead of an extraction-driven one: one that keeps the original image, processed image, transcript, uncertain readings, translation, material annotations, and provenance notes in view at once; that documents what was enhanced, suppressed, inferred, and left undecidable; and that marks inference as inference, as when an early machine reading placed Heiman’s tablets in Vilna rather than Viliampolė and was corrected only by re-examination under magnification. The ethical demand, the study concludes, is not that uncertainty be romanticized—transcription, translation, and clarification remain necessary—but that clarification remain answerable to the damaged form through which the writing survived. The faintness of Schulzová’s final couplet, the depth and soot of Heiman’s clay, the crowded textile of Bramson’s cloth, and the advertisement beneath Radnóti’s last poem are not obstacles to the evidence. They are part of it, and the future of archival computation should be measured not only by what it can recover, but by whether it can preserve the uncertainty through which these last writings continue to speak.

Subject of Research: How computational archival workflows transform Holocaust last writings into machine-readable data and what is lost in that transformation.

Article Title: Material uncertainty in computational workflow in the archives: Holocaust last writings beyond machine-readable extraction

Article References: Material uncertainty in computational workflow in the archives: Holocaust last writings beyond machine-readable extraction. (n.d.). https://doi.org/10.1007/s00146-026-03378-y

Image Credits: AI Generated

DOI: 10.1007/s00146-026-03378-y

Keywords: material uncertainty, computational archives, Holocaust last writings, handwritten text recognition, digital humanities, machine-readable archives, archival materiality, OCR, HTR, cultural heritage, AI ethics, Material