New AI Framework Blends Infrared and Visible Images by Seeing Others and Focusing Itself

In the world of computer vision, few tasks are as deceptively simple as looking at two pictures of the same scene and deciding what to keep from each. Infrared cameras capture the heat that objects radiate, cutting through darkness, smoke, and fog, while visible-light cameras record the textures, colors, and fine details that human eyes evolved to appreciate. Fusing these two streams of information into a single, coherent image is one of the foundational problems in robust scene perception, and it underpins applications ranging from autonomous driving at night to military surveillance, search-and-rescue operations, and industrial inspection. A new study published in Complex & Intelligent Systems by Shuang Ma, Jiangbin Zheng, Yang Li, and Chengcheng Ma of Northwestern Polytechnical University, working with Chengcheng Ma of Xi’an XiangTeng Microelectronics Technology, proposes a framework called S2Fusion that tackles this problem with an unusual combination of biological inspiration and physical analogy.

The core insight behind the work is captured in its evocative title: see other, focus self. The authors argue that many existing fusion networks suffer from a structural weakness. They typically extract features from the infrared and visible images independently, using separate encoding pathways, and only bring the two streams together late in the pipeline. This design limits fine-grained cross-modal interaction, meaning the network never truly learns, at the level of individual pixels and small neighborhoods, how the thermal signature of an object relates to its visible texture. At the same time, these methods often rely on general-purpose global modeling mechanisms borrowed from natural language processing, which can weaken the global consistency of the fused result and leave the fusion process difficult to interpret. S2Fusion is designed to address both shortcomings simultaneously through a local-to-global collaborative architecture.

At the local level, the framework introduces what the authors call the See-Other-Focus block, or SOF. The name reflects its inspiration in human visual perception: when we examine a scene, our attention to any given location is shaped by context, and we selectively amplify what matters while suppressing what does not. In the SOF block, the context used to guide this attention comes not from the image being processed but from the complementary modality. Information from the infrared image helps shape how the network aggregates local features of the visible image, and vice versa. Technically, this is achieved by using cross-modal context to generate dynamic convolution kernels, which are then applied to adaptively aggregate local features. Rather than applying a fixed, static filter everywhere in the image, the network computes a bespoke filter for each location, one that is tuned to highlight the complementary information that the other modality says is important.

This dynamic-kernel mechanism is more than an aesthetic choice. In infrared and visible image fusion, the two modalities carry fundamentally different kinds of information. An infrared image might show a pedestrian as a bright thermal blob even in complete darkness, while the visible image preserves the texture of clothing, the edges of buildings, and the gradations of daylight. A fusion network must decide, location by location, which source to trust and how to blend them. Static convolution kernels apply the same weighting everywhere, which can wash out critical details or introduce artifacts. By letting the complementary modality drive the generation of the aggregation kernels, the SOF block facilitates the selection and preservation of complementary information at a fine granularity, ensuring that a warm signature in the infrared channel can sharpen attention on exactly the visible-light structures that matter at that spot.

Local interaction alone, however, cannot guarantee that the final fused image hangs together as a coherent whole. A network that processes every neighborhood independently can produce locally plausible patches that contradict one another globally, creating inconsistent brightness, broken structures, or seams across the image. To solve this, the authors developed the second pillar of S2Fusion: the Shared Heat Conduction Operator, or SHCO. Its inspiration comes from physics, specifically the principle of heat conduction. Just as heat flows through a material until temperature equalizes, spreading energy from hot regions to cold ones according to physical laws, the SHCO spreads information across the image so that locally extracted complementary features are integrated into a globally consistent representation. The metaphor is not merely decorative; it provides a principled mathematical structure for long-range dependency modeling.

What makes the SHCO distinctive is where it performs this modeling: in the frequency domain. Instead of computing pairwise relationships between every pair of pixels in the spatial domain, which is computationally expensive and scales poorly with image size, the operator transforms features into the frequency domain and models long-range cross-modal correlations there. It uses shared frequency value embeddings, meaning that both modalities are represented in a common frequency-based vocabulary, allowing the network to capture how low-frequency structure and high-frequency detail from the infrared and visible sources relate to each other across the entire image. Frequency-domain approaches to long-range modeling have gained traction in recent deep learning research because the fast Fourier transform offers an efficient route to global receptive fields, and the SHCO leverages this efficiency while grounding the operation in the interpretable language of heat diffusion.

The combination of the two components gives S2Fusion a clear architectural narrative. First, the SOF blocks enrich the features of each modality with fine-grained, cross-modal guidance at the local level, so that every neighborhood already knows something about what its counterpart modality contains. Then, the SHCO coordinates these enhanced features through long-range dependency modeling, diffusing the locally gathered complementary information into a single globally coherent representation from which the fused image is reconstructed. The framework thus moves deliberately from local to global, with each stage addressing a failure mode of prior methods: the loss of fine-grained interaction in independently encoded pipelines, and the loss of global consistency in loosely coupled fusion schemes.

The authors evaluated S2Fusion extensively on benchmark datasets for infrared and visible image fusion, comparing it against state-of-the-art methods. According to the study, the framework consistently outperformed its competitors across three key dimensions: information fidelity, meaning how much of the source information from both modalities survives in the fused image; structural preservation, meaning how well edges, contours, and object boundaries are maintained; and visual consistency, meaning how natural and artifact-free the result appears to a human observer. These are precisely the qualities that matter in downstream applications. A fused image that loses thermal contrast can hide a person in the dark; one that breaks structural continuity can mislead a navigation system; one that flickers or shifts in brightness can fatigue human operators monitoring video feeds.

Beyond raw performance, the emphasis on interpretability sets this work apart in a field often criticized for black-box behavior. By anchoring its two central mechanisms in recognizable principles, attention guided by contextual information from another source, and diffusion of information following the logic of heat conduction, S2Fusion offers researchers a conceptual vocabulary for understanding why the network makes the fusion decisions it does. The physical analogy of the heat conduction operator, in particular, suggests a path toward fusion architectures whose global behavior can be reasoned about analytically rather than only observed empirically. The work was supported by the National Natural Science Foundation of China under Grant No. 62202385, and the article is published open access, making the full technical details available to the community.

The implications extend well beyond the fusion of infrared and visible imagery. The local-to-global design pattern, in which cross-modal context generates dynamic local operators and a physics-inspired frequency-domain mechanism enforces global coherence, could inform other multi-sensor problems, such as combining medical imaging modalities, merging radar and camera data for autonomous vehicles, or integrating satellite imagery across spectral bands. As sensing systems proliferate and the demand grows for machines that can perceive the world reliably in darkness, fog, and clutter, frameworks like S2Fusion point toward a future in which artificial perception borrows the best of both worlds: the selective, context-driven attention of human vision, and the lawful, globally consistent diffusion of physical processes. The study, received in August 2026 and published on 14 October 2026, marks a thoughtful step in that direction.

Subject of Research: A local-to-global deep learning framework for fusing infrared and visible images using cross-modal dynamic kernels and frequency-domain heat conduction modeling

Article Title: See other, focus self: local-to-global cross-modal guidance for infrared and visible image fusion

Article References: See other, focus self: local-to-global cross-modal guidance for infrared and visible image fusion. (n.d.). https://doi.org/10.1007/s40747-026-02547-2

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02547-2

Keywords: infrared and visible image fusion, S2Fusion, cross-modal interaction, dynamic convolution kernels, heat conduction, frequency domain, long-range dependency, computer vision, deep learning, scene perception, image processing, attention mechanism