The central problem the paper identifies is a coupled failure mode. Relative-depth models such as Depth Anything v2 and Marigold produce visually convincing depth maps that generalize well across domains, but their outputs are not metrically calibrated, so converting them to true distances typically requires per-image scale and shift fitting against ground-truth data, which is unavailable in real deployments. Conversely, metric-aware systems such as ZoeDepth, Metric3Dv2, DepthPro, ZeroDepth and UniDepth anchor absolute scale better, yet can blur object boundaries and produce locally inconsistent geometry when the scene distribution shifts. FuseDepth attacks both weaknesses simultaneously instead of trading one for the other.

The architecture works in two stages. In the first, a frozen Depth Anything v2 backbone is adapted with lightweight low-rank adapters, known as LoRA modules, and paired with a boundary-regularized local metric-scale field. Instead of applying a single global affine calibration, a small convolutional head predicts spatially varying scale and shift values, biased toward low-frequency variation and piecewise smoothness. Crucially, the regularization is relaxed near estimated occlusion boundaries, allowing metric scale to change legitimately where objects end, while penalizing arbitrary pixelwise calibration everywhere else. No per-image or per-dataset scale fitting occurs at inference.

The second stage injects explicit high-fidelity visual priors. A pretrained YOLOv8 detector proposes object regions, and each proposal is refined with BiRefNet, an image-matting model that recovers far sharper alpha boundaries than conventional segmentation masks. The gradients of these alpha mattes are aggregated into a scene-wide contour map, and overlapping or adjacent surface hypotheses are linked into a deterministic surface-interaction graph. From this graph, the method derives a soft jump set marking locations where depth may legitimately discontinuously change and where smoothness assumptions should be suspended.

A complementary geometric prior comes from Omnidata, a frozen monocular surface-normal estimator. Surface normals describe the local orientation of planes and curved surfaces, information that is orthogonal to boundary evidence. FuseDepth enforces a projective compatibility between predicted depth gradients and the normal field, computed under the camera’s intrinsic matrix, but this constraint is deliberately suppressed wherever the graph-derived jump set indicates an occlusion boundary. The result is a piecewise regularization strategy: smoothness and normal consistency hold within surfaces, while boundary-preserving correction takes over at discontinuities.

A compact residual network of roughly 0.2 million parameters then refines the coarse metric depth. Rather than simply concatenating inputs, the refiner emits three residual proposals per pixel, one each for contour, normal and coarse-depth corrections, together with softmax-normalized arbitration weights that mix them locally. These learned gates allow the model to favor the contour branch near detected boundaries and lean on normals or the coarse depth in smooth or uncertain regions. The trainable components are learned only through the final depth losses, without any explicit reliability labels.

Evaluation follows a deliberately strict protocol. FuseDepth is trained on roughly 50,000 images from NYUv2, ScanNet and KITTI, and tested zero-shot on three unseen benchmarks spanning panoramic HDR scenes with LiDAR depth, long-range driving footage and diverse indoor and outdoor RGB-D imagery. Relative-depth baselines receive only one affine mapping fitted once on the training mixture and then frozen. Across all targets, FuseDepth posts the lowest errors, reaching an absolute relative error of 0.105 on SYNS, 0.171 on DDAD and 0.182 and 0.276 on the indoor and outdoor DIODE splits, while also recording the lowest per-image log-scale bias, a direct measure of absolute-scale stability.

Mechanism-isolation experiments support the design choices. Removing the contour prior, the normal prior, or the graph structure each degrades accuracy, and replacing the learned gate with plain concatenation or fixed equal weighting is consistently worse. Deliberately corrupted external priors, such as alpha masks taken from the wrong image or normals from a different scene, cause only graceful degradation because the gate shifts its mass away from the corrupted branch. Statistical analysis shows the improvements over the strongest baseline, UniDepth, are significant at the 95 percent level on SYNS and DIODE, while the DDAD advantage is small but reproduced across three independent training seeds and all geographic subsets.

The authors are candid about limitations. The pipeline depends on the quality and cost of its external priors, runs at 15.4 frames per second with 7.4 gigabytes of peak GPU memory, and cannot fully certify that the frozen upstream models never saw the evaluation datasets during their own pretraining. Low-quality contour priors naturally occur on roughly 6 to 12 percent of target images. Even so, the study demonstrates that thoughtfully composing existing foundation models can beat monolithic systems on strict metric transfer, suggesting a future where visual AI advances by orchestrating specialized experts rather than simply scaling up single networks.

Subject of Research: Zero-shot metric monocular depth estimation via fusion of frozen foundation visual priors

Article Title: FuseDepth: Zero-shot metric depth with semantic–geometric fusion of foundation visual priors

Article References: Javidnia, H. (2026). FuseDepth: Zero-shot metric depth with semantic–geometric fusion of foundation visual priors. Machine Learning with Applications, 26, Article 101010. https://doi.org/10.1016/j.mlwa.2026.101010

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101010

Keywords: monocular depth estimation, zero-shot learning, metric depth, computer vision, foundation models, Depth Anything v2, image matting, surface normals, LoRA, machine learning, depth discontinuities, prior fusion

The research, led by Shu Zhang of Huazhong Agricultural University together with Jiapeng Chen, Huan Shen, Quan Liu, and corresponding author Jing Xie, addresses a problem that has long plagued practitioners: labeled stereo data is scarce, expensive, and often unavailable for the specific environments in which a vision system must operate. Stereo matching works by comparing two images captured from slightly offset cameras and computing disparity, the horizontal shift of corresponding pixels, which is inversely proportional to depth. Networks trained on benchmark datasets such as Scene Flow learn this correspondence task superbly, but the statistical properties of those images differ from those of new deployment scenes in lighting, texture, camera geometry, and noise characteristics. This distribution mismatch, known as domain shift, degrades disparity predictions precisely in the regions that matter most for safety, such as thin structures, reflective surfaces, and textureless expanses.

Previous approaches to adaptation have taken several routes. Some methods fine-tune on the target domain using self-supervised photometric losses, exploiting the fact that a correct disparity map should allow one stereo view to be reconstructed from the other. Others rely on domain translation, synthesizing target-like images, or on confidence-based selection of reliable predictions. The new framework departs from these strategies by drawing on an entirely different source of supervision: the recent generation of monocular depth foundation models, such as Depth Anything, Metric3D, and UniDepth, which have been trained on massive unlabeled image collections and generalize impressively across scenes. Rather than ignoring this wealth of relative depth knowledge, the authors harness it as weak ordinal supervision for the stereo network.

The key insight lies in how that supervision is used. Converting monocular depth predictions directly into metric disparity labels is fraught with difficulty. Monocular depth estimates suffer from scale ambiguity, meaning the model may predict depths that are globally stretched or compressed relative to true geometry. Moreover, when depth is expressed in inverse form, as is common in disparity-based formulations, errors are amplified for distant objects, where small absolute depth errors translate into large inverse-depth errors. The team sidesteps both pitfalls by using only relative depth ordering. Their proposed Relative Disparity Loss enforces a simple ordinal constraint: whenever the monocular model judges one pixel to be closer than another, the stereo network is encouraged to predict a larger disparity for that pixel. Because ordinal relationships are invariant to scale and far more robust than absolute values, this constraint transfers reliable geometric knowledge across domains without importing the monocular model’s metric errors.

Ordinal cues alone, however, cannot guarantee accurate and stable adaptation, so the framework adds a consistency-aware learning mechanism with two complementary components. The first is multi-resolution prediction consistency. The stereo network’s predictions are computed at multiple input scales, and the variance across these scales serves as a pixel-wise reliability estimate. Pixels whose disparity predictions fluctuate wildly when the image is resized are deemed unstable and down-weighted in the training loss, while pixels that remain consistent across scales are trusted more heavily. This automatic reliability weighting protects the adaptation process from being corrupted by unreliable regions such as occlusions, sky, and reflective surfaces, which are precisely the areas where naive self-supervision tends to fail.

The second component is feature-level stereo consistency. Instead of constraining only the final disparity maps, the method enforces alignment between left and right image features after warping by the predicted disparity. This feature-space constraint gives the network a richer learning signal than photometric reconstruction alone, because it operates on learned representations that encode semantic and geometric structure rather than raw pixel intensities. As a result, the model becomes markedly more robust under illumination variations, one of the most common causes of performance collapse when synthetic-trained models meet the real world. Importantly, both the monocular depth model and the multi-resolution prediction machinery are used only during training; at inference time, the adapted stereo network runs as a standard, efficient two-view model with no additional computational burden.

The experimental evaluation spans three of the most demanding benchmarks in stereo vision: KITTI, with its autonomous-driving imagery; Middlebury, featuring high-resolution indoor scenes with subpixel-accurate ground truth; and ETH3D, which includes challenging textureless and outdoor environments. Across these targets, the framework achieves competitive or superior domain adaptation performance compared with existing self-adaptive methods, all without ever touching real-world ground-truth disparity during training. Qualitative results are particularly striking in the regions that have historically defeated stereo algorithms. In textureless walls where correspondence matching has no traction, on specular and reflective surfaces where photometric assumptions break down, and around thin structures such as poles and branches where disparity discontinuities are sharp, the adapted network produces visibly cleaner and more coherent depth maps than baselines.

The significance of this work extends beyond a single benchmark leaderboard. It demonstrates a practical recipe for combining the generalization strength of large monocular foundation models with the metric precision of stereo geometry, using the former to supervise the latter in a way that respects the strengths and weaknesses of each. Ordinal supervision sidesteps scale ambiguity, consistency weighting filters out unreliable pixels, and feature alignment hardens the network against lighting shifts. For industries deploying 3D vision, from agricultural robotics to autonomous navigation and industrial inspection, the approach promises models that can be adapted to a new site using only unlabeled stereo pairs captured on location, dramatically reducing the cost and time of deployment.

The study was supported by the National Natural Science Foundation of China under grant 42271357 and the Biological Breeding-National Science and Technology Major Project under grant 2023ZD04029. As monocular depth foundation models continue to improve, the framework’s reliance on relative depth ordering positions it to benefit automatically from future advances, since better ordinal predictions will yield stronger adaptation signals. The work points toward a future in which the divide between benchmark excellence and field reliability finally narrows, allowing stereo vision systems to earn their impressive numbers where it counts: in the unpredictable, uncontrolled, and unlabeled real world.

Subject of Research: Self-supervised domain-adaptive stereo matching using monocular depth cues and consistency-aware learning

Article Title: Domain adaptive stereo matching with consistency-aware learning and monocular depth cues

Article References: Zhang, S., Chen, J., Shen, H., Liu, Q., & Xie, J. (2026). Domain adaptive stereo matching with consistency-aware learning and monocular depth cues. International Journal of Machine Learning and Cybernetics, 17(9), Article 459. https://doi.org/10.1007/s13042-026-03295-y

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03295-y

Keywords: stereo matching, domain adaptation, monocular depth estimation, consistency-aware learning, self-supervised learning, disparity estimation, depth perception, computer vision, KITTI, Middlebury, ETH3D, deep learning