New Open-Source Tool Fixes the Hidden Refraction Errors That Warp Underwater 3D Vision

When a camera peers through water, light plays a trick that most computer vision systems simply ignore. Every ray of light bends the instant it crosses an air–water boundary, and the amount of bending depends on the angle at which the ray strikes the surface. Standard multi-camera calibration assumes that light travels in straight lines, an assumption baked into the pinhole camera model that underpins nearly every 3D reconstruction pipeline in existence. Whenever a camera images a scene through an aquarium lid, a submersible viewport, or an open water surface, that assumption quietly fails, and the resulting errors propagate into every triangulated position, velocity estimate, and distance measurement downstream. A new open-source software package called AquaCal, published in the journal SoftwareX, now offers a rigorous, tested fix for one of the most common rig geometries in experimental science.

The scale of the problem is easy to underestimate because the failure mode is nearly invisible. When a pinhole calibration is fitted to data collected through a refractive interface, the optimization does not crash or warn the user; instead it absorbs much of the mismatch into an inflated focal length and spurious lens distortion. Earlier studies by Sedlazeck and Koch, and by Jordt and colleagues, characterized this behavior for cameras mounted close behind underwater ports, finding that residual triangulation error grows with the camera’s standoff from the interface and with the distance to the imaged point, eventually reaching several percent of the camera-to-point distance. Jordt’s group observed depth errors of roughly 20 centimeters for surfaces at 2 meters. Those studies examined standoffs of centimeters; a fixed camera array imaging an entire aquarium tank from a meter or more away operates an order of magnitude beyond that range, where the errors grow in direct proportion to the standoff.

What makes the bias especially dangerous is that the checks practitioners usually run cannot detect it. Distances between nearby reconstructed points remain sub-millimeter under both a pinhole model and a corrected one, across a wide range of detection noise. Revealing the systematic depth-axis error requires an absolute-position check or a held-out rigid object spanning a long depth baseline, measurements that few pipelines routinely make. The affected fields are broad, spanning behavioral biology, aquaculture monitoring, underwater archaeology, and robotic navigation, and once the bias enters at calibration it silently contaminates everything reconstructed from the model.

Refraction-aware calibration theory is itself well established. Treibitz and colleagues showed that flat-interface refraction produces depth-dependent distortion that no radial polynomial can capture across the imaging volume, Agrawal and collaborators derived the general multi-layer flat-refraction theory, and Chadebecq’s team extended refractive geometry to two-view reconstruction. Recent systems such as Refractive COLMAP, the CalibMar toolbox, and a virtual-imaging method for binocular laser scanners build on these foundations. But all three estimate a separate interface distance for each camera, treating one physical plane as many. For a fixed array sharing a single water surface, that per-camera approach leaves each camera’s position along the interface normal redundant with its own interface height, an estimate that Treibitz’s work shows to be ill-conditioned unless the target reaches the edge of the frame, where refraction is strongest. No existing open-source tool was built to exploit the shared interface as geometric structure.

AquaCal, developed by Tucker Lancaster and Patrick McGrath at the Georgia Institute of Technology, fills that gap. It is a pip-installable Python library that models refraction at a single planar interface explicitly, applying Snell’s law in both projection directions, and folds the physics into one joint bundle adjustment over the camera extrinsics, a single global interface height, and the per-frame placement of the calibration target. The world frame is oriented with its Z axis along the interface normal, so the interface becomes the single horizontal plane at a height the software calls water_z. Each camera’s physical standoff, the gap that enters Snell’s law, is then derived directly from that one shared parameter. In synthetic tests, replacing the shared parameter with a per-camera estimate placed one physical surface at twelve different heights spanning 11 millimeters, and recovered every camera’s standoff less accurately across all ten random seeds tested.

The calibration runs in three stages, each designed around the conditions under which its parameters are best estimated. Stage 1 estimates lens intrinsics from in-air video of a small ChArUco board, a chessboard pattern embedded with ArUco fiducial markers, using OpenCV and supporting standard, rational, and fisheye lens models. This separation matters because under refraction a camera’s focal length and its standoff trade off against each other, making a joint fit from underwater observations alone poorly conditioned, whereas in air the projection is unambiguous. Stage 2 seeds the final joint solve, which is non-convex and converges reliably only from a starting point already close to the solution, by chaining pairwise relative camera positions and fusing the redundant links through rotation averaging, without requiring any single board placement visible to every camera at once. Stage 3 then estimates everything together, minimizing the refractive reprojection error across all observations, with an optional second pass that unlocks each camera’s focal length and principal point only after the geometry has converged.

The engineering beneath the pipeline is as carefully considered as the geometry. AquaCal computes the Jacobian of its optimization by finite differences over an explicit block-sparse pattern, applying Curtis–Powell–Reid column grouping so that the number of evaluations is a property of a single observation rather than of the array, meaning the savings grow as cameras and, more steeply, frames are added. A Huber robust loss down-weights outlier corner detections without a separate filtering stage. Cameras with high-distortion optics or limited target visibility can be designated auxiliary, registered after the fact against the frozen shared parameters so they cannot perturb the primary solution, and flagged in the output so downstream users can weigh their accuracy cost. The whole optimization scales to rigs of a dozen or more cameras in under twenty minutes on a workstation, with peak memory use near 11 gigabytes, and terminates at a cost tolerance of 1e-8 rather than an iteration cap.

The validation results are striking. In synthetic tests of an idealized twelve-camera rig about a meter above a tank, with 0.5 pixels of Gaussian noise added, the refractive model held focal lengths near ground truth with a mean absolute drift of just 0.066 percent, while the non-refractive baseline drifted by 6.83 percent, consistently overestimating focal length to compensate for the apparent-depth compression that refraction introduces. The refractive model drove reprojection error to the noise floor at 0.498 pixels, while the baseline plateaued about 2.5 times higher. The gap widened dramatically in 3D reconstruction. Triangulating held-out board placements at depths from 0.07 to 1.47 meters below the surface, the refractive model’s depth-axis error stayed flat between 1.7 and 1.9 millimeters across the entire tested volume, while the baseline climbed from 17.6 millimeters to 243 millimeters at the farthest point, roughly 2.5 meters from the reference camera, an approximately 132-fold degradation. Local scale accuracy told a complementary story: the refractive model held inter-corner distance error flat between 0.38 and 0.42 millimeters at all depths, while the baseline traced a U-shaped curve, accurate only within the band the calibration target had swept.

On the physical rig, a thirteen-camera array mounted on a hexagonal ceiling structure about a meter above a 2460-liter tank with an open water surface, the primary cameras achieved sub-pixel mean reprojection accuracy of 0.82 pixels, and held-out triangulated inter-corner distances showed a mean absolute error of 0.258 millimeters, a 0.43 percent mean relative error across 7,762 comparisons, with minimal systematic scale bias. The rig is not a test bed but a production platform for continuous behavioral monitoring of freely swimming Lake Malawi cichlids, supporting multi-fish 3D positioning, inter-individual distance measurements, and animal–substrate interaction analyses. Two downstream libraries, AquaMVS for refraction-aware multi-view stereo of the tank substrate and AquaPose for 3D pose estimation of swimming fish, already consume AquaCal camera models directly, and manuscripts describing them are in preparation.

The implications reach well beyond aquaria. Because the model parameterizes refraction by an arbitrary ratio of refractive indices, the pipeline extends without modification to non-aqueous media such as oil baths, with reconstruction accuracy holding across indices from 1.333 to 1.55, and a side-mounted rig imaging through a vertical interface requires no change to the model. The software, released under an MIT license with full documentation, archived input data and results on Zenodo, and Jupyter tutorials introducing both the pipeline and the refraction problem itself, replaces the improvised one-off corrections laboratories have historically relied on with a standardized, comparable implementation. The authors point to multi-layer and curved interfaces, such as dome housings, and online recalibration from tracked-animal correspondences as the most immediate extensions, both of which sit within the existing optimization architecture. For any laboratory counting on cameras to measure the underwater world accurately, the message is clear: the bending of light at the water’s surface is no longer an acceptable unknown.

Subject of Research: Refraction-aware multi-camera calibration for imaging through planar air–water interfaces

Article Title: AquaCal: refraction-aware multi-camera calibration for planar air–water interfaces

Article References: Lancaster, T., & McGrath, P. (2026). AquaCal: refraction-aware multi-camera calibration for planar air–water interfaces. SoftwareX, 36, Article 103091. https://doi.org/10.1016/j.softx.2026.103091

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103091

Keywords: camera calibration, refraction, Snell's law, 3D reconstruction, bundle adjustment, multi-camera arrays, underwater imaging, computer vision, open-source software, behavioral biology, aquaculture, ChArUco board