Optimized fast radiative transfer improves FY-3F water-vapor channel simulations

A targeted change to a fast radiative transfer algorithm reduced simulated errors in the strongest water-vapor absorption channels of the Fengyun-3F satellite’s Hyperspectral Infrared Atmospheric Sounder II, according to a development and technical paper that compared the method with line-by-line calculations and satellite observations. The researchers modified the RTTOV v13.1 scheme by screening unstable training samples and weighting the least-squares regression used to derive atmospheric transmittance coefficients. Their most comprehensive configuration, LSM3, produced the smallest deviations from the LBLRTM v12.11 benchmark and lowered the mean observation-minus-background bias at the 1506.875 cm−1 absorption-peak channel by about 0.1 kelvin relative to the baseline RTTOV model.

The work addresses a vulnerable part of infrared atmospheric remote sensing. HIRAS-II is a hyperspectral Michelson interferometer on FY-3F that measures 3,053 channels between 650 and 2,550 cm−1. It is designed to support vertical retrievals of temperature, humidity, ozone, and atmospheric composition from the surface to the lower stratosphere. Those retrievals and the assimilation of satellite observations into numerical weather prediction systems depend on a forward model that can reproduce the radiance observed in each channel. Line-by-line models provide highly accurate calculations but are too computationally expensive for routine operational use, so fast models approximate atmospheric transmittance with polynomial relationships between predictors and layer coefficients.

RTTOV represents such a compromise between speed and fidelity. The model calculates optical depth as the negative logarithm of channel transmittance, then estimates that quantity from predictors involving pressure, temperature, humidity, and viewing geometry. Its coefficients are fitted by least squares using a standard set of 83 atmospheric profiles covering 101 pressure levels from 1,100 to 0.005 hPa and six satellite zenith angles. The profiles span temperatures from 159.62 to 318.26 kelvin and represent a wide range of seasonal and meteorological conditions. For this study, LBLRTM v12.11 generated monochromatic transmittance at 0.001 cm−1 resolution from 650 to 2,600 cm−1, accounting for 26 uniformly mixed gases, water-vapor lines and continuum, ozone, six viewing angles, and refractive effects.

The central difficulty emerges near the 6.7-micrometer water-vapor band. When water vapor is abundant, the cumulative transmittance from the middle or lower atmosphere to the top of the atmosphere can become extremely small. Calculating the optical depth of an individual layer then involves dividing by a very small ratio of successive transmittances, making the regression numerically unstable. The problem is compounded by the HIRAS-II spectral response function: because its truncated interferogram produces a sinc-shaped response with oscillating wings, convolution can create physically unrealistic negative channel transmittance in strong absorption bands. Transmittance should lie between zero and one, so such values distort the absorption represented by the regression.

The researchers first introduced a channel-dependent screening threshold. When cumulative transmittance from a level to the top of the atmosphere fell below 10−4, samples beneath that level were excluded from the layer-by-layer regression because they were considered vulnerable to computational errors. Analysis of the 83 profiles at six zenith angles produced 498 atmospheric samples per pressure level. A threshold of 10−5 was found to remain numerically unstable, whereas an excessively large threshold could prematurely discard useful lower-atmosphere information, particularly in dry profiles. The selected 10−4 threshold balanced numerical stability against retention of training data; it had little effect on channels in the wings of the absorption band but removed more samples from lower layers as a channel approached the absorption peak.

Four schemes were then evaluated. LSM0 represented the unfiltered baseline with ordinary least squares. LSM1 applied the 10−4 threshold and still used ordinary least squares. LSM2 added a cumulative-transmittance weight to the regression, giving less influence to near-surface layers in which the radiation reaching the atmosphere has already been strongly absorbed. LSM3 additionally weighted both the predictors and optical-depth terms before fitting. The basic weight was the cumulative transmittance from a layer to space multiplied by itself, reflecting the progressively weaker contribution of lower layers. The analysis was conducted for channels near the left wing, center, and right wing of the 6.7-micrometer band, including 1380, 1423.75, 1506.875, 1646.875, and 1,730 cm−1.

Against LBLRTM, thresholding removed much of the instability seen in the baseline. For the 1380 cm−1 channel, its effect was concentrated mainly below about 600 hPa in dry profiles, where the improvement was clearest; in water-rich profiles, the effect was small. At 1506.875 cm−1, the threshold produced a more marked improvement, while its effect remained limited in channels dominated by weaker trace-gas absorption near the band edges. Across the 1,300–1,900 cm−1 medium-wave spectrum, the mean brightness-temperature RMSE declined progressively from about 0.10 kelvin for LSM1 to 0.05 kelvin for LSM2 and 0.045 kelvin for LSM3. The corresponding mean maximum deviation fell from approximately 0.45 to 0.22 and 0.15 kelvin. LSM3 also limited the maximum channel-transmittance deviation to below 0.1, with mean and standard deviation below 0.02 in most medium-wave channels.

The optimized scheme was then tested with clear-sky FY-3F HIRAS-II observations using ERA5 temperature and humidity profiles as model inputs. The comparison covered latitudes from 60° south to 60° north, with observations from August 2024 and February 2025 representing summer and winter. Clear-sky fields of view were selected through geometric overlap with higher-resolution MERSI cloud masks, requiring all overlapping MERSI pixels to be cloud-free. The test used the same coefficient file, predictors, emissivity treatments, cloud screening, and observation data for both RTTOV and LSM3, changing only the transmittance parameterization. The resulting brightness-temperature RMSE was about 1 kelvin for both schemes in the illustrated cases, with larger values in summer than in winter. LSM3 generally reproduced the spatial bias pattern of baseline RTTOV rather than introducing a new error structure.

The clearest observation-based gain appeared at the absorption peak rather than in the band wings. At 1506.875 cm−1, the mean observation-minus-background bias for RTTOV was approximately 0.40 kelvin over combined sea and land surfaces in August 2024 and 0.30 kelvin in February 2025. With LSM3, those values fell to about 0.31 and 0.21 kelvin, respectively, while the standard deviation remained comparable. The observation-minus-background RMSE time series for LSM3 and RTTOV nearly overlapped, indicating that the optimization chiefly reduced systematic bias rather than markedly changing the overall error magnitude. Away from the peak, the optimized and baseline simulations remained very similar, with their brightness-temperature differences generally below 0.1 kelvin and transmittance differences near zero.

Several limitations remain. The evaluation concentrated on clear-sky, low-to-middle-latitude conditions and two representative months, rather than the full range of seasons, cloud regimes, surface types, and atmospheric states encountered in operational use. The authors identified possible contributions from errors in ERA5 profiles, uncertainty in land-surface emissivity, instrument calibration, and residual thin or sub-pixel clouds; they noted that a change of 0.01 in emissivity can produce roughly 0.5 kelvin of brightness-temperature deviation in infrared window channels, although the effect is generally smaller in absorption channels. Relaxing clear-sky screening to admit as much as 5 percent cloudy MERSI pixels increased biases for both schemes but did not change the relative advantage of LSM3. The study is a technical evaluation rather than evidence that the new coefficients have already entered operational FY-3F data assimilation, although the authors argue that the physically motivated screening and weighting strategy could also be adapted to strong carbon dioxide or ozone bands and to HIRAS instruments on other Fengyun platforms.

Subject of Research: Optimization and satellite validation of fast atmospheric transmittance calculations for FY-3F HIRAS-II water-vapor channels

Article Title: Optimization of the fast layer transmittance algorithm in RTTOV v13.1 for strong water vapor absorption channels of the FY-3F HIRAS-II instrument using LBLRTM v12.11

Article References: Zhang, P., Zhang, P., Ma, G., Li, R., Lee, L., Bai, W., & Qi, C. (2026). Optimization of the fast layer transmittance algorithm in RTTOV v13.1 for strong water vapor absorption channels of the FY-3F HIRAS-II instrument using LBLRTM v12.11. Geoscientific Model Development, 19(20), 10077-10101. https://doi.org/10.5194/gmd-19-10077-2026

Image Credits: AI Generated

DOI: 10.5194/gmd-19-10077-2026

Keywords: RTTOV v13.1, LBLRTM v12.11, FY-3F, HIRAS-II, water-vapor absorption, radiative transfer, satellite remote sensing, transmittance regression, observation-minus-background bias, numerical weather prediction, Optimization, fast