Smoothness Trick in Solar Aureole Data Catches Clouds That Fool Aerosol Sensors

Invisible saboteurs have long haunted one of atmospheric science’s most trusted data streams. Thin cirrus clouds, made of ice crystals so small they scatter sunlight much like airborne dust, slip past the automated filters designed to keep NASA’s AERONET aerosol measurements clean. Now a team led by Alexander Sinyuk of Science Systems and Applications, Inc. and NASA’s Goddard Space Flight Center has unveiled a deceptively simple fix: watch how smoothly the sky’s brightness changes over the course of a day. When that smoothness breaks, clouds are almost certainly to blame. The new technique, published in Atmospheric Measurement Techniques, promises cleaner aerosol data for climate models, satellite validation, and air quality research worldwide.

AERONET, the Aerosol Robotic Network, has been measuring aerosol optical depth (AOD) with ground-based sun photometers since 1993. Its data underpin estimates of how aerosols influence Earth’s energy balance and serve as the gold standard for checking satellite retrievals. But the entire enterprise rests on a fragile assumption: that the measurements describe aerosols, not clouds. A thin veil of cirrus can masquerade as an aerosol layer, inflating AOD values and distorting inferred particle properties. Cloud screening has therefore always been a critical component of the network’s data processing, and any undetected cloud contamination quietly corrupts downstream science.

The current Version 3 screening algorithm became fully automatic by exploiting the solar aureole, the bright region of sky surrounding the sun. The angular shape of sky radiance in the aureole depends on particle size, so the algorithm parameterizes that shape using the curvature of the radiance curve and compares it against thresholds calibrated with lidar observations. Modern Cimel instruments support this with a curvature scan (CCS) that measures sky radiance from 3 to 7.5 degrees scattering angle in 0.3-degree steps after every AOD measurement. Yet the method is fundamentally threshold-based, and when cirrus parameters fall outside the pre-set limits, the clouds go undetected. The problem is most acute at high latitudes, where in situ aircraft studies have shown that cirrus ice crystals can shrink to sizes comparable to aerosol particles, with integral crystal radii as low as roughly 3 micrometers in the Arctic.

The failure mode is vividly illustrated at the OPAL site in Eureka, Canada, at about 80 degrees north latitude. There, the researchers found that Level 2 AOD time series, already passed through Version 3 screening, still displayed sharp upward spikes in AOD paired with sharp downward spikes in the Angstrom exponent, a size-sensitive parameter. Both signatures point to large ice crystals contaminating the record. The curvature parameters showed telltale spikes too, but most fell short of the operational thresholds, so the clouds slipped through. Similar failures appeared across northern Canada, Greenland, and the Arctic, where cold temperatures favor small ice crystals that scatter light like aerosols.

The team’s insight came from a different way of looking at the same data. In cloud-free conditions, the diurnal variation of sky radiance measured at the 3.3-degree scattering angle, the second smallest angle of the curvature scan, should be smooth, because it is driven almost entirely by the slow, predictable change in solar zenith angle. Radiative transfer calculations confirm this smooth behavior for an aerosol-laden atmosphere. The researchers showed that cloud-free days at sites as different as Ilorin in Nigeria, dominated by heavy dust, and the DEWA Research Center in the United Arab Emirates yielded radiance time series that fit polynomials with high correlation coefficients. On partially cloudy days, by contrast, the smooth aerosol signature is interrupted by irregular spikes, and those spikes become the fingerprint of cloud contamination.

From this observation the team built what they call the FD algorithm. It takes the full day of CCS(3.3) radiance measurements and computes first differences between values at neighboring time stamps. The standard deviation of these differences is compared against a threshold, ultimately set at 3.0, estimated from cloud-free days. If the standard deviation exceeds the threshold, the algorithm flags the largest value in each offending pair as cloud contaminated and removes the associated AOD measurement, then recalculates and iterates. The process stops when the standard deviation drops below the threshold or begins to increase, the latter signaling that the algorithm has started eating into genuine aerosol variability or that the day is completely overcast with no smooth aerosol signature to preserve. Crucially, no polynomial detrending is applied beforehand; the raw measurements speak for themselves.

Applying the algorithm across all AERONET sites for 2022 revealed both its power and its limits. At fine-mode-dominated sites such as Thule in Greenland and Hokkaido University in Japan, the Angstrom exponent rose systematically after screening, while the number of AOD measurements fell, with the two changes strongly anti-correlated, exactly what one expects when large cloud particles are removed. But at Capo Verde, where transported desert dust dominates, the algorithm stripped out large numbers of observations with almost no change in the Angstrom exponent, indicating it was deleting cloud-free data. The team’s remedy was an elegant second criterion: track how the Angstrom exponent changes with each algorithm iteration. Cloudy cases show a steep, highly correlated linear increase, while cloud-free cases show a nearly flat, uncorrelated drift. The slope of a linear regression between iteration number and Angstrom exponent, with a threshold of 0.01, now vetoes the FD results whenever the exponent barely moves, restoring wrongly eliminated data. Satellite imagery from Terra and Aqua MODIS confirmed the logic on test days at Capo Verde, where the slope criterion saved roughly 25 percent of mostly cloud-free observations.

Validation came from an entirely independent source: the Micro-Pulse Lidar Network, or MPLNET, whose vertically resolved backscatter profiles are particularly sensitive to cirrus. The team collocated AERONET and MPLNET data in space and time, averaged over one-hour periods, and drew on eight stations from GSFC in the United States to Kanpur in India and Barcelona in Spain. Comparing the frequency distributions of FD standard deviations for 90 percent clear and cloudy conditions fixed the threshold at 3.0, consistent with the case-study estimate. Performance was scored with the Matthews Correlation Coefficient, a stringent metric that rewards accuracy across true positives, true negatives, false positives, and false negatives simultaneously. On average, the FD algorithm outperformed Version 3 Level 1.5 screening by about 0.02 in MCC, a modest but consistent gain that held across solar zenith angle thresholds of 30, 60, and 90 degrees.

The detailed metrics tell a nuanced story. The FD algorithm identified a higher percentage of true positives and fewer false negatives than Version 3 alone, meaning it caught more cloud-contaminated measurements, at the cost of only slightly more false positives. Both algorithms maintained high true negative rates, confirming reliable clear-sky identification. Both showed declining MCC at larger solar zenith angles, largely because the lidar points straight up while the sun photometer looks toward the horizon, so the two instruments can sample different patches of a variable sky. Globally, the FD test removed an additional average of about 3.1 percent of data points from Version 3 Level 1.5, a small fraction that nonetheless matters enormously for field campaigns and high-cirrus regions where every clean observation counts.

The new technique is slated for inclusion in AERONET Version 4, expected in 2027, where it will run as an augmentation to the existing curvature algorithm rather than a replacement. Its beauty lies in parsimony: no new instrument, no new thresholds on angular shape, just the recognition that the atmosphere’s aerosol burden changes slowly and gracefully while clouds punctuate the record with spikes. For a network whose data feed climate assessments, satellite calibration, and aerosol retrievals on every continent, teaching an old scan a new trick may be one of the quiet but consequential upgrades in modern Earth observation.

Subject of Research: Cloud screening of aerosol optical depth measurements using time series smoothness of solar aureole sky radiances in the AERONET network

Article Title: Employing smoothness of the time series of sky radiances measured in the solar aureole for cloud screening

Article References: Sinyuk, A., Eck, T. F., Slutsker, I., Lewis, J., Grigorov, P., Smirnov, A., Schafer, J. S., Sorokin, M., Lind, E., & Gupta, P. (2026). Employing smoothness of the time series of sky radiances measured in the solar aureole for cloud screening. Atmospheric Measurement Techniques, 19(18), 6125-6144. https://doi.org/10.5194/amt-19-6125-2026

Image Credits: AI Generated

DOI: 10.5194/amt-19-6125-2026

Keywords: AERONET, aerosol optical depth, cloud screening, cirrus clouds, solar aureole, sky radiance, curvature scan, MPLNET, Angstrom exponent, Matthews correlation coefficient, sun photometer, atmospheric measurement