Negative Spectral Estimates Are the Secret to Unbiased Turbulence Spectra

Every scientist who has ever stared at a gappy time series knows the frustration: the data arrive at perfectly regular intervals, but somewhere in the middle, whole stretches are simply gone. A sensor failed, a cloud blocked the view, a satellite track ended early. The instinctive fix is to fill the holes, interpolate, or pretend the missing values are zeros, and then compute a spectrum as if nothing happened. A new study published in Advances in Statistical Climatology, Meteorology and Oceanography shows that this instinct, and a popular recent shortcut built on it, quietly corrupts one of the most important numbers in fluid dynamics: the slope of the power spectral density, the fingerprint that tells researchers which regime of turbulence they are looking at.

The study, by Cédric Chavanne of the Institut des sciences de la mer at Université du Québec à Rimouski, revisits a problem that has haunted spectral analysis since the 1960s: how to estimate the power spectral density, or PSD, of a signal sampled at uniform intervals when some of those samples are missing. The PSD describes how the total power of a signal is distributed across frequencies, and its slope in log-log space is the standard diagnostic for turbulence regimes in meteorology, oceanography, and engineering. If the estimated slope is biased, the diagnosis of the physics is wrong, no matter how beautiful the plot looks.

The mathematical backbone of the paper is the Wiener-Khinchin theorem, a cornerstone of signal processing that connects two seemingly different views of a random process. The theorem states that the spectral density of a stationary random process is the Fourier transform of its autocorrelation function, a measure of how strongly the signal at one moment resembles the signal a short time later. For finite, discrete data, Chavanne derives a subtle but consequential refinement: the familiar periodogram estimator is exactly the finite Fourier transform of what he calls the circular unbiased estimator of the autocorrelation, not the standard biased or standard unbiased estimators found in most textbooks. This circular estimator wraps the time series around on itself, treating it as periodic, and it has a crucial property: it is positive semidefinite, which guarantees that its Fourier transform yields only positive spectral values when no data are missing.

Everything changes when samples vanish. Skipping missing samples is mathematically equivalent to replacing them with zeros, which multiplies the underlying signal by a sampling series of ones and zeros. In the frequency domain, that multiplication becomes a convolution, smearing the true spectrum with the spectrum of the sampling pattern. The circular autocorrelation estimator, computed naively, becomes biased because fewer data pairs contribute at each lag. The classical remedy, dating back to work by R. H. Jones in 1962, is to normalize each lag by the actual number of available data pairs, producing an unbiased autocorrelation estimate. But there is a catch that has bothered researchers for decades: this unbiased estimator is no longer guaranteed to be positive semidefinite, so its Fourier transform can dip below zero at some frequencies, producing spectral power estimates that are negative, which is physically nonsensical for any single realization.

That awkward negativity is precisely what a 2021 study by Gao and colleagues tried to eliminate by taking the absolute value of the Fourier transform of the unbiased autocorrelation estimator, forcing every spectral estimate to be positive. The trick was adopted in several subsequent publications, including analyses of satellite wind and wave data, dust storm observations, and even a widely reported claim about hidden turbulence in van Gogh’s The Starry Night. Chavanne’s central message is blunt: the absolute-value fix makes things worse, not better. By folding negative estimates into positive ones, the procedure injects a systematic bias that depends on the number and distribution of the missing samples, and the resulting spectra are more distorted than those from the classical biased estimators the fix was meant to improve. Spectral slopes computed this way, he argues, cannot be trusted for quantitative diagnosis of turbulence regimes.

The counterintuitive resolution is that negative spectral estimates are not a bug to be patched but a necessary feature of unbiasedness. For any single realization of a random process, an unbiased estimator must be allowed to wander below zero occasionally, because only then does the average over many independent realizations converge to the true, always-positive spectrum. Chavanne demonstrates this with synthetic data generated from fractional Brownian motion, a mathematical process whose spectral slope is known exactly. Using a Hurst parameter of one third, which corresponds to the famous minus five-thirds slope of isotropic turbulence in the inertial range, and a Hurst parameter of one half, corresponding to a slope of minus two, he generated ten thousand realizations of 1,024 samples each and then deleted a third of the samples using two different schemes: a Bernoulli mechanism, where samples vanish independently at random, and a batch-Bernoulli mechanism, where samples disappear in consecutive runs of random length, mimicking realistic data outages.

The results were striking. When averaged over ten thousand realizations, the unbiased estimators recovered the theoretical spectral slopes across nearly the entire frequency range for both missing-data schemes, while the standard biased correlogram and the widely used Lomb-Scargle periodogram showed clear distortions, including a leveling of the spectrum at high frequencies and a biased slope for the batch-missing case. The absolute-value estimator of Gao and colleagues was strongly biased under both schemes. Histograms of the estimates at a single frequency told the deeper story: the biased estimators always produced positive values, but their averages sat noticeably away from the true value, whereas the unbiased estimators produced distributions straddling zero, with negative values appearing regularly, yet their averages landed almost exactly on the truth. The price of unbiasedness is variance, and the standard unbiased estimator carried the largest variance of all.

To show the method works on real data, not just synthetic mathematics, Chavanne turned to laboratory measurements of decaying turbulence in an active-grid-generated air flow, a benchmark dataset collected in a return-type Corrsin wind tunnel and originally reported in 2003. The streamwise velocity was sampled at 40 kilohertz, and the fifteen-minute record was divided into 2,195 overlapping segments of 32,768 samples each. He then artificially deleted half the samples in each segment using the same Bernoulli and batch-Bernoulli schemes, a severity beyond which time series are usually considered unusable for conventional spectral analysis. Because the unbiased estimators are noisy with only a few thousand realizations, he applied band-averaging over 25 frequency bins per decade. Even with half the data gone, the unbiased estimators recovered the minus five-thirds spectral slope over the same two-decade frequency range as the complete record, while the absolute-value estimator remained strongly biased and the standard biased estimator failed under the Bernoulli scheme.

Beyond the conceptual correction, the paper offers practical advantages. Chavanne proposes using the circular unbiased autocorrelation estimator rather than the standard unbiased one, because the number of available data pairs at each lag is always greater or equal, which reduces the variance of the resulting spectra. Computing it via the fast Fourier transform also requires only half the memory, since the standard unbiased estimator needs the series zero-padded to separate overlapping contributions. An efficient algorithm, generalized to estimate the cross-spectral density of two gappy signals, is provided in a public Matlab implementation, lowering the barrier for other researchers to adopt the method.

The implications reach well beyond the wind tunnel. Spectral slopes diagnose turbulence regimes in the atmosphere and oceans, calibrate satellite retrievals of wind and waves, and underpin interpretations of scaling in systems as varied as geophysical flows and, controversially, works of art. Wherever the underlying data have gaps, and they almost always do, the temptation to force every spectral estimate positive introduces a hidden bias that scales with the missing fraction. Chavanne’s study, supported by the Natural Sciences and Engineering Research Council of Canada, argues that the community should embrace the ugly negative numbers, average them honestly, and let the truth emerge from the mean. Sometimes, in statistics as in life, the path to a positive answer runs directly through the negatives.

Subject of Research: Unbiased nonparametric estimation of the power spectral density from uniformly spaced data with missing samples

Article Title: Asymptotically-unbiased nonparametric estimation of the power spectral density from uniformly-spaced data with missing samples

Article References: Chavanne, C. (2026). Asymptotically-unbiased nonparametric estimation of the power spectral density from uniformly-spaced data with missing samples. Advances in Statistical Climatology, Meteorology and Oceanography, 12(1), 59-72. https://doi.org/10.5194/ascmo-12-59-2026

Image Credits: AI Generated

DOI: 10.5194/ascmo-12-59-2026

Keywords: power spectral density, missing samples, spectral analysis, turbulence, Wiener-Khinchin theorem, autocorrelation, periodogram, Lomb-Scargle periodogram, fractional Brownian motion, bias, time series, signal processing