In an extensive experimental study evaluating techniques to accelerate machine learning workflows, researchers S. Göttlich, J. Heieck, and A. Neuenkirch evaluated the practical efficacy of low-discrepancy point sets for large-scale data compression. Published in the Journal of Mathematics in Industry, the investigation systematically compared Quasi-Monte Carlo compression frameworks against clustering algorithms when training neural networks and approximating regression error functions. While theoretical numerical analysis has long celebrated low-discrepancy sequences for their deterministic uniform coverage in multidimensional integration, the experimental results demonstrated that adaptive clustering techniques significantly outperform Quasi-Monte Carlo methods across complex learning tasks. The findings provide critical practical insights into the trade-offs between rigorous non-random error bounds and data-driven heuristic reductions in contemporary applied mathematics and artificial intelligence.
Data reduction serves as a vital mathematical strategy designed to shrink extensive datasets into representative subsets while preserving the core underlying relationships needed for predictive modeling. In standard supervised regression settings, a dataset consists of N points within an s-dimensional unit cube paired with corresponding real-valued scalar responses. Practitioners seek to calibrate a parameterized predictor function by minimizing the empirical quadratic loss, defined as the mean squared distance between predicted values and actual observations across all training instances. However, optimizing modern deep architectures requires evaluating this loss function and its multidimensional parameter gradients over dozens or hundreds of successive training iterations. The resulting computational expenditure scales proportionally with the product of optimization steps and the total data size N. By compressing the original sample into a much smaller representative set of L points, where L is substantially smaller than N, engineers can evaluate an approximate objective function with dramatically lower per-iteration computational costs.
Quasi-Monte Carlo points, also known as low-discrepancy point sets, offer a deterministic methodology for generating evenly dispersed coordinates across an s-dimensional unit hypercube. Building upon foundational numerical quadrature concepts, a 2021 framework by Josef Dick and Michael Feischl proposed compressing known regression data using digital nets combined with an analytical averaging procedure. In this QMC-averaging scheme, the compressed coordinates are fixed in advance as digital net points, while sample information is captured through precalculated algebraic weights reflecting local point densities across elementary intervals. By isolating these weights from the model parameters, practitioners can precompute all geometric factors once prior to training, enabling rapid subsequent evaluations of the weighted surrogate loss function during iterative gradient descent.
A prominent conceptual attraction of the QMC-averaging framework is its foundation in rigorous deterministic error bounds derived through the Koksma-Hlawka inequality and Walsh series analysis. When using digital nets of order one constructed in prime base b, the bound on the absolute difference between the true empirical error and the compressed loss surrogate decays at an asymptotic rate of order one-half, comparable to random Monte Carlo subsampling but free of stochastic uncertainty. Furthermore, for digital nets of higher order alpha where alpha is at least two, theoretical convergence rates accelerate to alpha divided by the quantity alpha plus one, aside from polylogarithmic factors. These mathematical guarantees establish provable convergence under assumptions of sufficient integrand smoothness measured by specific partial mixed derivative norms and bounded Hardy-Krause variation. Nevertheless, achieving these theoretical convergence rates in computational practice requires satisfying strict algebraic relations governing partition resolutions and net parameters, highlighting potential discrepancies between theoretical bounds and empirical performance.
Despite appealing theoretical properties, the QMC-averaging formulation introduces substantial algorithmic complexity when calculating the required density and response weights across multidimensional spaces. Calculating these weights involves summing point counts and response values over numerous elementary intervals spanning multiple spatial partitions, necessitating alternating summations with binomial coefficients that are highly vulnerable to catastrophic numerical cancellation. To determine whether empirical shortcomings originated from the weight calculations or from the geometry of digital nets themselves, the researchers engineered an alternative baseline termed the QMC-Voronoi method. This hybrid technique maintains the deterministic digital net coordinates as fixed cluster centers but discards the complex averaging weights entirely. Instead, it partitions the input domain using Euclidean Voronoi cells, assigning each data point to its nearest digital net coordinate and computing compressed responses as simple arithmetic averages within each cell, thereby providing a clear structural comparison between spatial partitioning strategies.
To benchmark these Quasi-Monte Carlo formulations against state-of-the-art adaptive compression, the authors examined the supercompress algorithm, an innovative clustering strategy tailored specifically for regression. Conventional K-means clustering operates exclusively within the input feature space, minimizing geometric distances between observations and cluster centroids without regard to corresponding output values. In contrast, the supercompress method iteratively partitions data to directly minimize residual variance in the response space. Beginning with a single partition, the algorithm repeatedly identifies the cluster exhibiting the largest response loss and splits it into two subclusters via localized input clustering until reaching a designated target of K compressed points. The researchers also examined a robust supercompress variant governed by a convex trade-off parameter lambda, which blends input-space spatial dispersion with response-space homogeneity, where setting lambda to zero recovers the pure supercompress model and setting lambda to one yields classical unsupervised K-means clustering.
The team conducted controlled numerical experiments across several standardized test functions to evaluate approximation errors under uniform sampling with additive Gaussian noise. Testing included a continuous non-smooth exponential function, a discontinuous step-exponential function, and the smooth multivariant Zhou benchmark function across dimensions ranging from two to ten. Throughout these controlled synthetic trials, the supercompress algorithm consistently delivered superior approximation accuracy across various compression rates. The newly designed QMC-Voronoi method markedly outperformed the classical QMC-averaging approach, confirming that numerical instabilities in the alternating binomial weight sums severely impaired the averaging technique. Furthermore, execution time benchmarks demonstrated that QMC-averaging incurred severe computational penalties that doubled rapidly with increasing dimensions, whereas both supercompress and QMC-Voronoi displayed robust scaling behavior. Only at the highest tested dimension of ten did QMC-averaging exhibit localized error reductions on specific functions, though its excessive computational cost precluded practical utility.
To assess practical performance in realistic machine learning workflows, the authors tested all compression frameworks by training neural networks on the MNIST benchmark dataset of handwritten digits. The full dataset comprises sixty thousand grayscale images of twenty-eight by twenty-eight pixels, representing an input dimensionality of seven hundred and eighty-four. To render the intensive weight calculations tractable for digital net construction, the researchers implemented a precompression protocol on an initial subset of ten thousand images, downsampling each image to a fourteen by fourteen matrix with an effective dimension of one hundred and ninety-six. A neural network architecture was trained over one hundred epochs using Sobol construction matrices with partition parameter nu set to two. Baselines included networks trained on uncompressed data, pure supercompress, robust supercompress, standard unsupervised K-means clustering, and the QMC-averaging formulation, tracking prediction accuracy, final loss, training duration, and multi-class confusion matrices.
The neural network experiments revealed an acute operational breakdown for both Quasi-Monte Carlo compression frameworks when confronted with real-world image structures. The QMC-averaging method collapsed completely, failing to learn digit representations and predicting only a single constant numerical class across the test partition. Mathematical analysis revealed that the vast majority of image pixels are background zeros, concentrating the data distribution tightly near the origin of the unit cube. Consequently, the uniform digital net placed nearly all its weight onto its first two points, rendering the remaining coordinates irrelevant. The QMC-Voronoi approach failed even more severely, as every single image in the dataset mapped into the Voronoi regions of just the first two net coordinates, yielding an unusable training set of two points. Standard unsupervised K-means also performed poorly, as clustering purely across pixel intensities grouped disparate digit classes together, yielding erroneous cluster averages.
In sharp contrast to the failure of uniform point sets, the response-guided supercompress algorithm delivered outstanding classification fidelity and computational acceleration on the neural network benchmark. At a twenty percent compression rate, supercompress decreased required training time by approximately eighty percent while suffering only a modest four percent reduction in classification accuracy relative to the uncompressed baseline. When retaining forty percent of the compressed points, the algorithm halved overall training duration while remaining within zero point two percent of the uncompressed model accuracy. The authors concluded that while low-discrepancy points remain mathematically elegant for smooth, uniformly distributed problems, practical machine learning demands adaptive, response-aware compression methods. These empirical outcomes reinforce the classical applied mathematics maxim that whereas regular problems yield to rigid, non-adaptive spatial sampling, irregular real-world data structures necessitate adaptive algorithmic architectures capable of capturing non-uniform response landscapes.
Subject of Research: Data compression methods for regression and neural network training
Article Title: Using low-discrepancy points for data compression in machine learning: an experimental comparison
Article References: Göttlich, S., Heieck, J., & Neuenkirch, A. (2025). Using low-discrepancy points for data compression in machine learning: an experimental comparison. Journal of Mathematics in Industry, 15(1), Article 1. https://doi.org/10.1186/s13362-024-00166-5
Image Credits: AI Generated
DOI: 10.1186/s13362-024-00166-5
Keywords: data compression, machine learning, low-discrepancy points, quasi-Monte Carlo, supercompress algorithm, neural network training, digital nets, Voronoi clustering, K-means clustering, MNIST dataset, regression analysis, Sobol matrices, quadratic loss function, empirical prediction error, star-discrepancy, Koksma-Hlawka inequality, Niederreiter-Xing matrices, numerical integration
Cite Scienmag News
Blake Davidson. (November 18, 2026). Supercompress Outperforms Low-Discrepancy Point Methods in Machine Learning Data Compression Tests. Scienmag. https://scienmag.com/supercompress-outperforms-low-discrepancy-point-methods-in-machine-learning-data-compression-tests/
Blake Davidson. "Supercompress Outperforms Low-Discrepancy Point Methods in Machine Learning Data Compression Tests." Scienmag, 18 November 2026, https://scienmag.com/supercompress-outperforms-low-discrepancy-point-methods-in-machine-learning-data-compression-tests/. Accessed 18 November 2026.
Blake Davidson. "Supercompress Outperforms Low-Discrepancy Point Methods in Machine Learning Data Compression Tests." Scienmag. November 18, 2026. https://scienmag.com/supercompress-outperforms-low-discrepancy-point-methods-in-machine-learning-data-compression-tests/








