Neural Networks Trained on a 24-Dimensional Lattice Could Shrink AI to 0.76 Bits per Weight

Every large language model in operation today stores its knowledge in millions or billions of floating-point numbers, each one a small, precisely measured quantity that occupies dozens of bits of memory. A new theoretical study published in Complex & Intelligent Systems argues that this continuous picture of neural computation may be an unnecessary luxury. Alexander Lavicka, an independent researcher based in Vienna, proposes treating the training of an artificial neural network not as a smooth trajectory through continuous space but as the evolution of a genuinely discrete complex system, one whose every parameter is locked from the start onto the points of a remarkable mathematical object: the Leech lattice, the densest known sphere packing in 24 dimensions.

The central claim of the paper is that conventional approaches to shrinking neural networks get the timing wrong. Post-training quantization, the standard industrial technique, first trains a network in full floating-point precision and then rounds its weights down to lower-precision formats after the fact. Scalar quantization methods treat each weight independently, ignoring the geometry that connects parameters to one another. Lavicka argues that both strategies neglect what he calls the topological limits of optimal information routing during the active learning phase. Instead of discretizing after training, his framework projects the network’s continuous parameter states onto the Leech lattice dynamically, throughout optimization, so that the model lives inside a vector-ternary logic from the moment of initialization.

The choice of lattice is not arbitrary. The Leech lattice, denoted Lambda-24, is a legendary structure in mathematics: a 24-dimensional arrangement of points that achieves the densest possible packing of spheres in that space, with no vectors of short length and an extraordinarily rich symmetry group. Each lattice point, or code atom, sits at a fixed distance from the origin, and the lattice contains exactly 196,560 such minimal vectors. Decoding a continuous vector to the lattice means finding the nearest atom, a task that corresponds to assigning the vector to the Voronoi cell it falls within. These Voronoi cells, the regions of space closest to each lattice point, are highly symmetric, and the paper leans on that symmetry for one of its most striking claims.

According to the study, the symmetric Voronoi cells of the Leech lattice act as optimal geometric filters during training. When stochastic gradient descent computes an update, the signal it carries is a mixture of coherent structural momentum, the meaningful direction in which the network’s parameters should evolve, and isotropic noise, the random jitter that arises from mini-batch sampling and other sources of stochasticity. Because the Voronoi cells partition space symmetrically in all directions, the quantization step rejects noise components that point equally in every direction while preserving the coherent component that carries the network’s learning signal. In this picture, the rigid discrete constraint is not a handicap imposed on training but an active filter that shapes it, grounded in the framework of rate-distortion theory, the branch of information theory that quantifies how much a signal can be compressed before its fidelity degrades beyond a chosen threshold.

The compression figures that emerge from this analysis are dramatic. Lavicka demonstrates mathematically that coupling a network’s weights natively to the Leech lattice compresses the continuous parameters to a theoretical density of 0.76 bits per parameter, well below even a single bit. The argument rests on a two-axiom foundation from which the codebook of lattice atoms is derived, together with an exact Voronoi decoding rule. Because the codebook has a fixed, finite set of atoms and the decoding rule is closed-form, the entire weight structure of a trained network can be stored as a list of integer indices into the lattice, each index requiring only 18 bits to address the 196,560 possible minimal vectors plus the zero vector.

To show that the idea is more than an elegant abstraction, the paper presents a proof-of-concept implementation trained on commodity GPU hardware. Two transformer models, named Pollux-1152 and Pollux-1920, were trained for 10,000 steps within the lattice-constrained space. The smaller model has a 287-million-parameter backbone embedded in a 404-million-parameter artifact, while the larger has a 796-million-parameter backbone within 991 million total parameters. When packed for deployment, the smaller model’s backbone occupies just 27 megabytes on disk and the larger’s just 76 megabytes, both at the predicted 0.76 bits per parameter. The architecture’s dimensions are dictated by the lattice itself: every hidden dimension must be an integer multiple of 24, so attention heads are exactly 24 units wide, embedding widths are 1,152 or 1,920, and even the vocabulary of 50,257 tokens is padded to 50,688 so that it divides cleanly by 24.

Notably, the training runs required no learning-rate schedule, no warmup phase, and no weight decay. The kinetic coefficients that govern how parameters move through the discrete state space are fixed before training by the geometry of the lattice itself, a property the author describes as endogenous kinetic coupling. The empirical results show stable training convergence within this rigidly constrained discrete space, suggesting that the topological bottleneck does not prevent learning, although it does change the character of what the models learn.

That change is visible in the models’ text generation. In qualitative examples included in the paper, both Pollux checkpoints produce text with strong structural fluency: grammatical constructions are consistent, morphological agreements are correct, and rhetorical transitions are coherent. Yet the models are factually unconstrained, inventing linguistically plausible but scientifically nonexistent concepts such as endolymphadoproteins or plasmid plates when asked about mitochondria or plate tectonics. The author attributes this to the topological bottleneck delaying the accumulation of high-entropy factual information, and frames it as consistent with the intended behavior of a stateless cognitive architecture, a design philosophy in which the network captures structure and syntax rather than a memorized store of facts.

There is an important engineering caveat. Modern GPU tensor cores are optimized for dense floating-point arithmetic and cannot natively parse the packed 18-bit combinatorial indices that the lattice encoding produces. The reference implementation therefore materializes decoded lattice vectors into dense half-precision weight matrices at each forward pass, a step that is functionally exact, reproducing bit-identical decoding results, but which does not yet deliver the runtime memory-bandwidth savings that the 0.76-bit code rate promises. Realizing that dividend, the paper notes, would require a native gather-lookup execution substrate, a hardware design that the author has partially covered with a pending international patent application on implementation-level software and hardware aspects, while explicitly leaving the underlying mathematical framework open for independent replication.

The study is careful to position itself as a proof of concept rather than a finished technology. Its stated aim is to explore the fundamental structural principles and theoretical limits of sub-1-bit neural discretization, providing initial empirical evidence that stable syntactic convergence can emerge naturally from rigid topological constraints. If the framework holds up under independent scrutiny, the implications reach beyond compression. Memory-bound hardware, in which the cost of moving data dominates the cost of computing it, is a central bottleneck of modern AI, and a scheme in which weights are natively discrete, geometrically structured, and addressable in under a bit per parameter offers a theoretical foundation for a new class of devices and, perhaps, for discrete cognitive architectures that treat intelligence itself as a lattice-constrained dynamical system. Whether the Leech lattice’s legendary symmetry proves to be the right geometry for machine learning at scale remains an open question, but the paper makes a provocative case that the densest packing in 24 dimensions may be more than a mathematical curiosity.

Subject of Research: Native vector-ternary neural network training via discrete projection onto the 24-dimensional Leech lattice

Article Title: Artificial neural networks as discrete complex systems: native vector-ternary training via the leech lattice (\Lambda _{24})

Article References: Lavicka, A. (2026). Artificial neural networks as discrete complex systems: native vector-ternary training via the leech lattice $$\Lambda _{24}$$. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-026-02517-8

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02517-8

Keywords: Leech lattice, neural networks, quantization, rate-distortion theory, sphere packing, vector quantization, discrete complex systems, information theory, model compression, transformers, cognitive architectures, stochastic gradient descent