New TDV-optimization method brings numerical rigor to classic phytosociological tabulation

Vegetation scientists have long relied on a painstaking manual technique known as phytosociological tabulation to sort field samples into floristically distinct groups, but the process is notoriously subjective and laborious. A new study published in Vegetation Classification and Survey introduces a numerical index called the Total Differential Value, or TDV, together with an optimization procedure that can reproduce the core mechanics of this classic method automatically. The work, grounded in the operational concept of differential species as classically illustrated by Heinz Ellenberg and Dieter Mueller-Dombois, offers what its author describes as a novel way to quantify the differential power of species in a vegetation table.

The concept of the differential species dates back more than a century to the work of Josias Braun-Blanquet and forms the foundation of the tabulation technique. Differential species are those with restricted distributions among the field samples, or relevés, under comparison: they are present in one or more emerging groups of relevés but absent from at least one group. In manual tabulation, researchers manipulate matrices of relevés, seeking subsets that can be distinguished floristically by these species patterns. The process, sometimes called tablework or table sorting, is potentially highly subjective and particularly complex even for moderately sized tables, because the number of groups is expected to emerge naturally during the manipulation.

The new approach hinges on a distinction that earlier numerical methods have overlooked: the difference between two kinds of species absences. Absences from some relevés within a group, where the species is otherwise present, are generally attributed to stochastic processes, such as the random failure of a species to establish in a plot despite being in the local species pool. Absences from all relevés representing a group, by contrast, are differentiating absences, attributed to a complete absence from the species pool of that group, for example due to biogeographical causes or strong environmental filtering. By leveraging the distribution of these differentiating absences among groups, the method quantifies exclusiveness degrees and distinguishes differential from non-differential species.

Technically, the Differential Value, or DiffVal, of a species given a partition of relevés is calculated by averaging partial differential values across the groups in which the species occurs. Each partial value multiplies the relative frequency of the species’ presences within a group, its constancy, by the relative frequency of its differentiating absences outside that group. The ratio of presences reaches its maximum of one only when the species occurs in every relevé of the group, while the absence ratio reaches one only when the species is completely absent from all other groups. A factor accounting for the number of groups in which the species occurs keeps DiffVal between zero and one. The Total Differential Value is then simply the mean of the DiffVal outcomes for all species in the table, providing a global measure of how well the groups can be distinguished by differential species.

Because only differential species contribute to TDV, the index reflects the strength of the differential species patterns in a classified table. Species that occur in all vegetation groups yield a DiffVal of zero and are effectively ignored, which means the optimization focuses solely on the subset of the matrix where differential species may occur. This property is central to the method’s ability to detect reticulate patterns, misaligned subsets of species that produce multiple sound groupings of relevés. Since multiple runs of the optimization starting from random partitions have a good chance of finding such patterns among local optima, the method can reveal different valid classifications within the same data set.

Optimizing TDV is a formidable combinatorial challenge. The number of possible partitions of a set with n elements corresponds to the nth Bell number, which grows astonishingly fast: just 10 relevés can be partitioned in 115,975 different ways, and 64 relevés into three groups in roughly 7.6 times ten to the eighteenth different solutions. Complete enumeration is therefore impracticable, and the author employs local search-based metaheuristics, including variations of the hill climbing algorithm, to maximize the index. An R package called diffval is now available, implementing DiffVal and TDV calculation as well as TDV-optimization through functions for hill climbing, simulated annealing, and, for the two-group case, mixed-integer linear programming. The author is also working on a mathematical proof showing that finding a partition that maximizes TDV is NP-hard for two or more groups.

The study demonstrates the method on both an artificial data set and a well-known real-world example. The artificial data set contained 107 species and 64 relevés with two distinct reticulated patterns of differential species distributed across three groups each. When compared against ten clustering and partitioning methods using seven statistics, TDV-optimization was the only method capable of converging to local optima that identified both artificial patterns. However, the highest-TDV partition isolated two single-relevé groups, and enforcing a minimum group size of four relevés placed the partitions capturing the artificial patterns in the first and second positions of the returned solutions, highlighting how the parameter can yield more balanced group sizes.

The real-world test used the classical Arrhenatheretum data set, which Ellenberg used in 1956 to meticulously illustrate the tabulation technique and which has since become a standard benchmark. Ellenberg highlighted 26 differential species in the data set, and TDV-optimization found a partition differing from Ellenberg’s three-group classification by only one discrepant relevé. This is a relevant result, as similar attempts found in the literature have consistently shown poorer performances, with earlier methods producing between two and eight discrepant relevés. Notably, the result was achieved using presence-absence data alone, suggesting that the long-held idea that such data were insufficient to classify this data set appears flawed, since Braun-Blanquet vegetation units are based on the presence of diagnostically relevant species while cover is used for physiognomic characterization.

The author argues that TDV-optimization is conceptually distinct from previously published clustering approaches on four counts. It does not incorporate a hierarchical structure, it does not rely on geometric features such as distances to centroids or medoids, it does not treat all species as equally informative, and it ensures the exclusiveness of informative species to one or more emerging groups. Among biclustering methods, the closest known approach is that of Češka and Roemer, whose procedure could ensure species exclusivity with a particular parameter setting but was not explored by the authors and does not return a partition of relevés. The traditional tabulation technique seeks exclusive-row and -column biclusters, and the new method aligns with a relaxed exclusive-rows, non-exhaustive-in-the-rows structure that accommodates dominant and companion species spanning all groups.

The author cautions that TDV-optimization is not an instantaneous method yielding a single answer but a tool for exploring vegetation data in search of meaningful classifications. The highest-TDV partitions may isolate outlier or extreme relevés, which could arise from stochastic processes, data errors, or genuinely distinct under-sampled vegetation types, and decisions about such cases must always be guided by expert knowledge not embedded in the data table. The method is expected to be particularly useful in floristically rich regions with high geographical turnover or strong environmental filtering, such as the Mediterranean region, while its utility may be limited where floristic variation is reflected mainly in differences in species cover-abundance. Applications could also extend to other scientific fields focused on partitioning binary data, such as defining subpopulations through exclusive gene blocks.

Subject of Research: A novel numerical index and optimization method for phytosociological vegetation tabulation based on differential species

Article Title:  TDV -optimization: A novel numerical method for phytosociological tabulation

Article References: Monteiro-Henriques, T. (2025). TDV-optimization: A novel numerical method for phytosociological tabulation. Vegetation Classification and Survey, 6, 99-127. https://doi.org/10.3897/vcs.140466

Image Credits: AI Generated

DOI: 10.3897/VCS.140466

Keywords: TDV-optimization, differential species, phytosociological tabulation, vegetation classification, Total Differential Value, biclustering, relevé partitioning, Braun-Blanquet, R package diffval, Arrhenatheretum data set, reticulate patterns, vegetation science