The study, published in Frontiers in Complex Systems, began with an unusual act of digital archaeology. Dario Zarcone, Salvatore Miccichè of the University of Palermo, and David Sánchez of the Institute for Cross-Disciplinary Physics and Complex Systems in Palma de Mallorca reverse-engineered a set of multimedia CD-ROMs released in 2001, which stored the complete archive of the Italian daily La Repubblica in a proprietary binary format. From those discs they recovered roughly 600,000 articles, about 300 million words, covering every issue published between 1 January 1985 and 31 December 2000. Because the archive came from an official publisher rather than from scanned paper, the text is largely free of the optical character recognition errors that plague digitized historical newspapers, and every article carries a precise publication date.

The period captured on those discs is one of the most consequential stretches in Republican Italian history. It spans the final years of the First Republic, the political system built after the Second World War and dominated for decades by the same party coalitions, and the corruption scandals known as Tangentopoli that dissolved those parties and cleared the way for the reconfigured Second Republic. It also encompasses German reunification, the Gulf War, the introduction of the euro, and the Kosovo War. The researchers’ central question was whether such historical turning points leave detectable statistical traces in the language of the press, and whether those traces could be found without any prior labeling of dates or events.

The first layer of analysis was lexical. After tokenizing, lowercasing, removing punctuation and stopwords, and lemmatizing the Italian text with the spaCy library, the team computed time series of word salience using the term frequency-inverse document frequency measure, or TF-IDF. This weighting scheme assigns each word an importance that grows with how often it appears in a given period but shrinks with how widely it is used across the whole corpus, so that common function words are suppressed and genuinely newsworthy terms stand out. Aggregating all articles published in each month into a single document produced a salience trajectory for every word in a filtered vocabulary of about 140,000 lemmas.

Those trajectories revealed a striking statistical signature. Word usage in the corpus is bursty: rather than flowing at a steady rate, certain words erupt in sudden spikes followed by lulls. The team quantified this with the Fano factor, the ratio of the variance of a word’s counts to its mean, borrowed from the statistical physics of counting processes. A Fano factor greater than one signals over-dispersion, and the empirical distribution of Fano factors across the vocabulary turned out to be heavy-tailed, meaning a small subset of words, typically event-related terms such as those naming the Kosovo conflict, show extreme burst-like behavior while ordinary words remain stable. Remarkably, the heavy-tailed shape of this distribution persisted when the data were aggregated at daily, weekly, monthly, quarterly, and yearly resolutions, suggesting that burstiness is a fundamental property of lexical dynamics in media discourse rather than an artifact of any particular timescale.

To locate sustained shifts rather than momentary spikes, the researchers framed word salience as a changepoint detection problem. They applied the Pruned Exact Linear Time algorithm, which efficiently finds points in a time series where statistical properties change abruptly, and set its penalty parameter through a bootstrap procedure: the original series were shuffled to destroy temporal structure, and the penalty was chosen as the smallest value at which the shuffled null signal would yield fewer than one changepoint, ensuring that detected breaks were unlikely to be noise. When the algorithm was run on the 10,000 most salient words, the densest cluster of changepoints fell squarely in the mid-1990s, aligning with the transition from the First Republic to the Second Republic. The words driving the peaks in August 1993 and January 1994 were the names of the old parties, alongside tangentopoli itself, while the April 1994 peak, coinciding with the pivotal general election, was dominated by terms for the new governing coalition. Secondary peaks tracked the Gulf War and German reunification in 1990 and the economic reforms toward the euro in 1997. The longer-term trends told the same story: party acronyms of the First Republic steadily declined in salience while the names of Second Republic politicians rose.

The second layer of analysis moved from individual words to meaning. The team represented every article as a vector in a reduced semantic space using Latent Semantic Analysis, a classical technique that applies singular value decomposition to the TF-IDF matrix, yielding latent dimensions that capture broad semantic contrasts in the corpus. By averaging the article vectors of each month, they constructed a media discourse vector, the center of mass of all published content, and traced its path through the semantic space over sixteen years. The trajectory was far from a random walk. Its first latent component turned out to be a political axis, with First Republic parties anchoring one end and Second Republic figures the other, while the second component separated economic stability from war and conflict, capturing the Gulf War early in the period and the Kosovo War late in it.

Measuring the cosine similarity between discourse vectors of different months produced a matrix with a block-diagonal structure: stretches of months in which discourse remained semantically coherent, separated by sharp boundaries of low similarity. Those boundaries aligned with the early-1990s political transition and with major international conflicts, and the structure proved robust across aggregation levels, from daily to yearly, hinting at a hierarchical organization of discourse with nested regimes and transitions at multiple timescales. The researchers draw an analogy to the Epps effect in quantitative finance, where empirical correlations between assets artificially decay at high sampling frequencies because trading is asynchronous. Here, daily discourse vectors appear noisy and fragmented because topics evolve asynchronously, but widening the aggregation window smooths out that noise and lets the longer, stabler phases emerge.

Perhaps the most evocative finding concerns entropy. The team computed the spectral entropy of the covariance matrix of article vectors within each period, a measure of how widely semantic content is dispersed across the latent dimensions. Lower entropy means the discourse has contracted onto fewer themes, with articles becoming more similar in content and framing even when not identical. The entropy minima aligned precisely with Operation Desert Shield in August 1990, Operation Desert Storm in February 1991, and the Kosovo War in April 1999, providing quantitative support for a long-standing qualitative claim in media studies: exceptional events compress the news agenda. The effect was sharpest at monthly aggregation, a window short enough that an episode dominating the headlines for weeks appears as a coherent contraction but long enough to smooth out the daily rotation of leading stories.

The authors are careful about the limits of their approach. The corpus reflects the editorial priorities of a single newspaper, though La Repubblica’s wide national circulation makes it a reasonable proxy for dominant public discourse, and the analysis stops at 2000. The choice of Latent Semantic Analysis over modern contextual embeddings was deliberate: its linear components can be inspected directly by looking at their word loadings, which matters when the goal is to interpret macro-dynamics of historical change rather than maximize predictive performance. Crucially, the entire workflow is unsupervised, requiring no prior labeling of historical periods, which makes it applicable to archives where such labels are absent or contested. The researchers see the method not as a replacement for traditional historical analysis but as a complement to it, offering a transferable recipe: reconstruct a diachronic corpus, extract lexical time series, map the semantic trajectory, and read historical turning points as regime shifts in the space of public discourse.

Subject of Research: Detecting historical turning points in Italian media: a complex systems approach to a diachronic news corpus

Article Title: Detecting historical turning points in Italian media: a complex systems approach to a diachronic news corpus

Article References: Zarcone, D., Miccichè, S., & Sánchez, D. (2026). Detecting historical turning points in Italian media: a complex systems approach to a diachronic news corpus. Frontiers in Complex Systems, 4, Article 1868587. https://doi.org/10.3389/fcpxs.2026.1868587

Image Credits: AI Generated

DOI: 10.3389/fcpxs.2026.1868587

Keywords: Detecting, historical, turning, points, Italian, media, complex, systems, approach, diachronic, news, corpus