Fine-tuned DeepSeek-R1 reasoning model restores missing characters in Dunhuang ancient Tibetan manuscripts

Deep in the caves of Mogao, on the edge of China’s Gobi Desert, thousands of ancient Tibetan manuscripts have waited more than a millennium for readers who could understand them. Many of these Dunhuang texts, written on fragile paper between the eighth and eleventh centuries, survive only in fragments. Water, insects, and the slow chemistry of decay have eaten away syllables, lines, and sometimes entire pages. For generations, the only way to reconstruct what was lost was the patient, expert judgment of philologists who could infer a missing character from context, grammar, and a lifetime of familiarity with the language. Now a team at Xizang University in Lhasa has shown that a fine-tuned large language model can do much of that work automatically, and do it well.

In a study published in npj Heritage Science, Liyuan Zhou, Jingyi Li, Hongmei Gao, and Dingguo Gao describe how they adapted DeepSeek-R1, a large-scale reasoning language model, into a specialized system they call DeepSeek-R1-Tibet. The model was trained to infer missing characters in ancient Tibetan manuscripts by reasoning over semantic context rather than simply guessing from surface patterns. Their experiments show that the fine-tuned reasoning model outperformed established multilingual baselines, including CINO, mBERT, and TisBERT, across all tested conditions. On the Hits@3 metric in the three-syllable condition, DeepSeek-R1-Tibet reached 0.667 compared with 0.456 for mBERT, a striking gap that reflects the growing advantage of generative reasoning when broader context must be weighed.

The stakes are more than academic. Tibetan ancient books carry rich historical, philosophical, and medical knowledge, much of it recorded nowhere else. The Dunhuang corpus, recovered from a sealed cave repository in the early twentieth century, includes some of the oldest surviving Tibetan documents in existence. But poor preservation has left many manuscripts fragmented and illegible, and traditional manual repair is inefficient and subjectively biased, as the authors note. Two experts may propose different reconstructions for the same damaged passage, and neither can easily justify the choice beyond intuition. A computational method that ranks plausible candidates offers a way to make that judgment faster, more consistent, and more transparent.

The technical core of the work lies in how the researchers bridged the distance between modern Tibetan and the ancient language of the Dunhuang manuscripts. DeepSeek-R1 was not designed for this task. Its pretraining data, like that of most large language models, is dominated by English, Chinese, and other high-resource languages, and its grasp of classical Tibetan vocabulary and orthography is thin. The team therefore integrated large volumes of modern Tibetan text with ancient Tibetan texts from Dunhuang, fine-tuning the model so that it became sensitive to the semantic context specific to ancient Tibetan. The result is DeepSeek-R1-Tibet, a reasoning model suited to the linguistic world of the manuscripts rather than the modern web.

Fine-tuning a reasoning model of this scale is a demanding engineering exercise. DeepSeek-R1 belongs to a class of models trained to produce explicit chains of reasoning before answering, a design that lets it decompose a problem into steps rather than emit a single guess. In the missing-character task, that capacity matters. A damaged manuscript rarely offers a clean local context; the meaning of a lost syllable may depend on grammatical particles several lines away, on formulaic phrases common in Buddhist liturgy, or on the structure of a medical treatise. A model that can reason over that broader context, weighing alternatives explicitly, is structurally better suited to the problem than one that classifies from a fixed window of neighboring tokens.

The baselines the authors chose make the comparison meaningful. CINO, mBERT, and TisBERT are pretrained multilingual models that have served as standard tools for Tibetan and related low-resource languages. They operate by encoding context into representations and predicting the missing item from those representations, an approach that works well when the relevant evidence is local. The experiments demonstrate that the fine-tuned DeepSeek-R1-Tibet achieves the largest performance gains across all conditions, and that the advantage widens as the task demands reasoning over more distant evidence. In the three-syllable condition, where the model must identify the correct character among candidates given a wider gap, the generative reasoning approach proves increasingly advantageous, as the authors put it.

The Hits@3 metric deserves a word of explanation, because it captures what reconstruction actually requires. Rather than demanding that the model produce exactly the right character on the first try, Hits@3 asks whether the correct answer appears among the model’s top three candidates. That framing mirrors real manuscript repair: an expert proposes a short list of plausible reconstructions, and the philologist selects among them. A score of 0.667 means that in roughly two out of three cases, the true character was among the model’s three best guesses, against fewer than half for mBERT. For a language as low-resource as ancient Tibetan, that is a substantial leap in practical usefulness.

Why does generative reasoning win in this setting? The authors’ results suggest that as the amount of context available for inference grows, the ability to deliberate over that context becomes the decisive factor. Classification models compress their evidence into a single forward pass, with limited capacity to revisit assumptions. A reasoning model can, in effect, read the passage, form a hypothesis, test it against grammatical and semantic constraints, and revise. For ancient Tibetan, where orthography changed over centuries and scribes varied in style, that iterative deliberation is exactly the kind of process human experts perform, now reproduced at machine speed.

The work was carried out at the College of Information Science and Technology at Xizang University and the Tibetan Information Technology Innovation Personnel Training Demonstration Base in Lhasa, and it was funded by the National Natural Science Foundation of China under a grant dedicated to research on text recognition methods for Dunhuang Tibetan books, along with support from the China High-level Talent Special Support Program and regional education research projects. That institutional grounding matters: the study reflects a deliberate national investment in building Tibetan-language artificial intelligence capacity, from training personnel to constructing the datasets that make fine-tuning possible in the first place.

The authors describe the work as opening new avenues for the digital preservation and intelligent organization of Tibetan ancient books. The implications reach beyond one language and one corpus. Dunhuang manuscripts exist in Chinese, Sanskrit, Sogdian, and other scripts, and fragmented texts are a global problem, from Dead Sea Scrolls to medieval European charters. The lesson of DeepSeek-R1-Tibet is that reasoning-capable large language models, carefully fine-tuned on domain data, can outperform conventional classification approaches precisely where context is ambiguous and evidence is scattered. As such models grow more capable and more affordable to adapt, the slow, subjective work of textual reconstruction may become a collaboration between human scholarship and machine deliberation, with the lost words of a millennium ago surfacing once more into readable light.

Subject of Research: A DeepSeek-R1 fine-tuning approach for missing character inference in Dunhuang ancient Tibetan texts

Article Title: A DeepSeek-R1 fine-tuning approach for missing character inference in Dunhuang ancient Tibetan texts

Article References: A DeepSeek-R1 fine-tuning approach for missing character inference in Dunhuang ancient Tibetan texts. (n.d.). https://doi.org/10.1038/s40494-026-03045-x

Image Credits: AI Generated

DOI: 10.1038/s40494-026-03045-x

Keywords: DeepSeek-R1, ancient Tibetan, Dunhuang manuscripts, large language models, fine-tuning, digital heritage, missing character inference, npj Heritage Science, low-resource languages, text reconstruction, approach, missing