The motivation behind the project is a persistent gap in the artificial intelligence landscape. Although open-source large language models have advanced rapidly in recent years, most of the research community’s effort has poured into general-purpose models trained predominantly on English data. That imbalance creates real problems for anyone studying or deploying AI in Chinese education. Chinese is a language with distinctive tokenization challenges, rich morphological structure, and a vast educational literature that English-centric models simply do not capture well. Educational applications add another layer of specificity: the vocabulary of pedagogy, curriculum standards, exam questions, and subject-specific reasoning in Chinese differs substantially from the web text that dominates most training corpora.

Rather than fine-tuning an existing model, the team took the more demanding route of training CELLM from the ground up, a decision that gave them full control over the data pipeline and the architecture. The process unfolded in two stages. The first was pre-training, in which the model learned the statistical structure of language from an open-source dataset drawn from the Chinese education domain. During this phase, the model absorbed the patterns of educational Chinese text, building the foundational knowledge that later stages would refine. The second stage was instruction fine-tuning, which teaches a base model to follow directions, answer questions, and behave like an assistant rather than a text predictor.

For that second stage, the researchers faced a familiar obstacle: high-quality Chinese instruction data for education was scarce. Their response was to build their own. They constructed a Chinese instruction dataset comprising over 258,000 data entries and, in the spirit of the entire project, released it openly. Instruction datasets of this kind typically pair prompts with high-quality responses, spanning formats such as question answering, explanations, and multi-turn dialogue. By curating the data themselves, the team could steer the model toward the kinds of interactions that matter in educational contexts, from explaining a mathematics problem to discussing language teaching strategies.

The architecture underlying CELLM draws on the core technologies that have defined modern open-source language models. The team reviewed and synthesized the design choices of representative open-source systems before settling on their own configuration. Among the techniques reflected in the model’s lineage are rotary position embeddings, an approach that encodes word order by rotating vector representations and has become a staple of contemporary transformer designs. The model also builds on advances in attention mechanisms, including grouped query attention, which reduces computational cost by sharing key-value projections across multiple query heads, a strategy popularized by efficient inference research.

Other components of the design reflect a decade of accumulated transformer engineering. The feed-forward layers employ variants of gated linear units, activation functions shown to improve transformer performance over standard alternatives. Training at scale was supported by DeepSpeed, the distributed training framework developed to make models with hundreds of billions of parameters feasible on real hardware. The team also drew on research into scaling laws and overtraining, which examines how model performance grows with parameters and data, informing how a relatively small model can be trained to punch above its weight class.

The choice of a 1.5-billion-parameter size is itself significant. In an era when frontier models boast hundreds of billions of parameters, a compact model might seem modest. But smaller models are dramatically cheaper to train, run, and deploy, which matters enormously for educational research groups, schools, and developers working with limited computing budgets. A 1.5-billion-parameter model can run on consumer-grade hardware, making it practical for experimentation and classroom applications alike. The trade-off is capability, and the evaluation results provide a transparent accounting of where the model stands.

That transparency is one of the study’s most valuable contributions. The researchers benchmarked CELLM across multiple evaluation datasets, including suites designed to measure massive multitask language understanding in Chinese, such as C-Eval and CMMLU, which test knowledge across disciplines and difficulty levels, alongside mathematical problem-solving benchmarks. The published results establish a reference baseline for future research, giving subsequent teams a clear point of comparison. In a field where claims often outpace evidence, a fully documented baseline for an education-specific Chinese model fills a genuine need.

The open-source release strategy amplifies the work’s potential impact. Everything generated during the study, including the models, the data, and the code, has been made publicly available. This stands in deliberate contrast to the closed ecosystems of the largest commercial AI systems, whose training data and internal workings remain opaque. Open release allows other researchers to scrutinize the model, reproduce the results, extend the training, or adapt the instruction dataset to adjacent domains. It also enables studies of the models themselves, including investigations of bias, generalization, and how a model’s capabilities trace back to its pre-training data, questions that are difficult or impossible to answer with proprietary systems.

For the field of Chinese education research, CELLM represents both a tool and a template. As a tool, it offers researchers a capable, domain-tuned language model they can study, fine-tune, and deploy without licensing barriers. As a template, it demonstrates that building a specialized model from scratch, with a purpose-built instruction dataset and rigorous public evaluation, is achievable by a small academic team. As open-source language models continue to proliferate across languages and domains, the Shanghai team’s work signals a shift toward AI research that treats education not as an afterthought of general-purpose systems, but as a domain deserving models, data, and benchmarks of its own.

Subject of Research: An open-source large language model trained from scratch for Chinese education research

Article Title: An Open-Source Large Language Model for Chinese Education Research

Article References: Liu, W., Hao, H., & Zhou, A. (2025). An Open-Source Large Language Model for Chinese Education Research. Frontiers of Digital Education, 2(2), Article 23. https://doi.org/10.1007/s44366-025-0060-0

Image Credits: AI Generated

DOI: 10.1007/s44366-025-0060-0

Keywords: large language models, open source, Chinese education, CELLM, instruction fine-tuning, pre-training, transformer architecture, rotary position embeddings, DeepSpeed, C-Eval, CMMLU, educational AI