The motivation behind the project is a persistent gap in the artificial intelligence landscape. Although open-source large language models have advanced rapidly in recent years, most of the research community’s effort has poured into general-purpose models trained predominantly on English data. That imbalance creates real problems for anyone studying or deploying AI in Chinese education. Chinese is a language with distinctive tokenization challenges, rich morphological structure, and a vast educational literature that English-centric models simply do not capture well. Educational applications add another layer of specificity: the vocabulary of pedagogy, curriculum standards, exam questions, and subject-specific reasoning in Chinese differs substantially from the web text that dominates most training corpora.

Rather than fine-tuning an existing model, the team took the more demanding route of training CELLM from the ground up, a decision that gave them full control over the data pipeline and the architecture. The process unfolded in two stages. The first was pre-training, in which the model learned the statistical structure of language from an open-source dataset drawn from the Chinese education domain. During this phase, the model absorbed the patterns of educational Chinese text, building the foundational knowledge that later stages would refine. The second stage was instruction fine-tuning, which teaches a base model to follow directions, answer questions, and behave like an assistant rather than a text predictor.

For that second stage, the researchers faced a familiar obstacle: high-quality Chinese instruction data for education was scarce. Their response was to build their own. They constructed a Chinese instruction dataset comprising over 258,000 data entries and, in the spirit of the entire project, released it openly. Instruction datasets of this kind typically pair prompts with high-quality responses, spanning formats such as question answering, explanations, and multi-turn dialogue. By curating the data themselves, the team could steer the model toward the kinds of interactions that matter in educational contexts, from explaining a mathematics problem to discussing language teaching strategies.

The architecture underlying CELLM draws on the core technologies that have defined modern open-source language models. The team reviewed and synthesized the design choices of representative open-source systems before settling on their own configuration. Among the techniques reflected in the model’s lineage are rotary position embeddings, an approach that encodes word order by rotating vector representations and has become a staple of contemporary transformer designs. The model also builds on advances in attention mechanisms, including grouped query attention, which reduces computational cost by sharing key-value projections across multiple query heads, a strategy popularized by efficient inference research.

Other components of the design reflect a decade of accumulated transformer engineering. The feed-forward layers employ variants of gated linear units, activation functions shown to improve transformer performance over standard alternatives. Training at scale was supported by DeepSpeed, the distributed training framework developed to make models with hundreds of billions of parameters feasible on real hardware. The team also drew on research into scaling laws and overtraining, which examines how model performance grows with parameters and data, informing how a relatively small model can be trained to punch above its weight class.

The choice of a 1.5-billion-parameter size is itself significant. In an era when frontier models boast hundreds of billions of parameters, a compact model might seem modest. But smaller models are dramatically cheaper to train, run, and deploy, which matters enormously for educational research groups, schools, and developers working with limited computing budgets. A 1.5-billion-parameter model can run on consumer-grade hardware, making it practical for experimentation and classroom applications alike. The trade-off is capability, and the evaluation results provide a transparent accounting of where the model stands.

That transparency is one of the study’s most valuable contributions. The researchers benchmarked CELLM across multiple evaluation datasets, including suites designed to measure massive multitask language understanding in Chinese, such as C-Eval and CMMLU, which test knowledge across disciplines and difficulty levels, alongside mathematical problem-solving benchmarks. The published results establish a reference baseline for future research, giving subsequent teams a clear point of comparison. In a field where claims often outpace evidence, a fully documented baseline for an education-specific Chinese model fills a genuine need.

The open-source release strategy amplifies the work’s potential impact. Everything generated during the study, including the models, the data, and the code, has been made publicly available. This stands in deliberate contrast to the closed ecosystems of the largest commercial AI systems, whose training data and internal workings remain opaque. Open release allows other researchers to scrutinize the model, reproduce the results, extend the training, or adapt the instruction dataset to adjacent domains. It also enables studies of the models themselves, including investigations of bias, generalization, and how a model’s capabilities trace back to its pre-training data, questions that are difficult or impossible to answer with proprietary systems.

For the field of Chinese education research, CELLM represents both a tool and a template. As a tool, it offers researchers a capable, domain-tuned language model they can study, fine-tune, and deploy without licensing barriers. As a template, it demonstrates that building a specialized model from scratch, with a purpose-built instruction dataset and rigorous public evaluation, is achievable by a small academic team. As open-source language models continue to proliferate across languages and domains, the Shanghai team’s work signals a shift toward AI research that treats education not as an afterthought of general-purpose systems, but as a domain deserving models, data, and benchmarks of its own.

Subject of Research: An open-source large language model trained from scratch for Chinese education research

Article Title: An Open-Source Large Language Model for Chinese Education Research

Article References: Liu, W., Hao, H., & Zhou, A. (2025). An Open-Source Large Language Model for Chinese Education Research. Frontiers of Digital Education, 2(2), Article 23. https://doi.org/10.1007/s44366-025-0060-0

Image Credits: AI Generated

DOI: 10.1007/s44366-025-0060-0

Keywords: large language models, open source, Chinese education, CELLM, instruction fine-tuning, pre-training, transformer architecture, rotary position embeddings, DeepSpeed, C-Eval, CMMLU, educational AI

The appeal of 3D data is easy to grasp. A photograph flattens the world onto a grid of pixels, discarding the very depth information that lets humans judge distance, volume and shape. Three-dimensional data, by contrast, carries precise spatial measurements, allowing an algorithm to know not just that a pedestrian is present but exactly how far away they stand and how much space they occupy. The survey identifies five dominant ways of representing this information: point clouds, voxel grids, depth and normal maps, polygonal meshes and neural fields. Each comes with trade-offs in computational cost, storage demands and expressive power, and choosing the right representation is often the first critical decision in any 3D pipeline.

Point clouds deserve particular attention because they dominate real-world sensing. Produced by LiDAR scanners and depth cameras, a point cloud is simply a set of points floating in space, each carrying a 3D coordinate and sometimes color or intensity. The trouble is that these points are irregular and unordered, which makes them fundamentally incompatible with the convolutional neural networks that revolutionized 2D image analysis. The field’s answer is sparse convolution, a clever trick that performs computation only on non-empty regions of a voxelized space. Because most of a scanned scene is empty air, sparse convolution slashes the computational burden dramatically. Its cousin, sub-manifold sparse convolution, goes further by restricting computation to kernel centers that already contain data, preventing the dilation that would otherwise destroy the input’s sparsity, though the dilation property itself can be useful for giving models contextual awareness of their surroundings.

With these foundations in place, the survey turns to the heart of modern 3D research: pre-training. The idea borrows directly from the playbook that made 2D vision so successful. Models pre-trained on ImageNet’s fourteen million labeled images learned general-purpose features that transferred effortlessly to new tasks. But 3D data lacks an equivalent of ImageNet, largely because annotating millions of 3D scans is prohibitively expensive. The solution has been self-supervised learning, in which models generate their own training signals from raw data. The survey organizes these efforts into three families: contrastive methods, masked autoencoder approaches and rendering-based techniques.

Contrastive learning teaches a network to pull similar data points together in an embedding space while pushing dissimilar ones apart, typically through an InfoNCE loss. In the 3D world this takes several forms. PointContrast, a landmark 3D-to-3D method, encourages networks to learn representations that remain stable under different viewpoints and noise, directly boosting downstream detection and segmentation. Other approaches reach across modalities: CrossPoint and SLidR bridge point clouds and rendered images, transferring knowledge from 2D backbones trained on vast image collections into 3D networks. Perhaps most strikingly, methods like PointCLIP and its successor PointCLIPV2 connect 3D perception to language, projecting point clouds into multi-view images and aligning them with the text embeddings of CLIP-style models. This enables zero-shot recognition, where a model classifies objects it has never been trained on simply by matching visual features to textual descriptions.

Masked autoencoder methods take a different route, inspired by the masked image modeling that powered recent breakthroughs in 2D vision. The core idea is to corrupt or hide parts of the input and force the model to reconstruct them, thereby learning the underlying structure of 3D shapes. VoxelMAE converts unstructured points into voxel grids and predicts the occupancy of masked voxels. PointMAE, borrowing a page from BERT, tokenizes point clouds using farthest point sampling and K-nearest-neighbor grouping, then trains with an L2 loss on masked token features; its descendant PointM2AE adds pyramid architectures that capture both fine geometric detail and high-level semantics, achieving state-of-the-art linear classification on the ModelNet40 benchmark. GD-MAE extends the paradigm to convolutional backbones favored in autonomous driving, with a masking strategy designed to prevent knowledge leakage during downsampling.

The third family is the most visually intuitive: rendering-based pre-training. Differentiable rendering makes the process of drawing an image from a 3D model mathematically transparent, so gradients can flow backward from the rendered image to the 3D representation itself. The Ponder framework describes 3D surfaces within a volume and optimizes them against 2D image projections using a NeRF-like structure, while PonderV2 scales the idea to outdoor scenes and can pre-train both 2D and 3D backbones. According to the survey, this rendering-based approach outperforms both contrastive and masked-autoencoder methods, suggesting that grounding 3D learning in the physics of image formation may be the field’s most promising direction.

These pre-trained representations then feed a rich ecosystem of downstream tasks. Classification, the field’s founding problem, began with PointNet’s pioneering use of shared multi-layer perceptrons on raw points and matured through architectures like DGCNN’s EdgeConv, which builds local graphs to learn edge embeddings. Segmentation assigns a category to every point in a scan and has spawned a fierce architectural competition: MinkowskiUNet applies U-Net-style designs to voxel data, Cylinder3D replaces cubic partitions with cylindrical ones to handle LiDAR’s varying point density, SphereFormer exploits the transformer’s global receptive field, and RangeFormer brings powerful 2D-style transformers to range-view representations. Detection, crucial for autonomous vehicles, spans LiDAR-based methods like PointRCNN and PVRCNN, camera-based systems like BEVFormer that follow Tesla’s vision-centric pipeline, and fusion approaches like PointPainting and LoGoNet that marry the precise geometry of LiDAR with the rich texture of images. Tracking, matching and registration round out the toolkit, with methods like GeoTransformer and robust variants of the classic ICP algorithm enabling machines to align separate 3D scans into coherent wholes.

Progress on all these fronts is measured against a demanding set of benchmarks. ModelNet40 and ShapeNet test synthetic shape understanding, ScanNetV2 and S3DIS probe indoor scene parsing, while KITTI, nuScenes, Waymo and ONCE push algorithms to their limits in real-world driving scenarios containing millions of annotated frames. Evaluation relies on metrics such as mean average precision, computed from intersection-over-union thresholds between predicted and ground-truth boxes, and mean intersection-over-union for segmentation, with registration tasks judged by rotation error, translation error and registration recall. The survey’s careful cataloging of these standards provides newcomers with an immediate map of how the field keeps score.

Looking forward, the authors identify several frontiers that could define the next decade. Large language models, brimming with world knowledge distilled from vast text corpora, are only beginning to be connected to 3D perception through systems like Uni3D-LLM. Knowledge transfer from the data-rich 2D and language domains into the data-starved 3D world remains a wide-open opportunity. Synthetic data could relieve the crushing cost of manual annotation, though bridging the gap between simulated and real scenes will demand advances in domain adaptation and neural rendering. Most ambitiously, the field awaits its own foundation model: while 2D vision has been reshaped by systems like Segment Anything, no equivalent exists for 3D, largely because large-scale 3D benchmarks are still lacking. Building one, the survey argues, could dramatically cut the design and deployment costs of countless 3D applications and pave the way toward a genuine era of 3D artificial general intelligence. For a field that measures the world in three dimensions, the trajectory is unmistakably upward.

Subject of Research: Self-supervised pre-training methods and downstream tasks for 3D point cloud perception

Article Title: Advances in 3D pre-training and downstream tasks: a survey

Article References: Hou, Y., Huang, X., Tang, S., He, T., & Ouyang, W. (2024). Advances in 3D pre-training and downstream tasks: a survey. Vicinagearth, 1(1), Article 6. https://doi.org/10.1007/s44336-024-00007-4

Image Credits: AI Generated

DOI: 10.1007/s44336-024-00007-4

Keywords: 3D perception, point clouds, self-supervised learning, pre-training, masked autoencoders, contrastive learning, sparse convolution, LiDAR, autonomous driving, neural rendering, foundation models, benchmarks

The trouble is that most state-of-the-art prediction methods come with a hidden price tag. To squeeze out ever-better accuracy, recent models have grown dependent on a sprawling ecosystem of external data: pre-trained protein language models trained on billions of evolutionary sequences, three-dimensional structures fetched from repositories such as the Protein Data Bank, gene ontology annotations, tissue expression profiles, and more. Each additional data source can help, but it also creates fragility. If a laboratory works with a poorly studied organism whose proteins lack rich annotations, or if access to a heavyweight pre-trained model is impractical, the entire prediction pipeline can stumble. A second, less obvious problem concerns scale. Protein-protein interaction networks are graphs, and the relationships within them exist at multiple levels simultaneously, from the local neighborhood of a single protein to long-range dependencies that stretch across the entire network. Capturing those multi-scale patterns typically demands computationally expensive graph transformers, whose attention mechanisms scale quadratically with network size and quickly become prohibitive.

A research team led by Yan Kang and Hu Yuan of Yunnan University, together with Xinchao Wang of Southeast University and Shengzhe Yao of Yunnan University, set out to tackle both limitations head-on. Writing in the journal Applied Intelligence, they introduce NMPPI, a lightweight framework that predicts protein-protein interactions using only the intrinsic information already present in the PPI network itself, namely protein sequences and the topology of the interaction graph. No external databases, no auxiliary annotations, no giant pre-trained language models. The work, published as volume 56, article 459 of Applied Intelligence, demonstrates that careful architectural design can match or beat far heavier systems while consuming a fraction of the training time and memory. The datasets underpinning the study derive from the widely used STRING database of protein-protein associations, and the authors state that the code will be publicly released on GitHub upon publication.

The first pillar of NMPPI is an unsupervised feature extraction strategy built on Node2vec, a random-walk-based algorithm originally developed for learning representations of nodes in arbitrary networks. Rather than treating the PPI graph as a static collection of labeled points, Node2vec simulates biased random walks that wander from protein to protein along interaction edges, producing sequences of node visits that behave like sentences in a language. A skip-gram style embedding procedure then learns vector representations in which proteins that occupy similar structural roles, or that frequently co-occur within short walking distance, end up close together in the embedding space. The elegance of this approach is that it extracts meaningful structural information purely from the order in which nodes appear in the generated sequences, requiring no supervision and no external knowledge. In effect, the network topology itself becomes the training corpus, and the resulting embeddings encode a compact summary of each protein’s relational position within the interactome.

Sequence information complements these structural embeddings. Each protein’s amino acid sequence carries the biochemical fingerprint that determines which binding partners it can physically engage, and NMPPI incorporates sequential features alongside the Node2vec-derived structural vectors to form a unified representation of every node. This dual-channel design, pairing what a protein is with where it sits in the network, allows the model to reason about interactions from two independent angles. It also directly answers the question the authors pose about feature extraction when external data is unavailable: by relying exclusively on sequences and graph structure, NMPPI remains fully functional even for organisms or datasets where nothing else can be obtained. That property matters enormously for labs studying neglected pathogens, rare species, or newly sequenced genomes, where the ecosystem of annotations that powers mainstream predictors simply does not exist yet.

The second pillar, and the technical heart of the paper, is a supervised learning model that fuses a Graph Isomorphism Network with a novel Graph-Mamba module. Graph Isomorphism Networks, often abbreviated GIN, are among the most expressive message-passing architectures available for graph learning, capable in principle of distinguishing graph structures that fool simpler convolutional variants. They excel at aggregating local neighborhood information, building up each node’s representation layer by layer. But message passing alone has a well-known blind spot: information travels only one hop per layer, so capturing dependencies between proteins separated by many edges requires deep stacks of layers, which invite over-smoothing and vanishing gradients. Long-range dependencies, precisely the multi-scale relationships the team wanted to capture, have historically been expensive to model in graphs.

The Graph-Mamba module addresses that bottleneck with a state space model, a class of sequence model that has recently surged in popularity as an efficient alternative to transformers. Where transformer attention computes pairwise interactions between every pair of tokens at quadratic cost, state space models compress historical information into a fixed-size hidden state that is updated recursively, yielding linear computational complexity in sequence length. Applied to the PPI network, Graph-Mamba treats node representations as a sequence and selectively filters which information to retain or discard as it sweeps through, capturing inter-node dependencies across the entire graph without the combinatorial blow-up of attention. The result is a model that perceives both the fine-grained local structure handled by the GIN and the sweeping global context handled by the Mamba-style module, achieving genuine multi-scale relational learning at a computational budget that scales gently with network size. The authors report that this combination lets NMPPI extract long-range dependencies that purely local architectures miss, while sidestepping the memory wall that plagues graph transformers.

The empirical case for the framework rests on extensive experiments across two real PPI datasets. According to the paper, NMPPI outperforms state-of-the-art methods on both benchmarks, and the advantages become even more striking when the model is pushed outside its comfort zone. In cross-dataset scenarios, where a model trained on one interaction network is evaluated on another, NMPPI showed superior generalization performance, suggesting that its reliance on intrinsic sequence and structural features produces representations that transfer rather than overfit to dataset-specific quirks. The efficiency numbers are equally eye-catching: NMPPI required only 27 percent of the training time and 36 percent of the parameter size compared with advanced competing methods. For research groups without access to large GPU clusters, those reductions can mean the difference between running a meaningful experiment overnight versus not running it at all, and they hint at a broader trend in machine learning for biology where leaner, purpose-built architectures are beginning to challenge the assumption that bigger is always better.

Beyond the headline results, the study sits at a fascinating intersection of several research currents. Protein language models such as ESM-2 and ProGen have dominated recent headlines for their ability to predict mutation effects and infer atomic-level structure at evolutionary scale, and graph neural networks have become the default tool for interaction prediction, with prior efforts like hierarchical graph learning frameworks and attention-based hybrids pushing accuracy steadily upward. Meanwhile, the Mamba family of state space models has been spreading rapidly from language modeling into vision and, increasingly, into biology, with early efforts such as Protein-Mamba exploring biological applications. NMPPI’s contribution is to show that these ingredients can be combined in a way that deliberately minimizes external dependencies, a design philosophy that could prove especially valuable as the field grapples with the uneven distribution of high-quality biological data across the tree of life.

The practical implications stretch from basic science to drug discovery. More complete and accurate PPI maps help researchers identify previously unknown components of signaling pathways, prioritize candidate drug targets whose disruption would selectively cripple disease-relevant protein complexes, and interpret the growing flood of genomic data pouring out of sequencing pipelines. A predictor that works without external annotations lowers the barrier to entry for scientists working on understudied organisms, from emerging pathogens to agriculturally important crops, and its modest computational footprint makes large-scale screening feasible on ordinary hardware. The authors acknowledge no conflicts of interest, and the work was supported in part by the National Natural Science Foundation of China and a Yunnan Provincial Major Science and Technology Project. With the code slated for public release, the broader community will soon be able to test whether this lean, topology-first approach becomes a new baseline for interaction prediction, one that proves the most important clues about how proteins connect were hiding in the network all along.

Subject of Research: Deep learning prediction of protein-protein interactions using graph neural networks and state space models

Article Title: NMPPI: multi-scale relationship structure with effective pre-training for protein-protein interactions

Article References: Kang, Y., Yuan, H., Wang, X., & Yao, S. (2026). NMPPI: multi-scale relationship structure with effective pre-training for protein-protein interactions. Applied Intelligence, 56(15), Article 459. https://doi.org/10.1007/s10489-026-07339-2

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07339-2

Keywords: protein-protein interactions, graph neural networks, Graph Isomorphism Network, Graph-Mamba, state space models, Node2vec, pre-training, computational biology, STRING database, multi-scale learning, protein sequences, machine learning