Adapting language models for fuel property prediction

Researchers at Tsinghua University have shown that general-purpose large language models, the same class of artificial intelligence systems behind modern chatbots, can be retrained to predict the physicochemical properties of fuels with striking accuracy. The team, based in the Department of Energy and Power Engineering and the Center for Combustion Energy, developed a framework called FuelProp-LM that predicts multiple fuel properties simultaneously from nothing more than a molecule’s SMILES string, the compact text notation chemists use to describe molecular structure. The study, published in ENGINEERING Energy, demonstrates how instruction tuning and in-context learning can transform open-source language models into practical quantitative structure-property relationship, or QSPR, predictors for fuel design.

QSPR modeling has long been a cornerstone of property-oriented fuel design. The approach establishes quantitative relationships between molecular structures and physicochemical properties, allowing researchers to screen candidate fuels for clean energy transition and high-efficiency engine design before synthesis. Yet conventional QSPR workflows carry well-known burdens. Models generally rely on manually engineered molecular descriptors, are usually developed to predict a single property, and often cannot readily incorporate newly available data. Adding fresh measurements may require descriptor reselection, model updates, or full retraining, limiting scalability and long-term applicability. FuelProp-LM was designed to bypass these constraints by unifying multi-property prediction within a single adaptable model.

To construct the framework, the researchers fine-tuned four compact open-source language models, each with fewer than 10 billion parameters: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Phi-3.5-mini-instruct, and DeepSeek-R1-Distill-Qwen-7B. Training drew on 100,000 data entries obtained from the PubChem database. Notably, no task-specific fuel property data was used during the instruction fine-tuning stage. Instead, the models learned transferable structural reasoning capabilities from general molecular data, a design choice that proved essential to generalizability across downstream fuel-property tasks.

Prediction itself relies on dynamic in-context learning. The team integrated a retrieval algorithm built on 166-bit MACCS molecular fingerprints and Tanimoto similarity search, executed efficiently through the Hierarchical Navigable Small World, or HNSW, algorithm. When a target molecule is submitted, the system automatically retrieves structurally similar reference molecules from updated property databases and weaves them into informative in-context prompts. This enables real-time database updating without any modification of model weights, so newly measured properties can be exploited immediately.

Evaluations covered 10 critical fuel property datasets spanning ignitability, sooting tendency, volatility, and thermodynamics. The fine-tuned 7-billion-parameter FuelProp-LM consistently outperformed the 685-billion-parameter industry-leading DeepSeek-V3.2 baseline and compared favorably with several conventional machine learning approaches. The result is a striking demonstration of parameter efficiency: a model roughly one-hundredth the size of the leading general-purpose LLM delivered competitive or superior predictive performance across all downstream tasks, without additional fine-tuning on the evaluated datasets.

Beyond raw accuracy, the framework eliminates the need for multiple independent QSPR models. Ten essential fuel properties are predicted within one language model using SMILES strings and retrieved in-context examples, removing the tedious process of manual descriptor construction and task-specific model retraining. The accuracy, generalizability, data scalability, and ease of use position FuelProp-LM as a flexible tool for future AI-assisted sustainable fuel design.

To explain why instruction tuning improves QSPR accuracy so dramatically, the researchers conducted an in-depth attention layer analysis, probing the internal workings of the Transformer architecture. The results show that fine-tuning fundamentally reallocates attention patterns within the intermediate layers responsible for semantic integration and reasoning. Attention shifted heavily toward the property description and the first in-context example, the structurally most similar molecule, rather than distributing uniformly across the prompt. Attention to the target molecule’s SMILES string also increased, strengthening the intrinsic mapping between chemical structure and target properties, while attention scores assigned to non-essential prompt components, such as repetitive question text or less relevant context examples, decreased significantly.

These mechanistic insights reveal how instruction tuning and in-context learning enhance molecular property prediction by sharpening chemically relevant attention patterns, offering a window into what is often treated as a black box. The findings suggest that domain-adapted smaller models can outperform far larger general-purpose systems on specialized scientific tasks when trained with the right instructional structure and contextual retrieval.

Professor Bin Yang, corresponding author of the study, said FuelProp-LM bypasses the tedious process of manual descriptor construction and task-specific model retraining. By offering high accuracy, data scalability, and deployment ease, he noted, the framework provides an accessible and flexible tool for future AI-assisted sustainable fuel design. As the clean energy transition accelerates and engine designers seek fuels tailored for efficiency and low emissions, language models adapted through instruction tuning and in-context learning may play a significant role in the next generation of fuel discovery.

Subject of Research: Tsinghua University researchers introduce FuelProp-LM, a language model framework combining instruction tuning and dynamic in-context learning to predict multiple fuel properties directly from molecular SMILES strings.

Article Title: Adapting language models for fuel property prediction

Article References: Adapting language models for fuel property prediction. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: large language models, fuel property prediction, QSPR, instruction tuning, in-context learning, SMILES, Tsinghua University, ENGINEERING Energy, AI chemistry, sustainable fuels, Adapting, language