ChatMPC Integrates Natural Language Processing with Predictive Control for Autonomous Driving

Researchers have developed ChatMPC, an autonomous driving framework designed to bridge natural language understanding and dynamic Model Predictive Control. In dynamic traffic environments, conventional autonomous driving systems frequently rely on static rule-based architectures and predetermined control strategies. While traditional rule-driven methods such as decision trees, threshold controllers, and classical Model Predictive Control provide trajectory planning with mathematical rigor and stability, their predetermined objective functions and fixed parameter settings restrict their ability to accommodate changing traffic conditions and personalized driver intentions. The ChatMPC system addresses this limitation by establishing a closed-loop, real-time mapping pipeline that converts spoken or typed natural language commands into continuous mathematical control parameters, objective functions, and constraints.

The overall ChatMPC architecture operates through a hierarchical design composed of two primary interconnected components: an Intent Recognition Module and an MPC Controller Module. When an operator issues a natural language instruction, such as a request to overtake or to follow a preceding car, the Intent Recognition Module parses the command into a structured, machine-readable format. Rather than relying on simple keyword spotting or shallow semantic parsing, which often misclassifies diverse phrasings that share identical core intents, the architecture processes the entire linguistic context. Once the driving intent is parsed and structured, it is transmitted to the downstream MPC Controller Module, which dynamically reconfigures its optimization problem by modifying target trajectories, cost function weights, and operational boundaries to match the requested maneuver.

The natural language intent recognition pipeline begins with a semantic encoding stage using Sentence-BERT, specifically utilizing the bert-base-nli-stsb-mean-tokens pre-trained model comprising 110 million parameters. This language encoder processes user commands and applies a mean pooling strategy across the token vectors of the final layer, yielding a dense 768-dimensional semantic embedding vector. In the subsequent semantic classification stage, this vector is supplied to a K-Nearest Neighbors classifier. By calculating the Euclidean distance between the generated sentence vector and reference samples stored in a pre-constructed intent vector library, the classifier identifies the closest matching intent category, such as overtaking, car-following, or maintaining distance, without depending on static grammar templates.

To train the intent recognition module and expand its semantic robustness, the researchers established an augmented training set containing 180 commands across key driving intents. The data augmentation strategy utilized multiple complementary techniques, including semantic substitution to introduce synonymous phrases while preserving core intent, and sentence reorganization to vary word order across complex descriptions. The dataset was further expanded through sentence transformation, which converted imperative instructions into interrogative or conditional formats, as well as grammatical diversity enhancements covering varied sentence structures. Additionally, back-translation cycles across Chinese and English were implemented to generate natural linguistic variations, ensuring the classifier reliably processes varied sentence structures in bilingual contexts.

Following classification, the ChatMPC control module converts the structured intent into specific mathematical parameters within the optimization formulation. When the system detects a car-following intent, the longitudinal reference state is dynamically assigned to maintain a fixed 15-meter separation behind the lead vehicle, while the lateral reference position is locked to the center of the lane at 1.875 meters. To ensure rapid response to speed fluctuations of the lead vehicle during following maneuvers, the control time step is set to 0.04 seconds, and lateral position constraints are restricted to a narrow interval between 1.5 meters and 2.5 meters. In contrast, an overtaking intent shifts the longitudinal reference point to 15 meters ahead of the lead vehicle, extends the prediction time step to 0.08 seconds to handle an extended planning horizon, and broadens lateral position constraints to a range of 1.5 meters to 6.5 meters to permit safe lane changes.

Trajectory prediction within the control framework relies on a physical foundation modeled as a two-degree-of-freedom bicycle kinematics model. This model assumes that the left and right tires share identical steering angles and rotational speeds, allowing them to be represented as single tires on each axle. The kinematics formulation further assumes front-wheel drive, zero sideslip angle, rear steering fixed at zero, and wheel velocity vectors aligned with rotational direction without wheel slip. Under these dynamics, the ego-vehicle state tracks longitudinal and lateral positions, yaw angle, and longitudinal velocity, governed by longitudinal acceleration and front-wheel steering angle control inputs. The lead vehicle is modeled deterministically as traveling at a constant velocity along the lane center to preserve computational focus on the ego-vehicle.

Safety critical boundaries are embedded directly into the optimization problem through multi-circle collision avoidance constraints. Both the ego-vehicle and the lead vehicle are mathematically represented using two circles centered at their respective front and rear axle locations, defined by a specific radius and safety threshold. The controller enforces a hard minimum distance constraint between any circle center on the ego-vehicle and any circle center on the lead vehicle, maintaining a minimum boundary of 3.0 meters across all operating modes. This geometric formulation prevents vehicle overlapping during left-side overtaking maneuvers and lane transitions, avoiding the conservatism of simplified point-mass models while preserving lower computational complexity than elliptical approximations.

The control performance and computational feasibility of ChatMPC were evaluated through simulation experiments conducted on a Matlab platform running on an Intel Core Ultra 9 185H processor with 32 gigabytes of random-access memory. In a simulated car-following scenario where the ego-vehicle started at 12 meters per second and the lead vehicle traveled at 10 meters per second, the system converged to the required 15-meter following distance within 1.85 seconds, defined by staying within a five percent tolerance band. Following convergence, the steady-state longitudinal tracking error reached 0.827 percent with a lateral displacement standard deviation of 0.01 meters. In a simulated overtaking test starting from the same initial speeds, the vehicle executed the lane change and overtaking maneuver in 3.75 seconds, maintaining a steady-state tracking error of 1.67 percent relative to the forward target state.

Real-time timing analysis demonstrated that continuous control computations met stringent operational constraints. Operating at a target control frequency of 20 Hz, the continuous Model Predictive Control solver was allocated a 50-millisecond hard execution budget per control cycle. Utilizing the fmincon optimization solver configured with a prediction horizon of five steps and a time-shifting warm-start strategy initialized by previous optimal solutions, the controller achieved an average solve time of 13.5 milliseconds and a worst-case solve time of 22.2 milliseconds, retaining a 55 percent computational safety margin. In the offline pipeline, module initialization required 8.57 seconds, model loading took 0.97 seconds, and training data processing required 5.42 seconds, while the online single intent recognition stage executed in an average of 0.33 seconds, yielding a total pipeline time of 15.29 seconds.

The study demonstrated the viability of translating natural language commands into dynamically reconfigured control objectives within a simulated environment. The authors noted that future investigations will aim to broaden the framework by incorporating multilingual support, in-context personalization, and reinforcement learning derived from human feedback. Subsequent development will also focus on extracting finer-grained continuous control parameters directly from natural language expressions to generate increasingly nuanced objective functions and constraint boundaries across dynamic multi-vehicle interactions.

To contextualize the development of the framework, earlier automotive natural language interfaces were restricted to non-critical auxiliary tasks like infotainment and navigation, or relied on rigid keyword spotting that mapped discrete commands like accelerating or turning directly to simple vehicle actions. While recent exploratory efforts attempted to incorporate large language models such as GPT into autonomous driving architectures, those implementations suffered from substantial inference overhead and latency that precluded dynamic vehicle control. Furthermore, initial conceptual proposals for ChatMPC by Miyaoka and colleagues adjusted controller parameters without accounting for underlying vehicle dynamics or multi-vehicle interactions, creating potential stability hazards in complex environments. By contrast, the current architecture was structured to translate nuanced driving intentions into continuous control signals while preserving formal dynamic and physical safety boundaries.

The mathematical optimization underlying the system coordinates state objectives, input energy, and physical movement limitations. The controller’s objective function balances terminal tracking error against cumulative state error and control expenditure across the prediction horizon, utilizing weighting matrices to safeguard passenger comfort and economic energy usage. In conjunction with trajectory tracking, the formulation enforces strict physical boundaries, including maximum longitudinal velocity, upper and lower acceleration bounds, maximum front-wheel steering angles, and allowable yaw angle ranges. These constraints prevent extreme maneuvers and ensure that the generated control inputs remain within the operational envelope of the vehicle platform.

Kinematic motion calculations also incorporate the geometric relationship between front-wheel steering and vehicle turning geometry. The turning curvature radius is defined by the distance between the front and rear axles divided by the tangent of the front steering angle, from which the vehicle’s yaw rate is derived directly as the product of velocity and the tangent of the steering angle over wheelbase length. In the simulation evaluations, the lead vehicle traveled along a 3.75-meter-wide lane, providing a constant-velocity benchmark to test intent extraction from representative commands such as following or overtaking. By resolving the conflict between rigid control constraints and flexible human requests, the framework maintains predictable physical motion while accommodating personalized driving styles.

Subject of Research: Language-driven model predictive control for autonomous driving

Article Title: ChatMPC: a language-driven model predictive control framework for adaptive and personalized autonomous driving

Article References: Xu, W., Yin, Z., Zhou, Y., Zhu, Y., Wang, M., Lei, J., & Chen, H. (2025). ChatMPC: a language-driven model predictive control framework for adaptive and personalized autonomous driving. Autonomous Intelligent Systems, 5(1), Article 27. https://doi.org/10.1007/s43684-025-00116-x

Image Credits: AI Generated

DOI: 10.1007/s43684-025-00116-x

Keywords: autonomous driving, model predictive control, natural language processing, Sentence-BERT, vehicle kinematics, trajectory planning, simulation, ChatMPC, language-driven, model, predictive, control