The platform at the heart of the work is Artaban, a quadruped robot with twelve degrees of freedom, each leg driven by three Maxon motors. It carries four camera modules, two standard RGB and two depth-sensing RGB-D units, plus a time-of-flight camera in its chest, an inertial measurement unit, motor encoders, temperature sensors, and a built-in speaker. A locomotion controller based on Nonlinear Model Predictive Control translates high-level velocity commands into optimized foot trajectories. Onboard computing comes from an Intel NUC 11 with a Core i7 processor and an NVIDIA RTX 2060 GPU running Ubuntu 20.04 and ROS Noetic.

What makes SMaRTAban distinctive is how it stitches together the pieces of a complete voice-command pipeline. Spoken input passes through voice activity detection adapted from the Google Speech Recognition library, then to OpenAI’s hosted Whisper service, which automatically detects the language and transcribes the audio. The transcript is handed to a ReAct agent built on the LangGraph framework, where GPT-4o reasons step by step and selects from seven custom tools: moving the robot, stopping it, sitting down, standing up, interpreting camera images through GPT-4o vision, synthesizing speech, and waiting. Everything is displayed in a React-based graphical interface so the operator can see every tool call, every response, and every error in real time.

Safety receives unusually careful treatment for a language-driven system. Because large language models are probabilistic by nature, the researchers deliberately avoid relying on the model alone to keep the robot safe. Each tool has a typed schema whose arguments are validated before execution, and invalid parameters are rejected and returned to the model as errors without any physical action. More importantly, nearly every tool runs inside a subprocess guarded by a custom “interruptible” decorator. When the operator presses the interrupt button in the GUI, the main process kills that subprocess and independently issues a zero-velocity command to the robot — no LLM consultation required. Shutting down the application triggers the same stop, and a physical emergency-stop button cuts motor power as a final backstop. The researchers are candid that this layered approach still lacks a controller-level watchdog or velocity timeout, so it reduces risk without providing a formal safety guarantee.

The evaluation is unusually thorough for this field. A pilot voice study collected 120 recordings of four phrases — two short and two long, in both English and Slovak — spoken ten times each by three volunteers. English transcription was highly accurate, with word error rates of just 0.04 to 0.08. Slovak proved much harder at the acoustic level: the short phrase “Choď dopredu” (go forward) reached a word error rate of 0.97, occasionally being misidentified as a different Slavic language. Yet here is the striking part — even when the transcription looked mangled, meaning often survived. The short Slovak phrase still scored 0.87 on BERTScore, a semantic similarity measure, and the system correctly interpreted the intended command in 87 percent of recordings. The longer Slovak phrase reached a BERTScore of 0.96 and 90 percent understanding despite a word error rate of 0.25.

The core of the paper is a controlled ablation spanning 4,800 accepted tool-calling trials: two models, GPT-4o and GPT-4o-mini, across six agent and prompting configurations, two languages, ten command scenarios, and twenty repetitions each. The scenarios covered translation and turning movements, posture changes, a vision query, and one demanding multi-step command requiring the robot to sit, wait, stand, move forward, turn right, stop, and then describe what it sees. The results tell a clear story about what actually makes language-driven robots reliable.

Simply giving the model access to tools was not enough. Without a task-specific system prompt, GPT-4o completed the full expected action sequence in only 29.75 percent of trials. Adding an explicit system prompt describing the action contract — move, then wait, then stop — lifted that to 52.50 percent. But the decisive jump came from few-shot prompting: three worked examples demonstrating the movement pattern. With iterative ReAct execution, the system prompt, and those demonstrations combined, GPT-4o achieved a tool-call F1 of 0.9977 and 96.5 percent exact-sequence success. GPT-4o-mini followed the same trend, reaching 82 percent exact-sequence success at roughly one-twentieth of the model cost, though it failed to produce a single exact sequence for the complex multi-step command.

That gap between individual correctness and complete sequences is one of the paper’s most important lessons. On the complex scenario, GPT-4o-mini earned a respectable F1 of 0.8764 because most of its individual tool calls were right — but the ordering was wrong, typically invoking the camera before the required final stop. GPT-4o completed 26 of 40 complex sequences, including all 20 Slovak repetitions. The authors argue, convincingly, that robotics evaluations of language models must report task-level sequence success, not just per-call accuracy, because a command with a missing or misplaced stop action can be physically dangerous even when it scores well on superficial metrics.

Latency measurements complete the practical picture. On the deployed robot’s own computer, the first tool callback arrived a median of 2.74 seconds after the nominal end of a recorded utterance, rising to 4.10 seconds at the 95th percentile. By contrast, the local ROS stop-service round trip took a median of just 4.28 milliseconds. In other words, cloud transcription and model inference dominate the response path, while local robot communication is nearly instantaneous. This supports a sensible division of labor: the LLM handles interactive, high-level interpretation, while time-critical stabilization and locomotion stay with deterministic local controllers. The study also verified conversation history retention, hazard detection — the system correctly flagged a carpet cutter with an exposed blade as dangerous — graceful error recovery, and reliable mid-execution interruption.

The authors are careful about the limits of their claims. The functional study used scripted commands in dry-run mode, so physical motion quality was not quantified; the voice study involved only three speakers and four phrases; Slovak difficulty at the transcription stage remains an upstream failure mode when meaning is genuinely lost; and dependence on remote APIs introduces network latency and availability concerns. Still, SMaRTAban stands out as the first system in its field to report an evaluated combination of bilingual voice control in English and Slovak, on-demand GPT-4o vision, operator-controlled interruption of running actions, and rigorous quantitative analysis of the whole command path. Future work points toward native multimodal audio input, locally hosted open-source models to remove cloud dependency, the Model Context Protocol for portable tool definitions, integration with SLAM for autonomous navigation, and formal human-robot interaction studies with non-expert users. For now, the image of a quadruped robot that understands “Choď dopredu” as readily as “Go forward” — and stops the instant you tell it to — marks a meaningful step from Čapek’s century-old fiction toward an everyday robotic companion.

Subject of Research: A voice-controlled large language model agent enabling bilingual natural-language control of a quadruped mobile robot with integrated vision and safety mechanisms.

Article Title: SMaRTAban: a voice-controlled LLM agent for quadruped mobile robotics with integrated vision

Article References: Zelenay, E., Kocúr, M., Lukáč, M., Duchoň, F., & Marko, R. (2026). SMaRTAban: a voice-controlled LLM agent for quadruped mobile robotics with integrated vision. International Journal of Intelligent Robotics and Applications. https://doi.org/10.1007/s41315-026-00593-0

Image Credits: AI Generated

DOI: 10.1007/s41315-026-00593-0

Keywords: large language models, quadruped robotics, voice control, Whisper, GPT-4o, ReAct agent, human-robot interaction, natural language interface, LangGraph, robot safety, computer vision, ROS