A Humanoid Robot Now Speaks Quechua, Thanks to Fine-Tuned Neural Speech Synthesis

In a milestone for inclusive robotics, researchers at Universidad Señor de Sipán in Chiclayo, Peru, have given a humanoid social robot the ability to speak Southern Quechua, the largest indigenous language family of the Americas with more than ten million active speakers. The team fine-tuned a multilingual neural text-to-speech model, XTTS-v2, on 30.5 hours of validated Quechua speech and deployed it directly on the embedded hardware of the UBTech Walker Tienkung humanoid robot. The work, published in Frontiers in Robotics and AI, establishes the first quantitative benchmark for speech synthesis in an American indigenous language on a robotic platform.

The linguistic gap the study addresses is stark. Of the roughly 7,000 languages spoken worldwide, fewer than 100 are supported by modern automatic speech processing systems. Voice interfaces for education, healthcare, and public services increasingly mediate access to civic life, and humanoid social robots deployed in multilingual societies risk reproducing existing patterns of exclusion if their speech systems operate only in high-resource languages. Quechua, despite its demographic presence across Peru, Bolivia, Ecuador, Colombia, and Argentina, remains severely underrepresented in speech technology.

Technically, the challenge is formidable. Quechua is an agglutinative language in which layers of suffixes chain to a root, so a single word can simultaneously encode tense, grammatical person, case relations, evidentiality markers, and discourse relations, producing lexical forms of high phonological and structural complexity. The researchers focused on the Cusco-Puno dialect of Southern Quechua, the most computationally documented variety, using a CC0-licensed corpus from the Mozilla Data Collective comprising 22,727 validated utterances from 77 unique speakers.

A key innovation was a tokenizer adaptation that exploits the phonological overlap between Quechua and Spanish. Because XTTS-v2 does not natively support Quechua, the team applied a runtime patch that intercepts the text preprocessor and redirects Quechua input to the Spanish preprocessing pipeline, whose Latin phoneme space is the most compatible with the official Quechua alphabet among the 16 languages in the base model. This strategy enables effective fine-tuning for a language entirely absent from the model’s training data without requiring a custom tokenizer or phoneme dictionary.

Fine-tuning ran over 15 epochs on an NVIDIA A100 40 GB GPU, completing 35,000 gradient update steps with a batch size of 8. The training cross-entropy loss fell from 4.084 to 2.652, a 35.0 percent reduction confirming effective adaptation to Quechua phonology and morphology. Component-level analysis revealed differentiated convergence: the text token component converged rapidly, while the mel-spectrogram component followed a more gradual descent, reflecting the greater complexity of the continuous acoustic space. Notably, the team documented that full FP32 precision with single-step gradient accumulation was mandatory for stability, as FP16 mixed precision produced NaN values from the earliest steps.

The deployment target was the Jetson AGX Orin 64 GB system-on-chip embedded in the Walker Tienkung, which delivers roughly 275 TOPS of INT8 AI performance and 64 GB of unified memory, enough to hold the 1.87 GB FP32 model without VRAM constraints. Inference latency was evaluated on seven test utterances spanning the morphological complexity spectrum of the dialect, from the single-word greeting allillanchu to compound sentences with evidentiality markers. On the A100, mean latency was 2,037 ms with a Real-Time Factor of 0.472; on the robot’s embedded SoC, mean latency was 5,913 ms with a mean RTF of 0.961.

Both configurations achieved a mean RTF below 1.0, demonstrating real-time Quechua speech synthesis directly on the robot’s hardware without external computing infrastructure, a critical consideration for Andean communities with limited connectivity. The 2.9-fold latency difference between the data center GPU and the embedded SoC is consistent with their floating-point performance ratio. While the 5,913 ms latency exceeds the roughly 300 ms perceptual threshold for natural conversational turn-taking, it is compatible with non-conversational applications such as guided tours, educational presentations, and interactive signage.

The intelligibility evaluation exposed a structural methodological problem rather than a synthesis failure. Whisper large-v3, applied with automatic language detection, yielded a Word Error Rate of 147.4 percent and a Character Error Rate of 69.1 percent, but Whisper has no native Quechua model and systematically assigned the synthesized utterances to six different languages, including Turkish, Swahili, Indonesian, English, Spanish, and Latin. WER values above 100 percent arise when the ASR system inserts complete words not present in the reference. Qualitative inspection revealed partial phonetic correspondences, such as allillanchu transcribed as ali yancun, suggesting the model synthesizes speech with genuine phonetic structure. The researchers argue that perceptual evaluation with native speakers is a necessity, not a preference, for validating TTS in unsupported indigenous languages.

The team has documented a complete deployment pipeline, encompassing corpus preprocessing, tokenizer adaptation, training stability configuration, and checkpoint loading, as a reproducible framework for other low-resource languages of the Americas. Planned next steps include ONNX export and dynamic INT8 quantization, projected to reduce the model to roughly 0.61 GB and improve the RTF on the Orin by a factor of 3 to 5, bringing projected RTF to approximately 0.19 to 0.32 and absolute latency to about 1,150 to 1,900 ms. Full ROS 2 integration via a dedicated synthesis node, with a 200-entry LRU cache for high-frequency phrases and priority-queue request management, is also specified.

Beyond the engineering achievement, the researchers frame the work as an affirmation of linguistic rights. A Spanish-Quechua-speaking humanoid robot deployed in museums, schools, and public institutions in Peru represents a novel contribution to inclusive human-robot interaction. The framework is designed to extend to other Quechua varieties such as Ayacucho-Chanka and Bolivian Quechua, and to other Andean languages such as Aymara, though the team notes that dialect-specific corpora and reassessment of the tokenizer strategy would be required. As social robots become increasingly present across Latin America, the study demonstrates that they can, and must, be designed to operate in the native languages of the populations they serve.

Subject of Research: Quechua speech synthesis on a humanoid robot

Article Title: Enabling Quechua speech in humanoid social robots through fine-tuned neural TTS on the UBTech Walker Tienkung

Article References: Castro-Mejia, P., Atalaya-Urrutia, W., Cruzata-Martinez, A., Salazar-Montoya, E., Jimenez-Garcia, A., & Arcila-Diaz, J. (2026). Enabling Quechua speech in humanoid social robots through fine-tuned neural TTS on the UBTech Walker Tienkung. Frontiers in Robotics and AI, 13, Article 1909076. https://doi.org/10.3389/frobt.2026.1909076

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1909076

Keywords: Quechua, text-to-speech, humanoid robots, indigenous languages, low-resource languages, XTTS-v2, Jetson AGX Orin, human-robot interaction, Enabling, speech, humanoid, social