The team built their experiment around a retained Arabic speech pool of 99,140 audio segments totaling 296.94 hours at 24-kilohertz mono quality, initialized from the ArVoice resource and expanded with locally curated recordings. Crucially, they kept this large pool separate from the much smaller fixed subsets actually used in the controlled adaptations, which ranged from 0.805 to 4.245 hours. The diagnostic benchmark itself was deliberately compact: 60 sentences split into three equal categories of general prose, case-sensitive material, and pause-sensitive material, with 99 predefined token-level case-ending targets covering genitive kasra, accusative fatḥa, nominative ḍamma, and a tiny three-target tanwīn class. Exact audits confirmed no overlap between the benchmark texts and the training manifests by raw text, normalized text, source identifier, or reference-audio hash, protecting the evaluation from contamination.

The controlled systems were designed as matched comparisons. Raw-M, Full-M, and Partial-M all used the same 1,000 audio clips, differing only in the text supplied during adaptation: ordinary undiacritized Arabic, complete synthetic marking from a locked diacritization pipeline, or a deterministic partial form that retains gemination marks and final marks while removing most internal short vowels. Rescue-M added a frozen punctuation-guided rescue subset of another 1,000 segments, recovered from material excluded by duration or text-length filters by splitting text at punctuation marks and placing audio boundaries at local low-energy points. All controlled adaptations started from the same off-the-shelf XTTS v2 checkpoint, trained with AdamW at a learning rate of 5×10⁻⁶ and an effective batch of 24, a configuration modest enough to run on a single NVIDIA GeForce RTX 2060 SUPER with 8 GiB of memory.

The headline result is stark: the untouched baseline won. In the canonical evaluation, the off-the-shelf model, labeled B0, achieved a word error rate of 0.224 and a character error rate of 0.071, measured by transcribing its synthesized speech with the faster-whisper recognition model and comparing against frozen references. Every controlled adapted condition had a higher modeled overall error rate after Holm correction for multiple comparisons. Full-M, for example, showed rate ratios of 1.351 for word error and 1.536 for character error relative to B0, both statistically significant. The adapted systems were also indistinguishable from one another after correction, meaning that neither full synthetic diacritization nor partial marking produced a stable benchmark-wide advantage over raw text supervision.

There was one tantalizing exception. On the case-sensitive subset, where grammatical endings matter most, Rescue-M produced the lowest descriptive character error rate at 0.065, narrowly below the baseline’s 0.067. But when the team ran their planned case-sensitive contrasts through the multiplicity-adjusted statistical models, none remained statistically distinguishable. The rescue condition’s apparent edge also failed a second test: it changes both the recovered content and the amount of training audio, so it cannot be treated as a pure single-factor ablation. The authors describe Rescue-M as descriptively lowest on this subset, not statistically superior, a distinction that matters enormously in an era when a single favorable number can be mistaken for a breakthrough.

The most dramatic failure came from an experiment that was never trained at all. The team inserted an explicit pause token, , into prompts at inference time, a symbol the model had never seen during training. The result was catastrophic: word error rate more than doubled, with a rate ratio of 2.159, and character error rate more than tripled at 3.515, both with adjusted p values below 0.001. The unseen control token disrupted lexical realization so severely that synthesized speech introduced spurious material around intended pauses. The lesson is unambiguous: textual resemblance to a control instruction does not make an unseen token semantically meaningful to a multilingual speech model, and explicit pause control requires a trained representation rather than an ad hoc inference-time symbol.

Training dynamics added another layer of nuance. The team examined checkpoints at 1, 3, and 13 total epochs, with the long point obtained by restoring the 3-epoch checkpoint and running ten additional epochs. Validation loss declined steadily across all observed checkpoints for every system, yet external error trajectories were non-monotonic and configuration-dependent. Raw-M worsened at 3 total epochs and returned close to its 1-epoch level at 13, while Rescue-M improved descriptively at 3 and worsened at 13. None of the nine planned within-system checkpoint contrasts remained statistically distinguishable after correction. In other words, validation loss alone would have selected progressively later checkpoints even though the external metrics followed different shapes, a mismatch that argues strongly against choosing Arabic TTS checkpoints from internal optimization curves without rescoring a frozen external set.

The human evaluation revealed perhaps the study’s deepest finding: the verdict depends on who is listening. Two primary evaluators, R1 and an independent external evaluator R2, each completed 495 case-ending judgments, 300 naturalness and pronunciation ratings, and 240 pairwise comparisons. Their strict raw agreement across the common decisions was just 0.451, with a Cohen’s kappa of −0.057, essentially chance. R1 assigned 53 of 99 strict-correct case-ending judgments to the baseline and 43 to Rescue-M, while R2 gave the baseline 58, Full-M 60, Partial-M 59, and Rescue-M 57, producing a completely different ordering. Pronunciation ratings, by contrast, favored the baseline more consistently than naturalness scores did, and automatic recognition error correlated more strongly with pronunciation correctness than with naturalness, suggesting that lexical-articulatory quality is a more stable construct than overall listening appeal.

The study’s central contribution is therefore methodological rather than a claim that one adaptation universally replaces the off-the-shelf model. The same system can look preferable under one automatic subset, less convincing under corrected inference, and different again under naturalness, pronunciation, case-ending, or pairwise judgments. Evaluation choice is part of the scientific result. The authors recommend that checkpoints never be selected from validation loss alone, that synthetic diacritization be audited as model input rather than presumed beneficial, that unseen control symbols be trained before use at inference, and that localized linguistic benefits be replicated by another evaluator before being presented as robust improvements. For applications requiring general lexical reliability, the untouched XTTS v2 configuration remains the safer reference under this benchmark, a conservative conclusion that applies to the evaluated data and model version rather than to Arabic adaptation in general.

Future work points toward manually verified or uncertainty-aware diacritization, matched controls for additional rescue hours, multi-seed continuation experiments to account for checkpoint variability and catastrophic forgetting, and a broader expert panel with more syntactic constructions and balanced ending classes. The team also envisions a public evaluation resource preserving raw and diacritized references, item-level recognition outputs, target-level case judgments, and anonymized listening decisions, which would make cross-system replication far more meaningful than comparisons based on unmatched headline error rates. For now, the message for anyone building Arabic speech technology is clear: the apparent benefit of adaptation depends materially on the evaluation metric, the checkpoint, and the evaluator, and only a declared diagnostic contract that separates lexical agreement, targeted linguistic realization, perceptual quality, and training dynamics can reveal what an adaptation campaign actually changed.

Subject of Research: Artificial Intelligence

Article Title: A pronunciation-sensitive analysis of XTTS adaptation for modern standard Arabic synthetic diacritization length rescue and training dynamics

Article References: Elbasri, A., Morchid, A., Oqaidi, K., Tairi, H., Yahyaouy, A., Chadi, M.-A., & Elamrani Abou Elassad, Z. (2026). A pronunciation-sensitive analysis of XTTS adaptation for modern standard Arabic synthetic diacritization length rescue and training dynamics. Discover Artificial Intelligence, 6(1), Article 1243. https://doi.org/10.1007/s44163-026-02214-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02214-y

Keywords: text-to-speech, Modern Standard Arabic, XTTS, diacritization, speech synthesis, ASR, machine learning, low-resource languages, pronunciation-sensitive, analysis, adaptation, modern