The research team, led by Yuriy Perezhohin with Mauro Castelli and Aleš Popovič, fine-tuned two multilingual foundation models on identical data for each language. The first was Parakeet-TDT-0.6B-v3, a compact 0.6-billion-parameter transducer built on a FastConformer encoder that decodes speech frame by frame, predicting each token and the number of audio frames to skip before the next one. The second was Whisper-large-v3, OpenAI’s 1.55-billion-parameter encoder-decoder Transformer, whose autoregressive decoder attends over the full encoded audio and conditions every token on the complete history of what it has already written, effectively acting as an audio-conditioned language model. Both models had seen Estonian and Slovenian during pretraining, so neither entered the comparison blind, which distinguishes this study from earlier architecture comparisons in English and Polish where systems were trained from scratch or paired asymmetrically.

The synthetic data pipeline began with GPT-5-mini generating roughly six thousand candidate sentences per language across three diversity categories: paraphrases of existing CommonVoice transcripts, domain-expanded sentences introducing vocabulary absent from the training distribution, and morphology-targeted sentences engineered to stress rare Estonian case forms and Slovenian dual-number constructions. A separate LLM-as-judge validation call then screened each candidate for grammaticality, naturalness, appropriate length, and language purity, passing 68.3 percent of Estonian and 74.5 percent of Slovenian candidates on the first attempt. Failed items were regenerated with corrective feedback, recovering more than ninety percent of them, and near-duplicates were removed using word-level Jaccard similarity. The final validated corpora comprised 5,852 Estonian and 5,848 Slovenian sentences, synthesized with eleven distinct voices into approximately 10.4 and 10.3 hours of audio respectively.

The gains were unambiguous in-domain. Fine-tuning on CommonVoice alone left Parakeet at a raw word error rate of 22.35 percent on Estonian and 14.08 percent on Slovenian; adding the full synthetic corpus pushed these down to 21.03 and 11.56 percent respectively. Every one of the sixteen comparisons between real-only and real-plus-synthetic training was statistically significant under paired bootstrap resampling with one hundred thousand iterations, at p less than ten to the minus five. Notably, Parakeet, with 2.5 times fewer parameters than Whisper, achieved the lower word error rate in every fine-tuned condition and on every test set under each model’s standard recipe. Before fine-tuning, the two systems had divided the spoils: Whisper dominated Slovenian zero-shot at 21.20 percent versus Parakeet’s 50.23 percent, while Parakeet led on Estonian at 27.19 versus 34.40 percent. Fine-tuning erased that division entirely.

Out-of-domain behavior told a sharper story. Parakeet improved on the FLEURS benchmark in every fine-tuned condition for both languages. Whisper did not. Fully fine-tuning Whisper on the 1,388 Slovenian CommonVoice sentences raised its FLEURS word error rate from 37.02 to 46.79 percent, a 9.8-point degradation relative to leaving the model untouched. The synthetic mix recovered part of this loss, with the best augmented configuration reaching 40.46 percent, but no configuration returned the model to its zero-shot level. The authors attribute this partly to a register mismatch: the Slovenian training sentences average 5.9 words in a conversational style, while the Slovenian FLEURS references average 19.7 words of encyclopedic prose. Estonian, whose training and evaluation lengths are closer, showed no comparable effect.

Beyond aggregate scores, the study deployed a two-pronged error analysis: automated signature detectors applied to all 95,070 re-aligned sentence pairs, and an exhaustive manual reading of a stratified 1,735-utterance sample. The detectors revealed that Whisper produces substantially more complete misrecognitions than Parakeet, at 1.6 to 2.0 times the rate in every fine-tuned regime, and also more Estonian case-suffix confusions and diacritic-only errors, patterns that were present before any fine-tuning and retained their direction throughout. Parakeet, by contrast, exhibited a stable language-conditioned boundary bias: it split Estonian compound words at morpheme boundaries, turning

Subject of Research: Synthetic speech data augmentation for low-resource Estonian and Slovenian automatic speech recognition

Article Title: Synthetic speech augmentation for low-resource Estonian and Slovenian ASR: Comparing Parakeet-TDT and Whisper

Article References: Perezhohin, Y., Castelli, M., & Popovič, A. (2026). Synthetic speech augmentation for low-resource Estonian and Slovenian ASR: Comparing Parakeet-TDT and Whisper. Machine Learning with Applications, 26, Article 101043. https://doi.org/10.1016/j.mlwa.2026.101043

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101043

Keywords: automatic speech recognition, synthetic speech, data augmentation, Estonian, Slovenian, Whisper, Parakeet-TDT, text-to-speech, low-resource languages, large language models, morphology, error analysis

The stakes are enormous. According to the World Health Organization, nearly 2.5 billion people, roughly one in four worldwide, are projected to have some degree of hearing loss by 2050, with an estimated 700 million requiring rehabilitation services. But the problem extends far beyond those with diagnosed hearing loss. Many people whose audiograms look perfectly normal still report substantial difficulty following conversations in noisy environments. This gap between what the audiogram says and what the listener experiences has long frustrated clinicians, because hearing thresholds alone provide an incomplete and often misleading picture of real-world functional hearing.

Scientists have proposed several mechanisms to explain these hidden deficits. Emerging evidence points to conditions such as cochlear synaptopathy, a loss of connections between sensory cells and auditory nerve fibers, and impairments in extended high-frequency hearing, both of which can degrade the neural coding of sound without shifting standard threshold measures. The result is a central challenge for hearing healthcare: current clinical tools do not accurately predict how well an individual will understand speech in the acoustically messy conditions of everyday life. Conventional linear statistical models built on audiometric and demographic variables capture some group-level trends, but they miss the nonlinear interplay among auditory, acoustic, and cognitive factors, such as attention and working memory, that shape each listener’s performance.

Enter machine learning. The new review, conducted by Prabhani Athukorala, Anu Nair, and Srikanta K. Mishra of The University of Texas at Austin and the University of South Dakota, followed PRISMA 2020 guidelines and prospectively registered its protocol. The team searched PubMed, Scopus, Web of Science, IEEE Xplore, and Google Scholar, screening 6,468 records after duplicate removal. They included studies using traditional machine learning models such as support vector machines and random forests, neural network architectures ranging from early artificial networks to deep and convolutional models, automatic speech recognition systems, and mechanistic auditory models that simulate the physiology of hearing itself.

The quantitative synthesis focused on the model-metric combinations with enough comparable studies to pool: support vector machine accuracy, support vector machine area under the curve, random forest accuracy, and random forest recall. Using random-effects meta-analysis with the DerSimonian-Laird method, the researchers found that support vector machines pooled at an accuracy of 0.71 and an area under the curve of 0.84, the latter indicating good discrimination across the included classification tasks. Random forests fared better on the available evidence, pooling at an accuracy of 0.86 and a recall of 0.81. Individual study estimates varied widely, however, with accuracies ranging from 0.53 to 0.96 depending on the task and dataset.

Those wide ranges point to the review’s most important caveat: heterogeneity. Between-study variability ranged from moderate to very high across analyses, reflecting genuine differences in participant populations, prediction targets, feature sets, outcome definitions, and validation procedures. Because each pooled analysis included only three or four studies, the authors urge cautious interpretation of the heterogeneity statistics and pooled estimates. Bootstrap resampling with 10,000 study-level draws produced estimates broadly consistent with the primary analyses, and sensitivity analyses, including exclusion of a large dataset of 12,697 subjects, shifted pooled values only modestly. But the evidence base remains too small and too varied to declare any single algorithm the winner.

Indeed, the review explicitly rejects the idea that one model class is universally superior. Random forests may excel at modeling nonlinear interactions among correlated clinical predictors, while support vector machine performance appears more sensitive to feature scaling and kernel specification. Neural networks showed real promise for problems involving complex spectrotemporal representations, with deep architectures outperforming conventional objective measures in speech intelligibility estimation. Yet greater complexity did not consistently help: in several studies, deep neural networks and multilayer perceptrons underperformed simpler algorithms when datasets were small relative to model capacity. The practical lesson is that neural networks suit high-dimensional acoustic or signal-level data, whereas traditional machine learning remains highly effective for smaller, structured clinical datasets.

Two other model families offered complementary perspectives. Automatic speech recognition systems, including hybrid deep neural network and hidden Markov model architectures, reproduced key patterns of human performance, such as declining recognition scores as signal-to-noise ratios worsen, and incorporating listener-specific auditory parameters improved predictions for hearing-impaired profiles. These systems also proved useful for automating the design of speech-in-noise tests themselves. Mechanistic auditory models, represented by a single study predicting consonant recognition, embedded cochlear filtering, compression, and hearing loss simulations directly into the prediction pipeline, demonstrating that reduced audibility and suprathreshold distortions jointly degrade speech perception. Such physiologically grounded models offer interpretability that purely data-driven approaches lack, and the authors suggest that future progress may lie in hybrid frameworks combining both.

Perhaps the most clinically intriguing finding concerns what the models use as inputs. Several studies showed that speech-in-noise outcomes can be predicted more effectively when models incorporate information beyond conventional audiometric thresholds. Extended high-frequency thresholds, acoustic features, behavioral measures, and even cortical speech-evoked neural responses all contributed predictive information in different analyses. Extended high-frequency hearing, in particular, was implicated in speech-in-noise variability among listeners with clinically normal audiograms, hinting that deficits invisible to the standard audiogram may underlie real-world listening difficulty. This multidimensional picture supports the hope that machine learning could eventually identify people whose communication struggles are disproportionate to their measured hearing loss.

The authors are careful, however, to distinguish proof of concept from clinical utility. Only one included study reported independent external validation on an additional dataset, and most relied on retrospective data with internal validation alone. Metrics such as accuracy, area under the curve, recall, and correlation are not directly comparable across tasks, and the small number of studies precluded publication-bias assessment and subgroup analyses. The path forward, the review concludes, requires larger, standardized, externally validated, and prospectively evaluated studies spanning multiple clinics, populations, languages, and listening environments. Until then, the models remain promising prototypes rather than diagnostic tools. Still, the trajectory is clear: as datasets grow and validation standards mature, data-driven prediction could transform how clinicians detect hidden hearing deficits, personalize hearing aids and cochlear implants, and design smarter listening technology for a world that is only getting noisier.

Subject of Research: Machine learning prediction of speech-in-noise perception in listeners with normal and impaired hearing

Article Title: Data-Driven Prediction of Speech-in-Noise Performance: A Systematic Review and Meta-analysis of Machine Learning Models

Article References: Athukorala, P., Nair, A., & Mishra, S. K. (2026). Data-Driven Prediction of Speech-in-Noise Performance: A Systematic Review and Meta-analysis of Machine Learning Models. Annals of Biomedical Engineering. https://doi.org/10.1007/s10439-026-04407-z

Image Credits: AI Generated

DOI: 10.1007/s10439-026-04407-z

Keywords: machine learning, speech-in-noise, hearing loss, support vector machine, random forest, neural networks, automatic speech recognition, auditory models, meta-analysis, audiometry, cochlear synaptopathy, hearing rehabilitation

The stakes are framed by United Nations Sustainable Development Goal 12.3, which calls on nations to halve global food waste by 2030. Progress toward that goal depends on accurate measurement, and that is precisely what is missing at the household level. National estimates of food waste are typically assembled from surveys, municipal waste audits, or extrapolations from retail and manufacturing data, none of which capture the true composition and quantity of what individual families discard. Without trustworthy household-level numbers, it is nearly impossible to know whether interventions such as meal planning apps, portion guidance, or composting incentives actually reduce waste. The Oregon State team argues that this lack of a robust measurement methodology has actively hindered global efforts to meet the 12.3 target, and their smart compost bin is designed to close that gap with hardware that fits naturally into the daily routine of a normal kitchen.

The device itself is elegantly simple in concept. It is a compost bin instrumented to collect three streams of data every time food waste is added: an image of the scraps, the weight of the deposit, and environmental readings from onboard sensors. By combining visual and mass information at the moment of disposal, the bin produces a rich, instance-level record of each waste event rather than a coarse weekly tally. This matters because food waste is notoriously heterogeneous. A single evening’s compost deposit might contain coffee grounds, melon rinds, chicken bones, and moldy bread, all commingled together. Separating and identifying those components after the fact is one of the hardest problems in waste research, and the smart bin’s approach of capturing data at the point of disposal sidesteps much of that difficulty.

One of the most practical innovations in the system is the way it streamlines annotation, the laborious process of labeling data so that machine learning models can learn from it. Traditional food waste studies require researchers to manually sort, identify, weigh, and photograph discarded items, a process that is slow, unpleasant, and prone to error. The Oregon State system merges weighing and imaging into a single step, so that the act of tossing scraps into the bin automatically generates a labeled data point. Even more cleverly, the bin supports hands-free verbal labeling at the time of measurement through automatic speech recognition. A user can simply say what they are throwing away as they drop it in, and the spoken description is captured alongside the image and weight data. This design turns an everyday chore into continuous, low-effort data collection, which is exactly what a large-scale measurement campaign requires.

The hardware is paired with a companion smartphone application called CompostSense, which serves two purposes at once. For households, the app provides waste analytics, giving users feedback about what and how much they are composting over time. That feedback loop is valuable in its own right, since behavioral research suggests that people waste less food when they can see the scale of their own discards. For the research team, the app doubles as a platform for collecting instance-level food waste image annotations, allowing participants to confirm or correct the labels attached to each waste event. This dual-use design means that every household enrolled in a study simultaneously benefits from the technology and contributes to the dataset that makes the technology smarter, a virtuous cycle that few prior food waste measurement tools have achieved.

Underpinning the whole effort is a candid assessment of why computer vision has not yet been applied successfully to household food waste. Vision models have transformed fields from medical imaging to autonomous driving, but they depend on large, well-annotated datasets. For commingled food waste, no such publicly available dataset exists, and the authors identify this paucity as the central reason researchers have been unable to leverage modern vision advances for automatic food waste measurement. Images of food waste are genuinely difficult: items are partially occluded, stained, decomposing, and piled on top of one another in ways that differ radically from the clean, well-lit food photography found in existing food recognition datasets.

To demonstrate that difficulty concretely, the team trained and evaluated computer vision models for food image recognition, including instance segmentation approaches built on modern YOLO-family architectures, and showed that performance does not transfer from ordinary food imagery to the food waste domain. A model that can segment a plated meal with impressive accuracy stumbles when confronted with a soggy, overlapping mass of scraps in a dimly lit bin. This negative result is arguably as important as the hardware itself, because it establishes empirically that simply reusing existing food recognition models will not solve the waste measurement problem. Instead, the field needs a large, purpose-built dataset of real commingled food waste images, collected in real kitchens under real conditions.

That dataset is the immediate goal of the project’s next phase. The authors describe a forthcoming field study in which fifty smart compost bins will be deployed to households to curate a large, novel dataset of commingled food waste images. Each bin will passively accumulate images, weights, environmental data, and speech-based labels over months of ordinary use, producing exactly the kind of in-the-wild training data that laboratory studies cannot replicate. With fifty simultaneous deployments, the study will also capture geographic and cultural variation in diets and disposal habits, which is essential if the resulting models are to generalize beyond a single community. The annotation workflow, with its single-step weighing and imaging plus verbal labeling, is designed to keep participant burden low enough that such a long-running deployment remains feasible.

The broader implications extend well beyond measurement. Reliable household food waste data would allow governments to benchmark progress toward SDG 12.3 with unprecedented granularity, letting cities compare neighborhoods, evaluate programs, and direct resources where waste is worst. For individual households, real-time analytics could nudge behavior in ways that surveys never can, showing a family that it discards the equivalent of several meals each week. Downstream, better sorted and quantified compost streams could improve biogas production and composting operations, since feedstock composition strongly affects both processes. The researchers are careful to frame the current work as infrastructure for science rather than a finished consumer product, but the architecture they describe, from sensor-laden bin to speech-labeled dataset to smartphone feedback, sketches a complete pipeline from raw kitchen scraps to actionable knowledge.

The study also offers a quiet lesson about how applied AI research actually progresses. Rather than promising a turnkey waste recognition system, Beery and Donnelly identify the true bottleneck, the absence of public commingled food waste data, and build the hardware, software, and human-centered annotation tools needed to remove it. The undergraduate engineering team at Oregon State University that fabricated the bins and built the CompostSense app contributed to a system whose value will compound over time: every household that uses it makes the underlying models better, and better models make the next generation of bins more capable. As the fifty-bin field study moves forward, the project stands as a template for tackling other stubborn sustainability measurement problems, showing that sometimes the most impactful AI research begins not with a clever algorithm but with a well-designed bin that listens, weighs, and watches as we throw food away.

Subject of Research: Automatic household food waste measurement using an AI-enabled smart compost bin with computer vision and speech recognition

Article Title: Towards automatic household food waste measurement with the smart compost bin

Article References: Beery, A., & Donnelly, P. J. (2026). Towards automatic household food waste measurement with the smart compost bin. Discover Sustainability. https://doi.org/10.1007/s43621-026-04774-6

Image Credits: AI Generated

DOI: 10.1007/s43621-026-04774-6

Keywords: food waste, smart compost bin, computer vision, machine learning, SDG 12.3, sustainability, household waste measurement, automatic speech recognition, instance segmentation, food waste dataset, composting, Oregon State University

The stakes could hardly be higher. Language impairments arising from aphasia after stroke, developmental language disorders, traumatic brain injury, neurodegenerative conditions such as Parkinson’s disease, and other neurological conditions affect communication in ways that standard speech technology was never designed to handle. Disfluent speech, word-finding pauses, mispronunciations, grammatical breakdowns, and atypical prosody can all scramble the acoustic and statistical patterns that modern recognition systems rely on. When a person with aphasia says a word haltingly or produces a neologism in place of the intended target, a recognition system trained predominantly on fluent, typical adult speech may simply fail, and that failure cascades downstream into every application that depends on accurate transcription.

The architecture of modern speech recognition helps explain why. State-of-the-art systems, including end-to-end neural models trained on tens of thousands of hours of audio, learn to map sound sequences to text by exploiting statistical regularities in their training data. Those regularities include not just phonetics but also the linguistic content of the speech itself. A recognizer hearing a garbled or incomplete utterance leans heavily on language-model priors to guess what was said, essentially autocorrecting toward plausible fluent speech. For typical speakers this bias improves accuracy, but for individuals with language impairments it can systematically distort what they actually said, replacing their intended words with the model’s own statistical expectations and effectively silencing their voice in favor of the algorithm’s prediction.

The research team evaluated how this plays out empirically by testing recognition systems on speech produced by individuals with language impairments and comparing performance against typical speech benchmarks. The results reveal a substantial performance gap. Word error rates climb steeply on impaired speech, and the errors are not randomly distributed: content words, which carry the semantic heart of a message, are disproportionately misrecognized or dropped, while function words are preserved. That asymmetry matters enormously, because a transcript that keeps the grammatical scaffolding but loses the meaningful content is nearly useless both for human readers and for any downstream language model asked to interpret, expand, or respond to the speaker’s intent.

On top of the transcription layer, the study examines large language models as assistive partners, the idea being that a person with aphasia might produce a fragmented utterance, which the recognizer transcribes imperfectly, and which the language model then attempts to repair, expand, or convert into a well-formed communicative act such as a text message or an email. In principle this pipeline could restore independence for people who struggle with everyday written and spoken communication. In practice, the researchers find that the pipeline inherits and sometimes amplifies the weaknesses of each stage. When the recognizer drops a content word, the language model cannot know what is missing, so it fluently completes the sentence with the wrong meaning, producing output that looks polished but betrays the speaker’s actual intent.

This problem of confident error propagation is one of the most consequential findings. Large language models are trained to produce coherent, plausible text, and they apply that objective whether or not the input transcript was accurate. The studies show that models asked to repair impaired-speech transcripts will happily generate grammatical, natural-sounding sentences that diverge from what the speaker meant. For assistive communication, such fluent fabrication is arguably worse than a raw, broken transcript, because conversation partners and caregivers may assume the polished output is trustworthy. The researchers emphasize that any deployed system must therefore build in transparency about uncertainty, letting users verify and correct the interpretation rather than presenting the model’s guess as fact.

Encouragingly, the work also identifies pathways toward improvement. Recognition accuracy for impaired speech improves substantially when systems are adapted, whether through fine-tuning on disordered speech data, personalizing acoustic models to an individual speaker’s voice and error patterns, or injecting contextual information about the communicative setting. Similarly, language models perform better as assistive aids when they are constrained, prompted with information about the speaker’s typical vocabulary and communication goals, or paired with interactive correction loops in which the user can confirm or reject candidate interpretations. These findings suggest that the technology is not fundamentally unsuited to this population, but that off-the-shelf deployment is. Deliberate, user-centered engineering is required to close the gap.

The study also raises important questions about evaluation methodology in the field. Benchmarks that dominate speech technology research contain almost no disordered speech, so headline accuracy figures say little about performance for this population. The authors argue for including individuals with language impairments in dataset collection, reporting performance stratified by speaker characteristics and impairment severity, and evaluating assistive pipelines end to end rather than optimizing transcription accuracy in isolation. A system that achieves slightly worse word error rates but preserves meaning and supports successful repair might serve users far better than one optimized purely for a conventional benchmark metric. Aligning evaluation with real communicative outcomes is presented as a necessary step for the field.

Beyond the technical conclusions, the research lands at a moment of intense public debate about artificial intelligence and accessibility. Voice interfaces are becoming the default way people interact with phones, homes, vehicles, and services, and large language models are being embedded in virtually every communication tool. If these systems systematically fail people with language impairments, the accessibility divide will widen even as technology advances, turning everyday tasks that others take for granted into new barriers. Conversely, if the gaps identified here are addressed through inclusive data, careful system design, and genuine involvement of affected communities, the same technologies could deliver on their long-promised potential: restoring voice, autonomy, and connection to people whose communication abilities have been compromised by injury or disease. The study’s message is ultimately one of cautious optimism grounded in rigor. The tools are powerful, the gaps are measurable, and the solutions are identifiable. What remains is the commitment to build speech and language AI not just for the typical speaker, but for the full diversity of human communication, ensuring that the next generation of voice technology leaves no voice behind.

Subject of Research: Evaluation of automatic speech recognition and large language models for assisting individuals with language impairments

Article Title: Assessing the use of automatic speech recognition and large language models for individuals with language impairments

Article References: Xu, G., Yu, H., Wei, L., Liu, Y., Liu, D., Xu, C., Li, J., Abbasi, A., Xiong, J., Yu, X., Zheng, Z., Shi, Y., & Qin, R. (2026). Assessing the use of automatic speech recognition and large language models for individuals with language impairments. Nature Communications. https://doi.org/10.1038/s41467-026-76677-z

Image Credits: AI Generated

DOI: 10.1038/s41467-026-76677-z

Keywords: automatic speech recognition, large language models, language impairments, aphasia, accessibility, assistive technology, speech recognition accuracy, word error rate, communication disorders, AI bias, inclusive AI, Nature Communications

The dataset, described in the Journal of Ambient Intelligence and Humanized Computing, contains 18.54 hours of audio spanning 9300 recordings produced by 34 bilingual participants, all of them fluent in both Mandarin and English. Unlike many earlier corpora, every single utterance in the collection involves code-switching. That design choice matters. Older resources such as SEAME, which stretches across roughly 190 hours, and TALCS, which covers 587 hours, contain substantial proportions of monolingual speech, meaning their effective supply of genuinely code-switched material is far smaller than their total length suggests. Some of those datasets, including the ASRU and TALCS corpora, are no longer publicly accessible at all, leaving the field with a shortage of usable, openly available benchmarks.

The construction of DOTA-ME-CS follows an unusual pipeline that blends large language model generation with human recording. The team used GPT-4o with carefully engineered prompts to produce scripted sentences across ten everyday scenarios: education, entertainment, environmental protection, food, health, home, life, pets, travel and work. Each prompt required the model to produce sentences that mimic daily conversational style, contain more English words than Mandarin words, and include at least one Mandarin word, with a dominant language assigned to every sentence. The authors justify this topic-anchored approach with a probabilistic argument: when a specific category is given, the probability of generating a relevant, high-quality sentence is higher than when the model is left to roam across all possible topics, which also reduces hidden cultural bias in the resulting scripts.

Human evaluators checked the generated scripts for grammatical problems and confirmed the presence of genuine switching, and a post-hoc naturalness study asked eight bilingual raters to score 100 randomly sampled stimuli on a five-point Likert scale. The average rating came in at 4.12 with a standard deviation of 0.61, and the median was 4.0, indicating that bilingual listeners generally perceive the generated sentences as natural, though the authors acknowledge that occasional awkwardness is an inherent limitation of LLM-based text generation. Bilingual volunteers, mostly college students with academic backgrounds in China and the United Kingdom, including roughly eighteen participants based at Imperial College London, then recorded the scripts on their own laptops or smartphones as 16-bit WAV files in quiet indoor settings. Recordings that failed basic quality checks were rejected and re-recorded, and accepted files were peak-normalised to 3 dBFS for consistent loudness.

The participants’ linguistic profiles were deliberately diverse. Twelve reported English dominance, eighteen reported Mandarin dominance and four identified as balanced bilinguals, and the mix of speakers from China and the United Kingdom ensures that the corpus spans multiple varieties of English and second-language accents. An average of 2.22 switching points per utterance, combined with a broad part-of-speech distribution across both languages, means the recordings capture switching at varied syntactic positions and grammatical categories rather than concentrating it in a single predictable spot. The dataset also includes longer recordings of roughly 10 to 15 seconds and about 100 words, an intentional response to the weakness current ASR models show on extended speech, where dependencies and critical information can be lost.

What sets the corpus apart most sharply is its AI-driven augmentation. Because human recordings were captured in quiet conditions at a normal pace, the researchers used the Librosa audio library to modify a random subset of clips. Playback speed was adjusted to 0.75x, 0.5x, 1.25x, 1.5x and 2x, with 200 recordings modified at each setting. Five categories of background noise, drawn from highways, war, natural sounds, white noise and playground environments, were added at 200 recordings per type, with the noise pitch scaled to 0.8x to account for the Lombard effect, the well-documented tendency of speakers to raise their vocal intensity in noisy surroundings. In addition, five AI-generated timbres, two male and three female, replaced the original voices in 200 recordings each, simulating both voice conversion deepfake scenarios and privacy-preserving speech processing conditions.

The accompanying data analysis is unusually thorough. Phoneme distributions catalogued with the International Phonetic Alphabet reveal that Mandarin contributes more multisyllabic pronunciations, tones, aspirated consonants and palatalised sounds, while English favours single-syllable vowels; shared features such as the sounds t and a suggest cues that future switching-detection models could exploit. Physical measurements show a frame rate of 46.7 kHz and a sample width of 2.0 bytes, meeting professional audio standards, with an average maximum pitch of 3617.13 Hz and an average per-recording pitch of 535.04 Hz. Formant frequencies computed through Linear Predictive Coding yield averages of 804.02 Hz, 4419.15 Hz and 7549.48 Hz for F1, F2 and F3, pointing to mid-to-low and front vowels, reduced lip rounding and retroflex consonants. The measured speaking rate of 2.05 words per second, with a standard deviation of 0.49, sits squarely within the typical range for conversational read speech, supporting the corpus’s ecological validity.

To establish baseline performance, the team benchmarked three pre-trained ASR models on the full dataset without fine-tuning: Whisper large-v3, a 1.5-billion-parameter Transformer encoder-decoder trained on 680,000 hours of multilingual audio; SenseVoice Small, a lightweight hybrid CTC-attention model of roughly 200 million parameters that also performs language identification, emotion detection and audio event detection; and Paraformer, a non-autoregressive Transformer with 220 million parameters optimised for Mandarin and built on continuous integrate-and-fire alignment. Paraformer and SenseVoice achieved the best results, with Whisper slightly behind, and a word error rate of 0.177 alongside a character error rate of 0.176. Compared against the publicly available ASCEND corpus, the models produced higher error rates on DOTA-ME-CS, which the authors read not as a defect but as evidence that their dataset poses a more demanding and therefore more useful benchmark.

Case studies expose exactly where today’s systems fail. In one example embedding the Chinese noun 菠萝包, meaning pineapple bun, inside an English sentence, Paraformer and SenseVoice produced garbled outputs such as bullleball and boobao, while Whisper translated the phrase correctly but destroyed the mixed-language structure by refusing to preserve the code-switch itself, a translation bias that also appeared on noise-free recordings. At double playback speed, Paraformer’s output strayed entirely from the intended meaning, SenseVoice descended into repetitions such as Miamiami, and Whisper mangled Florida lifestyle into Forensic Lifestyle. Altered timbres degraded English recognition across the board, and all three models performed worst at 2.0x speed. Whisper proved the most resilient to background noise, while Paraformer was the most sensitive to voice changes.

The implications reach beyond engineering. Reliable code-switching recognition would make automated transcription, voice assistants and translation systems far more useful for the hundreds of millions of people who live their linguistic lives between two languages, reducing a bias that currently disadvantages non-monolingual speakers. The authors stress that the study received ethical approval from Imperial College London’s ethical board, that all participants gave informed consent, and that privacy protections were maintained throughout. They are candid about limitations: funding constrained the scale of the corpus, and scripted reading, while reducing transcription errors and ethical risks, sacrifices the spontaneity of natural conversation. Even so, their deliberate trade-off of quality and coverage over raw scale gives the community something it has lacked, a fully public, exclusively code-switching Mandarin-English benchmark with baseline scores, rigorous acoustic analysis and data, code and recordings all released through a public repository. As speech technology races toward multilingual ubiquity, datasets like this one may determine whether the next generation of voice interfaces can finally follow the way people actually talk.

Subject of Research: A Mandarin-English code-switching speech dataset for advancing automatic speech recognition research.

Article Title: DOTA-ME-CS: daily oriented text audio-Mandarin English-Code switching dataset

Article References: Li, Y., Wei, Z., Yu, H., Xue, J., Zhou, H., & Schuller, B. W. (2026). DOTA-ME-CS: daily oriented text audio-Mandarin English-Code switching dataset. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-026-05119-x

Image Credits: AI Generated

DOI: 10.1007/s12652-026-05119-x

Keywords: code-switching, automatic speech recognition, Mandarin, bilingual speech, speech dataset, Whisper, Paraformer, SenseVoice, data augmentation, phonetics, large language models, voice conversion

The motivation behind the toolkit is rooted in a long-standing tension within the field. Laboratory speech, produced in word-reading or sentence-repetition tasks, gives experimenters precise control over variables and supports repeatability, which is why it has anchored theoretical work in speech perception and production for generations. But an accumulating body of evidence shows that listeners and speakers dynamically shape one another in everyday conversation, adapting to accents, talkers, and social-linguistic associations in ways that rigid laboratory tasks cannot capture. The classic observer’s paradox, articulated by William Labov in 1972, compounds the problem: people who know they are being recorded often adjust their speech, consciously or not, making genuinely naturalistic samples scarce and difficult to obtain.

The logistical hurdles do not end there. Field recordings demand extensive preparation, community relationships, and careful attention to the researcher’s positionality. Reading tasks systematically exclude speakers — such as many heritage bilinguals — who are conversationally fluent but may lack literacy in the language being studied. Funding constraints make it increasingly difficult to build and share open-access speech corpora, a burden that falls hardest on graduate students, early-career scholars, and researchers working on minoritized language communities. The authors argue that these barriers do not merely slow research down; they distort the theoretical record, because dominant models of speech perception and production were built on idealized, read speech and often treat real-world linguistic diversity as noise rather than as the phenomenon itself.

TAPA’s answer is to stitch together a chain of proven open-source components into one Python pipeline that can be installed with a single command and run either locally or in Google Colab, where free GPU access removes the need for any local setup. The journey begins with a YouTube audio downloader built on yt-dlp, which pulls the audio stream from any standard video URL and converts it to an MP3 file; users can equally supply their own local audio. Transcription is handled by OpenAI’s Whisper, which produces word-level transcripts with precise start and end timestamps. Speaker diarization — deciding who spoke when — relies on two pretrained neural models: Silero VAD, which detects stretches of actual speech, and Resemblyzer, which maps each detected segment to a 256-dimensional voice embedding in which clips from the same speaker cluster together. Those embeddings are then grouped to assign every transcribed word to its speaker.

From there, the pipeline turns to fine-grained phonetic measurement. The Montreal Forced Aligner produces phone-level boundaries linking the audio to the transcript, using an American English acoustic model with a dictionary-based proportional-timing fallback when alignment is unavailable. Vowel formants — the resonant frequencies F1 and F2 that define vowel identity — are extracted with Praat-parselmouth, using the Burg algorithm over the central portion of each vowel after trimming the edges to suppress coarticulation, with the maximum formant ceiling adjusted adaptively to each speaker’s pitch. Stop consonant voice onset time, the interval between a stop’s burst release and the onset of voicing, is measured by Dr. VOT, a neural model that predicts burst and voicing boundaries directly from the waveform. Fricative spectral moments — center of gravity, standard deviation, skewness, and kurtosis — are computed over the middle 80 percent of each fricative interval. The output is a set of CSV or JSON files containing per-token measurements and per-speaker summaries.

To demonstrate what the pipeline can do, the team pointed it at a demanding piece of real-world audio: the 90-minute 2016 U.S. presidential debate between Hillary Clinton and Donald Trump, complete with audience noise, podium reverberation, overlapping talkers, and the performative speech style of political theater. From that single recording, TAPA extracted nearly 33,000 acoustic segments — 19,855 vowels, 6,547 stops, and 6,372 fricatives. The case study focused on the two main candidates, yielding more than 7,000 monophthong tokens each for vowel-space analysis, alongside thousands of stop and fricative tokens per speaker. Plotting the F1-by-F2 vowel spaces revealed that Clinton’s vowel categories occupied a larger acoustic area than Trump’s, and comparing both speakers’ category means against the classic Hillenbrand reference norms showed the expected centralization of vowels in continuous speech relative to citation-form productions.

Crucially, the team did not simply trust the automated output. A phonetically trained research assistant hand-coded stratified samples of vowels, stops, and fricatives from the debate audio, blind to the pipeline’s results, and the two sets of measurements were compared token by token. Vowel formants emerged as the pipeline’s strongest suit: agreement with expert measurements was high, with correlations of 0.89 for F1 and 0.87 for F2, mean absolute errors of 37 and 125 hertz respectively, and no meaningful bias for F1. Fricative measurements showed a more differentiated picture. Spectral standard deviation agreed strongly with hand coding overall, and center of gravity tracked the human reference closely for the sibilant fricatives — correlations of 0.87 for /s/ and 0.94 for /ʃ/ — while the spectrally indistinct non-sibilants /f/ and /θ/ agreed far more weakly, a pattern consistent with long-standing findings that these sounds are hard to classify from spectral moments alone. Clinton’s /s/ center of gravity sat roughly 900 hertz above Trump’s, echoing documented patterns of gender-linked variation in sibilant production.

Stop consonants told a more cautionary tale. Although the aggregate voice-onset-time means for voiceless stops /p/, /t/, and /k/ landed squarely in the expected American English long-lag range of roughly 69 to 83 milliseconds, the per-token picture was far less reassuring. The Dr. VOT model, which was trained on isolated single-word productions with stop classes artificially rebalanced to a 50:50 positive-to-negative ratio, mislabeled 45 to 61 percent of voiceless onset stops as prevoiced — an outcome incompatible with English phonology — and its per-token magnitudes correlated essentially not at all with expert measurements (r = −0.04), diverging by 50 to 80 milliseconds on average. The authors attribute this to a training–deployment mismatch: continuous, noisy debate audio with a natural 95:5 ratio of positive to negative stops violates both of the model’s core training assumptions. Rather than hide the problem, they report it transparently, omit voiced stops from the case study entirely, and recommend that aggregate means be treated as descriptive rather than per-token measurements.

That candor reflects the paper’s broader stance on automation in science. The authors are explicit that TAPA is meant to aid researchers, not replace them: acoustic training must continue, research must be deployed and interpreted by humans, and the energy costs of training and fine-tuning large models warrant critical scrutiny. The pipeline inherits the biases of its components — the default configuration is American English-centric, diarization degrades on heavily overlapping speech, and forced alignment stumbles on out-of-vocabulary names — so the team recommends that every substantive deployment include a small hand-coded validation subsample from the researcher’s own corpus, and that publications report which component produced each measurement. With those safeguards in place, TAPA offers something the field has lacked: a free, modular, inspectable path from a web video or field recording to speaker-level, phone-level acoustic data at a scale that would once have required years of manual annotation, potentially opening naturalistic speech research to communities and questions that laboratory methods have long left on the periphery.

Subject of Research: An open-source computational pipeline for automated acoustic-phonetic analysis of naturalistic multi-speaker speech recordings

Article Title: Toolkit for acoustic–phonetic analysis of naturalistic speech data

Article References: Kutlu, E., Peters, E., Tapanes, C., Chandio, S., & Khalid, O. (2026). Toolkit for acoustic–phonetic analysis of naturalistic speech data. Behavior Research Methods, 58(10), Article 289. https://doi.org/10.3758/s13428-026-03174-y

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03174-y

Keywords: acoustic-phonetic analysis, naturalistic speech, speaker diarization, forced alignment, vowel formants, voice onset time, fricative spectral moments, open-source software, speech perception, sociophonetics, automatic speech recognition, Behavior Research Methods