New Open-Source Toolkit Turns Any Speech Recording Into Acoustic-Phonetic Data

For decades, the speech sciences have faced an uncomfortable paradox: some of the most important questions about how humans actually talk can only be answered with natural, spontaneous speech, yet the data that researchers most easily collect and analyze is almost always carefully scripted laboratory audio. A newly published open-source pipeline called the Toolkit for Acoustic–Phonetic Analysis, or TAPA, promises to change that balance. In a paper in Behavior Research Methods, a team led by Ethan Kutlu of the University of Iowa describes a single integrated system that takes raw audio from almost any source — a YouTube video, a podcast, an interview recording made in the field — and converts it into speaker-labeled, phonetically aligned, acoustically measured data, all without requiring users to write custom code.

The motivation behind the toolkit is rooted in a long-standing tension within the field. Laboratory speech, produced in word-reading or sentence-repetition tasks, gives experimenters precise control over variables and supports repeatability, which is why it has anchored theoretical work in speech perception and production for generations. But an accumulating body of evidence shows that listeners and speakers dynamically shape one another in everyday conversation, adapting to accents, talkers, and social-linguistic associations in ways that rigid laboratory tasks cannot capture. The classic observer’s paradox, articulated by William Labov in 1972, compounds the problem: people who know they are being recorded often adjust their speech, consciously or not, making genuinely naturalistic samples scarce and difficult to obtain.

The logistical hurdles do not end there. Field recordings demand extensive preparation, community relationships, and careful attention to the researcher’s positionality. Reading tasks systematically exclude speakers — such as many heritage bilinguals — who are conversationally fluent but may lack literacy in the language being studied. Funding constraints make it increasingly difficult to build and share open-access speech corpora, a burden that falls hardest on graduate students, early-career scholars, and researchers working on minoritized language communities. The authors argue that these barriers do not merely slow research down; they distort the theoretical record, because dominant models of speech perception and production were built on idealized, read speech and often treat real-world linguistic diversity as noise rather than as the phenomenon itself.

TAPA’s answer is to stitch together a chain of proven open-source components into one Python pipeline that can be installed with a single command and run either locally or in Google Colab, where free GPU access removes the need for any local setup. The journey begins with a YouTube audio downloader built on yt-dlp, which pulls the audio stream from any standard video URL and converts it to an MP3 file; users can equally supply their own local audio. Transcription is handled by OpenAI’s Whisper, which produces word-level transcripts with precise start and end timestamps. Speaker diarization — deciding who spoke when — relies on two pretrained neural models: Silero VAD, which detects stretches of actual speech, and Resemblyzer, which maps each detected segment to a 256-dimensional voice embedding in which clips from the same speaker cluster together. Those embeddings are then grouped to assign every transcribed word to its speaker.

From there, the pipeline turns to fine-grained phonetic measurement. The Montreal Forced Aligner produces phone-level boundaries linking the audio to the transcript, using an American English acoustic model with a dictionary-based proportional-timing fallback when alignment is unavailable. Vowel formants — the resonant frequencies F1 and F2 that define vowel identity — are extracted with Praat-parselmouth, using the Burg algorithm over the central portion of each vowel after trimming the edges to suppress coarticulation, with the maximum formant ceiling adjusted adaptively to each speaker’s pitch. Stop consonant voice onset time, the interval between a stop’s burst release and the onset of voicing, is measured by Dr. VOT, a neural model that predicts burst and voicing boundaries directly from the waveform. Fricative spectral moments — center of gravity, standard deviation, skewness, and kurtosis — are computed over the middle 80 percent of each fricative interval. The output is a set of CSV or JSON files containing per-token measurements and per-speaker summaries.

To demonstrate what the pipeline can do, the team pointed it at a demanding piece of real-world audio: the 90-minute 2016 U.S. presidential debate between Hillary Clinton and Donald Trump, complete with audience noise, podium reverberation, overlapping talkers, and the performative speech style of political theater. From that single recording, TAPA extracted nearly 33,000 acoustic segments — 19,855 vowels, 6,547 stops, and 6,372 fricatives. The case study focused on the two main candidates, yielding more than 7,000 monophthong tokens each for vowel-space analysis, alongside thousands of stop and fricative tokens per speaker. Plotting the F1-by-F2 vowel spaces revealed that Clinton’s vowel categories occupied a larger acoustic area than Trump’s, and comparing both speakers’ category means against the classic Hillenbrand reference norms showed the expected centralization of vowels in continuous speech relative to citation-form productions.

Crucially, the team did not simply trust the automated output. A phonetically trained research assistant hand-coded stratified samples of vowels, stops, and fricatives from the debate audio, blind to the pipeline’s results, and the two sets of measurements were compared token by token. Vowel formants emerged as the pipeline’s strongest suit: agreement with expert measurements was high, with correlations of 0.89 for F1 and 0.87 for F2, mean absolute errors of 37 and 125 hertz respectively, and no meaningful bias for F1. Fricative measurements showed a more differentiated picture. Spectral standard deviation agreed strongly with hand coding overall, and center of gravity tracked the human reference closely for the sibilant fricatives — correlations of 0.87 for /s/ and 0.94 for /ʃ/ — while the spectrally indistinct non-sibilants /f/ and /θ/ agreed far more weakly, a pattern consistent with long-standing findings that these sounds are hard to classify from spectral moments alone. Clinton’s /s/ center of gravity sat roughly 900 hertz above Trump’s, echoing documented patterns of gender-linked variation in sibilant production.

Stop consonants told a more cautionary tale. Although the aggregate voice-onset-time means for voiceless stops /p/, /t/, and /k/ landed squarely in the expected American English long-lag range of roughly 69 to 83 milliseconds, the per-token picture was far less reassuring. The Dr. VOT model, which was trained on isolated single-word productions with stop classes artificially rebalanced to a 50:50 positive-to-negative ratio, mislabeled 45 to 61 percent of voiceless onset stops as prevoiced — an outcome incompatible with English phonology — and its per-token magnitudes correlated essentially not at all with expert measurements (r = −0.04), diverging by 50 to 80 milliseconds on average. The authors attribute this to a training–deployment mismatch: continuous, noisy debate audio with a natural 95:5 ratio of positive to negative stops violates both of the model’s core training assumptions. Rather than hide the problem, they report it transparently, omit voiced stops from the case study entirely, and recommend that aggregate means be treated as descriptive rather than per-token measurements.

That candor reflects the paper’s broader stance on automation in science. The authors are explicit that TAPA is meant to aid researchers, not replace them: acoustic training must continue, research must be deployed and interpreted by humans, and the energy costs of training and fine-tuning large models warrant critical scrutiny. The pipeline inherits the biases of its components — the default configuration is American English-centric, diarization degrades on heavily overlapping speech, and forced alignment stumbles on out-of-vocabulary names — so the team recommends that every substantive deployment include a small hand-coded validation subsample from the researcher’s own corpus, and that publications report which component produced each measurement. With those safeguards in place, TAPA offers something the field has lacked: a free, modular, inspectable path from a web video or field recording to speaker-level, phone-level acoustic data at a scale that would once have required years of manual annotation, potentially opening naturalistic speech research to communities and questions that laboratory methods have long left on the periphery.

Subject of Research: An open-source computational pipeline for automated acoustic-phonetic analysis of naturalistic multi-speaker speech recordings

Article Title: Toolkit for acoustic–phonetic analysis of naturalistic speech data

Article References: Kutlu, E., Peters, E., Tapanes, C., Chandio, S., & Khalid, O. (2026). Toolkit for acoustic–phonetic analysis of naturalistic speech data. Behavior Research Methods, 58(10), Article 289. https://doi.org/10.3758/s13428-026-03174-y

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03174-y

Keywords: acoustic-phonetic analysis, naturalistic speech, speaker diarization, forced alignment, vowel formants, voice onset time, fricative spectral moments, open-source software, speech perception, sociophonetics, automatic speech recognition, Behavior Research Methods