Why OpenAI Whisper Fails on Amharic: Token Fragmentation & the 8.4% WER Solution

OpenAI's Whisper is widely celebrated as the default foundation model for speech-to-text. Across English, Spanish, and standard European languages, it regularly achieves sub-5% Word Error Rates.

Yet, when developers and researchers deploy Whisper Large-v3 on Amharic (አማርኛ), the results are often baffling: broken sentences, sudden bursts of Latin transliteration, missed homophones, and catastrophic failure on conversational audio.

In our empirical evaluations across three acoustic splits and four regional dialects, Whisper Large-v3 registered an average 22.8% WER on clean studio audio and ballooned to 41.6% WER on mobile field recordings.

This is not simply a matter of "more training data." It is the direct consequence of five fundamental architectural and linguistic friction points that foundation models overlook.

The 5 Core Failure Modes of Foundation Models on Ge'ez Audio

1. Byte-Pair Encoding (BPE) Token Fragmentation

Whisper uses a Byte-Pair Encoding tokenizer with a vocabulary optimized overwhelmingly for Latin scripts. In UTF-8, each Ge'ez fidel character (such as "አ", "በ", "ት") is encoded using 3 bytes.

Because Amharic was represented sparsely in Whisper's initial pre-training corpus, the tokenizer does not contain full word or syllable tokens for most Ge'ez combinations. Instead, the model is forced to predict Ge'ez text byte-by-byte. When the acoustic encoder experiences minor uncertainty—such as a speaker pausing or background street noise—the autoregressive decoder loses its sequence alignment and begins predicting high-probability Latin byte sequences. This is the root cause of Whisper's notorious "Latin transliteration hallucinations" on Ethiopian audio.

2. Unwritten Consonant Gemination (ሸረት)

In the Ge'ez writing system, consonant doubling (gemination, known locally as ሸረት) is completely unwritten. This represents a profound linguistic challenge:

An acoustic model that lacks deep morphological priors will misinterpret the acoustic duration of the consonant, leading to severe downstream grammatical errors in transcript interpretation. Andebet resolves this by incorporating a morphologically informed neural rescorer trained specifically to disambiguate gemination with 98.4% contextual precision.

3. The Ge'ez Homophone Disambiguation Problem

Classical Ethiopic orthography contains multiple distinct letters that represent identical or near-identical phonetic sounds in modern spoken Amharic:

• The "H" triplet: ሀ (Hahu), ሐ (Hamer), and ኀ (Harm)
• The "S" pair: ሰ (Nigus) and ሠ (Esat)
• The "Ts" pair: ጸ (Tselot) and ፀ (Tsehaye)
• The "A" pair: አ (Alef) and ዐ (Ayn)

Foundation models pick whichever character appeared most frequently in their scattered web scrapes, creating inconsistent, unprofessional transcripts within the exact same document. Andebet applies context-aware orthographic normalization that standardizes spelling according to contemporary Ethiopian Ministry of Education guidelines while retaining liturgical forms in appropriate religious contexts.

4. Colloquial Bilingual Code-Switching

In urban Ethiopian media, business meetings, and tech podcasts, speakers constantly switch mid-sentence between Amharic and English:

When fed into Whisper or Google Cloud Speech, the models lock into single-language mode: either the English words are transliterated phonetically into unreadable fidel characters, or the Amharic is completely dropped. Andebet was trained specifically on multilingual conversational datasets to maintain native dual-script fidelity.

5. Cellular 8kHz Compression and Mobile Memos

Global models assume clean 16kHz audio captured by studio desktop microphones. In reality, the vast majority of Ethiopian audio is recorded via Telegram voice notes, WhatsApp voice memos, or telephone interviews over Ethio Telecom's network.

Under heavy 8kHz AMR/Opus compression, Whisper's error rate spikes to 48.3%. Andebet incorporates specialized acoustic downsampling augmentation, ensuring robust transcription even on low-bandwidth field audio.

Empirical Benchmark Data (2026)

Below are the empirical Word Error Rate (WER) and Character Error Rate (CER) results measured across 120 hours of test audio across three acoustic evaluation splits: