There is a common assumption in language technology that a language is the right unit of coverage. A model either supports Somali or it does not.
For speech recognition this assumption is wrong, and it is wrong in ways that show up immediately when you test a system on real audio from a specific place.
The dialect landscape
Somali is conventionally divided into three broad dialect groups: Northern Somali, on which the standard written form is based; Benaadir, spoken along the central coast including Mogadishu; and Maay, spoken in the southern interior and different enough that mutual intelligibility with the standard is limited.
Within Northern Somali there is further variation, and the Somali spoken in Djibouti sits within that range while carrying its own characteristics. Djiboutian Somali has absorbed vocabulary from French and Arabic through a distinct colonial and administrative history. It sits alongside Afar in a bilingual environment that shapes pronunciation. Its broadcast register — the Somali of radio and television news — has its own conventions developed over decades of national broadcasting.
None of this makes Djiboutian Somali a separate language. Speakers understand each other across the region without difficulty. But automatic speech recognition is considerably less flexible than a human listener.
What a model actually learns
A speech recognition model learns a mapping from acoustic patterns to text. That mapping is built entirely from the examples it has seen.
When a model is trained predominantly on standard Somali read speech — which describes most of the publicly available Somali data — it learns the acoustic realisations present in that data. Presented with a pronunciation it has not encountered, or a loanword outside its vocabulary, or a rhythm characteristic of a different region, it does not gracefully approximate. It produces the nearest thing it knows, which can be quite far away.
This is why aggregate word error rates on a language can be misleading. A system reported as achieving reasonable performance on Somali may have been evaluated on data from one dialect region and one speech register, and may perform substantially worse on another.
Domain compounds dialect
Dialect is one axis. Domain is a second, and the two multiply.
Most available Somali speech data is read speech — someone reading prepared text in a quiet environment. Broadcast audio is different in almost every respect. Speakers talk at natural pace with natural disfluency. There are multiple speakers, sometimes overlapping. There is background music, room tone, telephone-quality remote audio, and the acoustic signature of whatever equipment recorded it, which for archival material may be several decades old.
A model trained on clean read standard Somali and evaluated on Djiboutian broadcast audio is being asked to generalise across dialect and domain simultaneously. Performance degrades on both axes.
What this implies for solving it
The implication is fairly direct: the training data has to match the dialect and the domain where the system will be used.
This is not a novel finding in speech recognition — domain match has been understood as critical for decades. It is worth restating in this context because it changes what the actual bottleneck is.
The bottleneck is not model architecture. Fine-tuning a small existing model on well-matched data outperforms a much larger model on mismatched data, which we observed directly: a 244-million-parameter model fine-tuned for an hour beat a system many times its size that had never seen comparable data.
The bottleneck is the data. Specifically, it is transcribed audio from the dialect and domain in question, produced by native speakers who can hear the distinctions that matter.
Why broadcast archives are the right source
For a dialect-and-domain matched corpus, broadcast archives are close to ideal, for reasons that are worth spelling out.
They are large. National broadcasters produce material daily over decades, which accumulates into a volume no purpose-built collection effort could match on a reasonable budget.
They are naturally domain-matched to a wide range of uses. Broadcast speech includes prepared news reading, spontaneous interview, discussion, and public address — a spread of registers rather than a single narrow one.
They are dialectally coherent. A national broadcaster in Djibouti produces Djiboutian Somali, consistently, because that is who its presenters are and who its audience is.
And they are culturally significant independently of their technical utility. The same archive that would train a speech recognition system is also a record of a country’s public life, which is a reason to work with it carefully rather than merely extractively.
The unit of the problem
The practical conclusion is that “does this system support Somali” is the wrong question. The right question is whether it has been trained and evaluated on the specific variety the user actually speaks, in conditions resembling how they will actually use it.
For most of the world’s languages, and for most dialects within those languages, the honest answer is no. That is the gap we are working in.
Our evaluation methodology and results are published as SASR Eval.
See our full methodology and per-clip results, or get in touch about dialect-matched Somali data.