Kurdish AI: teaching computers a low-resource language

Artificial intelligence has learned to listen, speak and translate in many languages, and Kurdish is still catching up. It is spoken by an estimated 20 to 30 million people, but far less of it has been recorded, typed and labelled in ways a computer can learn from. This guide explains, in plain words, what that means and what it takes to build Kurdish AI.

What “low-resource” means

AI systems learn from examples. A speech recognizer learns from many hours of recordings, each with a matching written transcript. A translation system learns from large numbers of sentences paired with their translations.

Researchers call a language low-resource when this kind of data is scarce. The term is mainly about data, not the number of speakers: a language can have millions of speakers and still be low-resource. Kurdish is a clear example, and studies of Kurdish language technology have described it as less-resourced for more than a decade.

The data that matters most is labelled data: recordings with transcripts, sentences with translations, words tagged with their meaning. Plain text from the internet helps too, but labelled data is what teaches a system a specific job, and making it takes careful human work.

Why Kurdish is hard for AI

Kurdish brings four challenges at once: several dialects, two main scripts, spelling variation and very little labelled data.

Several dialects

Kurmancî (Northern Kurdish) and Soranî (Central Kurdish) are the two largest varieties, and Southern Kurdish is a third group. They differ in sounds, words and grammar, so a system trained on one does not automatically understand the other.

Look at “I am fine”. Kurmancî uses three words, Ez baş im, while Soranî packs the same meaning into a single word. A computer has to learn both patterns separately. Read more in Kurmancî vs Soranî.

Two main scripts

Kurmancî is mostly written in Latin letters, and Soranî in an Arabic-based script that runs from right to left. In the former Soviet Union, Kurmancî was also written in Cyrillic. To a computer, the two ways of writing “hello” in the table above have nothing in common, even though a Kurdish listener hears almost the same word. Our Kurdish alphabet guide shows both scripts letter by letter.

Spelling variation

Many everyday words are written in more than one way. Here are pairs from our own lessons that a computer must learn to treat as the same word:

There is a hidden layer, too. People type Kurdish on many different keyboards, including ones made for other languages, and these can store letters that look identical as different characters. One Soranî letter can reach a computer in several forms, so text has to be cleaned before an AI can learn from it. Our Kurdish keyboard guide shows how to type every letter.

Little labelled speech and text

Kurdish speech recognition is a young field. In 2019, researchers presenting one of the first Soranî speech datasets wrote that they could not find any earlier studies on Kurdish speech recognition. Larger speech collections for Soranî and Kurmancî have been built since then, but they are still modest in size.

Translation has the same problem. Studies on Kurdish machine translation describe the supply of Kurdish text paired with its translation as scarce.

Speech recognition, text-to-speech and translation

These three are often lumped together as “Kurdish AI”, but each one does a different job. Take the simplest Kurdish word:

Silav! /siˈlav/ سڵاو! silaw Hello!

• Speech recognition turns speech into text. You say Silav out loud, and the computer writes it down.
• Text-to-speech goes the other way. It reads written Kurdish aloud in a computer voice.
• Machine translation turns text in one language into another. It reads Silav and gives you “Hello”.

Pronunciation feedback builds on speech recognition. Instead of only writing down what you said, the system compares your Silav with the word you were asked to say and points out the sounds that need work.

Each job needs different data. Speech recognition has to understand everyone, so it learns best from many voices: young and old, every accent, quiet rooms and noisy ones. Text-to-speech has to know how every written word should sound. Here Kurdish has an advantage, because both alphabets mostly write one letter for each sound. Soranî also writes almost all of its vowels as letters, while Arabic usually leaves short vowels out. The one exception is a short “i”, like the first vowel of سڵاو, which is not written.

Our own Kurdish speech recognition and voices

ZimanGo builds its own Kurdish speech recognition for Kurmancî and Soranî. In speaking exercises you say a word or sentence out loud, and it checks your pronunciation and shows you what to fix.

The audio in every lesson comes from our own Kurdish text-to-speech voices. They read the same sentences you see on screen, and those sentences are written by Kurdish speakers: the founder writes the Kurmancî lessons, and Soranî speakers write the Soranî ones.

Building for Kurdish from the start means handling both scripts, the sounds of each dialect and the spelling variation described above. It is slow, careful work, and recordings that learners choose to share help it improve.

How learners can help, only if you say yes

The best way to make Kurdish speech recognition better is more real Kurdish voices, from every region and accent. That is why ZimanGo asks learners whether their speaking-exercise recordings may be used to improve it.

• Your choice. Your recordings help improve our speech recognition only with your explicit consent. You can say no, or change your mind at any time, and still use every speaking exercise.
• Kept for a limited time. Recordings are kept for up to 24 months after they were made, or until you delete them or your account, whichever comes first. You can ask us to delete them at any time.
• Never misused. We never use your voice for voice cloning, for advertising or to identify you.

Speaking exercises are always optional, so you can also skip them. The full details are in our privacy policy.

Kurdish words for the digital world

If you are learning Kurdish, the vocabulary of technology is a good place to see the language at work. These words come from our lessons, most of them from the one on technology and modern life:

Some new things have more than one name. In the lessons, “app” appears both as sepan and as Aplîkasyon, which is exactly the kind of variety an AI has to learn.

Put a few words together and you can talk about your day. Ez li ser komputerê dixebitim means “I work on the computer”. In Soranî you say لەسەر کۆمپیوتەر کار دەکەم.

What comes next

Kurmancî and Soranî are available today. Behdînî (Badini), Zazakî (Kirmanckî) and Hewramî (Gorani) are coming soon. Bringing each one in will take more recordings, more voices and careful checking by people who speak it. Our guide to Kurdish dialects explains how they fit together.

Varieties with fewer written texts and recordings start from even less data, so every carefully written sentence and every shared recording counts. Progress is steady rather than sudden.