Shruti, 22 small speech-to-text models that run on your device

Shruti is now 22 speech-to-text models, one for each of the 22 languages of India's Eighth Schedule, the mother tongue of 117 crore people (Census 2011, grouped). Each is one small file, 29.7 MB (29.5 MB for Hindi), and each also handles English. It works offline: on your phone, your laptop or in a browser, and your voice never leaves your device.

Building for these languages has usually meant a cloud bill by the minute, a model in gigabytes, or a big company's permission. We want any developer to be able to add them to an app with no key, no account and no server. Shruti is a file, under Apache-2.0, so if we vanish tomorrow your app keeps working.

Many people mix English into what they say. Shruti tries to write that down the way they would type it, in their own script with English words in English:

• Hindi: मेरा order कब आएगा
• Marathi: उद्या सकाळी नऊ वाजता physics ची exam आहे
• Tamil: நாளைக்கு காலையில ஒன்பது மணிக்கு physics exam இருக்கு
• Bengali: বাচ্চাদের রবীন্দ্রসঙ্গীতের rehearsal শনিবার বিকেল চারটে হবে
• Telugu: హైదరాబాద్‍లో కొత్త company నుంచి offer letter వచ్చింది

Try it on your own device, with no sign-in. The full numbers are in the technical report.

What's in the family

Hindi, Marathi, Tamil, Bengali, Telugu, Gujarati, Odia, Kannada, Urdu, Malayalam, Assamese, Sanskrit, Punjabi, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Nepali, Santali and Sindhi. Eight of them (Odia, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri and Santali) aren't on OpenAI Whisper's language list, read on the release date. IndicConformer covers all 22.

Each model covers one language plus English, so you pick the file you need. All 22 are built on Cactus Whistle, an open speech model from Cactus Compute. We added vocabulary for each language and fine-tuned all the weights on public speech datasets. They run in Cactus's open-source needle engine. The weights are open, under Apache-2.0.

On a five-year-old phone with 6 GB of RAM, in the native engine, the 21 newer models run between 8.6 and 12.6 times faster than real time, in 64 MB of memory. Shruti Hindi was measured another way: a 60-second recording in 9.5 seconds, in 80 MB. All 22 invented no words on 100 silent clips. That test covers silence only, not noise or music.

Add it to your app

Download one language's file and transcribe. There's no key and no sign-in:

# pip install cactus-needle
# hf download udgamlabs/shruti-tamil shruti-tamil.cact --local-dir .
from needle import Whistle

model = Whistle("shruti-tamil.cact")
print(model.transcribe("clip.wav")["text"])

The Python package sends anonymous usage counts by default; set NEEDLE_TELEMETRY=0 to turn that off. The command-line engine and the browser build make no network calls. In a phone app, you link the engine's C library and ship the file with the app. Section 10 of the report has the steps for each route, and the GitHub repository has examples you can run.

Prefer ONNX Runtime, the browser or sherpa-onnx? All 22 languages also come as ONNX packs, a 32 MB or 53 MB download, under Apache-2.0: pip install shruti-onnx, npm i shruti-onnx onnxruntime-web, or udgamlabs/Shruti-ONNX. On a phone the needle engine is faster and lighter, and every score on this page is the engine's.

We tested each pack against the engine under six written rules. None passed all six: 13 languages came out mixed and 9 failed, Hindi among them. A failed rule isn't always worse, since a pack can miss by scoring better. Section 10 of the report gives each language's result and says which misses were worse.

How good is it?

It depends on the language, so we won't give you one number for the family.

We tested each language on the same clips and with the same scorer for every model, measured as word error rate, where lower is better. The test that matters most for Shruti is Indic DiarBench, from Sarvam AI and AI4Bharat (paper, CC BY 4.0): real conversations, kept out of every model's training. We score it two ways, because the answer changes:

• English in Latin: English words count as right only if they are written in English letters. That's what Shruti is built to do, and it's where its leads appear.
• Native: English words are expected in the language's own script. Here most larger rivals pull ahead.

Here is where each language stands, with the size of the rival next to it. These are DiarBench scores with English in Latin, except where a row says otherwise.

For seven languages, we make no comparison we'd stand behind:

• Malayalam: 56.8 against IndicConformer Malayalam's 61.7, but that lead doesn't hold when we resample whole conversations or ignore script. Over half of words are wrong.
• Sanskrit: IndicConformer Sanskrit is ahead on the larger test (30.9 vs 24.4), and our conversation test has only 55 turns. This is spoken Sanskrit. We haven't tested chant or recitation.
• Dogri: our conversation test has only 24 turns, too few to say anything.
• Kashmiri: its one lead (43.0 vs IndicConformer Kashmiri's 48.7) is on a test that shares a source with our training data, and our independent test has only 32 turns, with no English in it.
• Maithili: IndicConformer Maithili is ahead or level on every test.
• Nepali: IndicConformer Nepali is well ahead on read speech (46.0 vs 29.9).
• Sindhi: it writes Devanagari, not Perso-Arabic, and its conversation test is only 27 turns.

Shruti Hindi is a different case: its file is about 100 times smaller than Whisper large-v3 (3,087 MB), with fewer errors on our Hindi tests (Monsoon 20.46 vs 33.29, FLEURS 20.14 vs 28.88, Lahaja 22.87 vs 31.19, on the same clips). Whisper large-v3 is far better on English (LibriSpeech 2.25 vs 5.76, on the same clips). On DiarBench, with English in Latin, Shruti Hindi scores 26.0 against SraVaani-1.0's 30.7 and IndicConformer Hindi's 39.1, and that lead over SraVaani-1.0 goes when script is ignored.

Where it falls short

• Read speech. On most languages, larger models are clearly more accurate on read speech. On FLEURS, for example, IndicConformer Punjabi scores 16.9 against Shruti Punjabi's 29.3, and IndicConformer Odia 25.9 against 39.5.
• The native reference. When English is expected in the language's own script, IndicConformer or SraVaani-1.0 is ahead on most languages. Our leads depend on writing English in English letters.
• High error rates. On real conversation, over half of words are wrong in Kannada, Malayalam and Tamil, even on the scoring that suits Shruti.
• Thin tests. Eleven languages have conversation tests under 100 turns, and five of those are too small to carry a claim. For Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Santali and Sindhi we measured one serious rival. Other models exist for many of these languages, and we haven't measured them.
• One script each. Sindhi and Konkani are Devanagari only, Manipuri is Meetei Mayek only, Santali is Ol Chiki only, and Kashmiri is Perso-Arabic only.
• Language choice. Left to choose by itself, the engine often treats Indian-accented English as Hindi, so set English when speech is entirely English. On Nepali, it sent 20 of 200 read clips wholly to English letters, so set Nepali when you know it's Nepali. That moves the Nepali FLEURS score from 46.0 to 42.2, which still leaves it behind.
• Plain English on our try box. The try box sends the language you pick. That keeps speech in the right script, and it costs some accuracy on plain English.
• What we haven't tested. Noise, music, far-away or speaker-played audio on these languages, a live microphone on most of them, and Linux, Windows and iOS for the 21 newer models. In loud noise, Shruti models sometimes repeat a phrase over and over (measured on Hindi-family models). There's no punctuation, and numbers come out as words.
• No speaker has checked most of this. Speakers of the 16 newer languages haven't read these transcripts. If you speak one of them, we'd like to know what you find.
• Data licences. Apache-2.0 covers our models and doesn't set any dataset's terms. Two sets behind Shruti Assamese, SPRING-INX Assamese R1 and R2, name no licence on their pages. The technical report lists every source.

Try it

• On your own device, in your browser
• The Hugging Face demo and the Shruti collection
• Code and examples on GitHub
• The technical report, with every language's table, intervals and limits

Shruti by Udgam Labs

Udgam Labs · AI research from first principles