Vemana
A 278M Telugu base model, trained from scratch and live in public in about 16 hours.
On fresh Telugu text, it is the best open base model under 1B parameters that we tested (bits per byte, October 2026).
Vemana is a small Telugu language model by Udgam Labs, with 278 million parameters and its own Telugu tokenizer. It is a base model: give it the start of a Telugu text and it carries on writing. It runs on a laptop-class CPU. We trained it from scratch on one H100 GPU in about 16 hours, for about $69 of GPU time, with the run live in public.
It is not the best Telugu model. Sarvam-1, about 9× larger, is level with it or better, and on knowledge tests Vemana is at chance. It does not answer questions or follow requests.
Try it now · in your browser · free, no login
Start a Telugu sentence and let Vemana continue it, on your own device.
Vemana runs entirely inside this tab: the 297 MB model downloads once and is kept by your browser, and your text never leaves your device. Give it the start of some Telugu text and it writes on.
Works best on a laptop or a recent phone with a good connection.
The text appears here.
Vemana is a base model: it continues text, does not answer questions or follow requests, and makes things up, so check anything that matters. Speed depends on your device.
Or run it yourself with transformers 5.19 or newer: keep the prompt and the output within 1,024 tokens, about 570 Telugu words. On a Mac mini’s CPU it writes about 65 tokens a second. For llama.cpp, use the GGUF files.
pip install "transformers>=5.19" torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Base")
inputs = tok("తెలుగు భాష", return_tensors="pt") # the tokenizer adds <s>
out = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.8, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))The result
Ahead of every other open base model under 1B that we tested, on fresh Telugu text.
On 416 Telugu Wikipedia pages created in September and October 2026, after the compared models were released, Vemana scores 0.361 bits per byte, against 0.383 for the next best, nanochat-telugu-560M, twice its size (paired 95% interval of the gap −0.024 to −0.021; lower is better). It also leads on 477 pages created in July and August and on 29 pages created 23–26 October. Same scoring script and pages for every model; measured by us on 27 October 2026.
We wrote the test, the winning margin and the checks down before training, and chose the final checkpoint as the release model in advance. A larger model is level or better: Sarvam-1 (2.5B), in the chart and under where it falls short.
- Parameters
- 278M
- 278,214,144. Telugu only; a base model that continues text
- Training
- ≈16 h
- One H100 GPU, live in public, about $69 of GPU time
- Tokens read
- 12.0B
- 5.1B unique tokens of public Telugu web text, read about 2.4 times
- Tokenizer
- 38–43%
- Fewer tokens per Telugu word than Gemma 3’s, from its own 32,000-token Telugu tokenizer
What it writes
Two continuations, sampled with temperature 0.8 and top-p 0.95. The prompt is in bold; the English translations are ours.
వర్షాకాలంలో సీజనల్ వ్యాధులు ఎక్కువగా వస్తాయి. అందువల్ల ఆరోగ్యంగా ఉండటం చాలా ముఖ్యం. మీరు వర్షాకాలంలో డెంగ్యూ, మలేరియా, టైఫాయిడ్, వైరల్ ఇన్ఫెక్షన్ల వంటి సీజనల్ వ్యాధుల బారిన పడే ప్రమాదం ఉంది. …కంప్యూటర్ అనేది మనకు అత్యంత అవసరమైన వస్తువులలో ఒకటి. కంప్యూటర్లు మనం చేసే చాలా పనులను చేస్తాయి, … ఈ సాధనాలలో ఒకటి CPU. …How it compares
Bits per byte measures how well a model predicts text it has never seen: the bits it needs for a page, divided by the page’s bytes. Lower is better, and models with different tokenizers can be compared fairly.
To be sure no model had seen the text, we built three sets of Telugu Wikipedia pages created after the compared models were released, with one builder and one set of rules: new pages only, no redirects, lists or bot pages, at least 120 words of prose and at least 85% Telugu script. No page is in two sets. One model, Dhee-Pravega-OG Atom, was released on 8 October without saying where its data ends, so the older pages could be in its training data, which could only help it. The 23–26 October pages are newer than every model.
Every gap comes with a paired 95% interval: we resample the pages 4,000 times and compare the two models on the same pages each time. A negative gap means Vemana is better.
| Model | Params | Sep–Oct 2026 (416 pages) | Jul–Aug 2026 (477 pages) | 23–26 Oct 2026 (29 pages) |
|---|---|---|---|---|
| Vemana-Base | 278M | 0.361 | 0.385 | 0.346 |
| nanochat-telugu-560M | 561M | 0.383 | 0.408 | 0.364 |
| Dhee-Pravega-OG Atom | 646M | 0.393 | 0.414 | 0.371 |
| Pothana Base v2 | 222M | 0.440(0.426) | 0.456(0.444) | 0.425(0.410) |
| Goldfish Telugu 1000mb | 125M | 0.458 | 0.480 | 0.435 |
| Gemma-3-1B | 1.0B | 0.484 | 0.502 | 0.441 |
| Qwen3.5-0.8B-Base | 0.87B | 0.584 | – | – |
| Gemma-3-270m | 268M | 0.585 | – | – |
| Qwen3-0.6B-Base | 0.6B | 0.692 | – | – |
Bits per byte, lower is better: the bits a model needs to predict a page, divided by the page’s bytes. One scoring script, the same pages and float32 weights for every model; a window of 1,024 tokens (Goldfish 512, its limit); measured by us on 27 October 2026. Pothana Base v2 has 222M unique parameters (370M with its shared blocks), and its tokenizer cannot write about 4% of the text; in brackets, its score on the text it covers. Dhee-Pravega-OG Atom was scored at its revision released under Apache-2.0.
Vemana minus nanochat-telugu-560M, paired 95% interval: −0.022 [−0.024, −0.021] on the Sep–Oct 2026 pages; −0.023 [−0.024, −0.022] on the Jul–Aug 2026 pages; −0.018 [−0.021, −0.015] on the 23–26 Oct 2026 pages. The lead over nanochat-telugu-560M also holds at a 512-token window (−0.017) and without the editor who wrote a third of the Sep–Oct pages (−0.023).
Multiple-choice tests
On Telugu reading, knowledge and reasoning tests Vemana is mostly at chance, like the other open models under 1B that we tested; they need a bigger model or more data. We report the numbers and claim nothing from them.
| Test | Vemana | Chance |
|---|---|---|
| Belebele-te | 25.1% | 25% |
| MILU-te | 25.3% | 25% |
| MILU-te, cloze form | 30.2% | 25% |
| IndicXNLI-te | 35.0% | 33.3% |
| IndicCOPA-te | 58.4% | 50% |
lm-eval 0.4.13, 0-shot, each model’s start token first, float32; accuracy %. Chance is 25% on Belebele and MILU, 33.3% on IndicXNLI and 50% on IndicCOPA (about 55% from answer length alone).
Where it falls short
- Larger models are better. Sarvam-1 (2.5B parameters, about 9× larger) is level with Vemana on the Sep–Oct pages (0.356 against 0.361) and the 23–26 Oct pages (0.337 against 0.346), and better on the Jul–Aug pages (0.365 against 0.385), without the top editor’s pages and at a 512-token window.
- Knowledge and reasoning tests are at chance, as for most open models under 1B we tested; Qwen3.5-0.8B does better on the letter forms of Belebele (35.6%) and MILU (31.1%). They need a bigger model or more data.
- Without punctuation and line breaks, its lead over nanochat-telugu-560M narrows to 0.008 bits per byte.
- It makes things up. Asked to continue “తెలుగు భాష” (the Telugu language), it wrote fluently that Telugu Language Day is celebrated on 21 September; it is 29 August.
- It loses the thread. A short story prompt (“ఒక రోజు ఒక రైతు”, one day a farmer) turns into the same sentence about a farmer and a cow, repeated with small changes.
- Base model only. It continues text; it does not answer questions or follow requests, and it has no safety tuning.
- 1,024-token context, the length it was trained and tested at (about 570 Telugu words).
| Pages created | Vemana | Sarvam-1 | Verdict |
|---|---|---|---|
| Sep–Oct 2026 (416 pages) | 0.361 | 0.356 | Level |
| Jul–Aug 2026 (477 pages) | 0.385 | 0.365 | Sarvam-1 better |
| 23–26 Oct 2026 (29 pages) | 0.346 | 0.337 | Level |
“Level” means the paired 95% interval of the gap includes zero. Same scorer and pages as the table above.
How it was trained
- 1
Data
Public Telugu web text from three open datasets: Sangraha (AI4Bharat, its human-verified Telugu part), FineWeb-2 (Hugging Face) and HPLT 3.0. Filtered for Telugu script and length, near-duplicates removed across the three, and the Sep–Oct test pages and the multiple-choice test sets removed from the training text. The Jul–Aug and late-October test pages were written in 2026, after the training text was collected (web crawls up to 2024). 5.1B unique tokens.
- 2
Tokenizer
Our own 32,000-token Telugu tokenizer, with byte fallback so any text can be written. It needs 38–43% fewer tokens per Telugu word than Gemma 3’s.
- 3
Run
12.0B token positions on one H100 in about 16 hours, about $69 of GPU time, live in public. The final checkpoint was chosen as the release model in advance.
- 4
Checkpoints
Every 10% of the run and the final model, all published.
The run was live in public: anyone could follow it as it trained on udgamlabs.com/live/vemana. Every checkpoint is published, so the run can be studied step by step.
Good fits
- A base for Telugu tools: fine-tuning for chat, summarisation or classification, and continued pretraining.
- Studies of how a small model learns a language, from any of the published checkpoints.
- Telugu text generation on a laptop-class CPU, where wrong facts are acceptable.
It can produce false, offensive or harmful text, and should not be used for decisions about people.
The model
- Parameters
- 278,214,144 (278M), with tied input and output embeddings
- Layout
- Gemma 4 text architecture: 18 layers, width 1,024, 8 query and 2 key-value heads, full attention
- Initialisation
- Random weights; trained from scratch, with our own 32,000-token Telugu tokenizer
- Training
- 12.0B token positions: 5.1B unique tokens of public Telugu web text, read about 2.4 times
- Context
- 1,024 tokens, the length it was trained and tested at (about 570 Telugu words)
- Runs with
- transformers 5.19 or newer; about 65 tokens per second on a Mac mini’s CPU
- Licence
- Apache-2.0 (Copyright 2026 Udgam Labs LLP); training data keeps each source’s licence
Licence
Apache-2.0, for the weights and the tokenizer. Copyright 2026 Udgam Labs LLP.
Training data: Sangraha by AI4Bharat (CC-BY-4.0), FineWeb-2 by Hugging Face (ODC-By 1.0; use also subject to Common Crawl’s terms of use) and HPLT 3.0 by the HPLT project (CC0 packaging). Details in the model’s NOTICE file. Thank you to everyone who built and shared them.
How to cite
@misc{udgamlabs2026vemana,
title = {Vemana: a 278M Telugu language model trained from scratch},
author = {Prasad, Jitesh K},
year = {2026},
howpublished = {Udgam Labs},
url = {https://huggingface.co/udgamlabs/Vemana-Base}
}Everything we released
- Vemana-Base
The model: weights, tokenizer, card and NOTICE
- Vemana-Checkpoints
Every 10% of the run and the final model
- Vemana-Base-GGUF
For llama.cpp and for running in the browser
- Vemana on Hugging Face Spaces
Try it without downloading
- GitHub: udgamlabs/vemana
The code
- The live run page
The training run as it happened