# Vemana, a small Telugu model trained live in public

Canonical: https://udgamlabs.com/blog/vemana · Updated: 2026-10-27

> Vemana is a 278M Telugu base model we trained from scratch, live in public, in about 16 hours. On fresh Telugu text it leads every open base model under 1B we tested.

Release · 2026-10-27 · By Jitesh K Prasad

Vemana is a small Telugu language model. It has 278 million parameters and its own Telugu tokenizer, and it runs on
a laptop-class CPU. We trained it from scratch, live in public, in about 16 hours on one GPU, for about \$69 of GPU
time.

> **The short version.** On Telugu Wikipedia pages written after the compared models were released, Vemana predicts text better than every other open base model under 1B parameters that we tested, including one twice its size. A larger model, Sarvam-1 (2.5B), is level or better.

## Why a small Telugu model

Telugu has its own script, and many of its words are long, built from a root and several endings. Tokenizers made
mostly for other languages cut those words into many small pieces, so each sentence costs a model more steps. Vemana's
tokenizer learned from Telugu alone and needs 38-43% fewer tokens per Telugu word than Gemma 3's.

Small general-purpose models are weak at Telugu: on our test, Gemma-3-270m, about Vemana's size, scores 0.585 bits per
byte where Vemana scores 0.361 (lower is better). We wanted a model that people who read and write Telugu can run on
their own computers, fine-tune for their own tools, and take apart.

## Trained live in public

Vemana learned from 5.1 billion tokens of public Telugu web text from three open datasets: Sangraha from AI4Bharat (its
human-verified Telugu part), FineWeb-2 from Hugging Face, and HPLT 3.0. We filtered it for Telugu script and length,
removed near-duplicates, and removed the September-October test pages and the multiple-choice tests from it; the other
test pages were written in 2026, after the text was collected (web crawls up to 2024). The model read it about 2.4
times.

The run took about 16 hours on one H100 GPU, about \$69 of GPU time, and anyone could follow it on
[udgamlabs.com/live/vemana](https://udgamlabs.com/live/vemana). A checkpoint from every 10% of the run is public, so you
can study how a small model learns a language, step by step.

## How we tested it

A score means little if the model has seen the test, so we used Telugu Wikipedia pages created after the compared
models were released: 416 pages from 1 September to 22 October 2026 (the main test), 477 from July and August, and 29
from 23 to 26 October, after every model here.

The measure is *bits per byte*: the bits a model needs to predict a page, divided by the page's length in bytes. Lower
is better, and it is fair between models with different tokenizers. Every model ran through the same scoring script
on the same pages.

We fixed the rule before training: at least 0.02 bits per byte better than the best open base model under 1B, and
still ahead at a shorter 512-token window, without the editor who wrote the most pages, and on the newest pages. We
chose the final checkpoint as the release model in advance. Every gap has a paired 95% interval: we resample the pages
4,000 times and compare two models on the same pages each time.

One caveat: Dhee-Pravega-OG Atom was released on 8 October and does not say where its data ends, so the older pages
could be in its training data. That could only help it.

## The results

![Predicting new Telugu text: bits per byte, lower is better, on 416 Telugu Wikipedia pages created 1 September to 22 October 2026. Sarvam-1 (2.5B) 0.356, Vemana (278M) 0.361, nanochat-telugu-560M 0.383, Dhee-Pravega-OG Atom (646M) 0.393, Pothana Base v2 (222M) 0.426 on the text its tokenizer covers, Goldfish Telugu (125M) 0.458, Gemma-3-1B 0.484](https://udgamlabs.com/models/vemana/fresh-text-bpb.png)

*The main test, 416 pages. Vemana is the dark bar. Sarvam-1, about 9× larger, is level with it: the interval of the gap includes zero.*

On the main test, Vemana scores 0.361 bits per byte. The next best open model under 1B, nanochat-telugu-560M, has twice
the parameters and scores 0.383: Vemana is 0.022 lower (95% interval 0.021 to 0.024). It also leads on the July and
August pages (0.385 against 0.408) and on the newest pages (0.346 against 0.364), and it passed every check in the rule.

Here is what it writes when given the start of a sentence (sampled; the English translation is ours):

> **వర్షాకాలంలో** సీజనల్ వ్యాధులు ఎక్కువగా వస్తాయి. అందువల్ల ఆరోగ్యంగా ఉండటం చాలా ముఖ్యం. మీరు వర్షాకాలంలో డెంగ్యూ,
> మలేరియా, టైఫాయిడ్, వైరల్ ఇన్ఫెక్షన్ల వంటి సీజనల్ వ్యాధుల బారిన పడే ప్రమాదం ఉంది. …
>
> *In the rainy season, seasonal illnesses are more common, so staying healthy matters a lot. In the rainy season you
> are at risk of seasonal illnesses such as dengue, malaria, typhoid and viral infections. …*

## Where it falls short

- **A bigger model is level or better.** Sarvam-1, with 2.5 billion parameters, about 9× Vemana's size, is level on the
  main test (0.356 against 0.361) and on the newest pages (0.337 against 0.346), and better on the July and August pages
  (0.365 against 0.385).
- **It knows little.** On Telugu multiple-choice tests of reading, knowledge and reasoning (Belebele, MILU, IndicXNLI),
  it is at chance, like most of the other open models under 1B that we tested.
- **It makes things up.** Asked to continue "తెలుగు భాష" (the Telugu language), it wrote fluently that Telugu Language
  Day falls on 21 September. It is 29 August.
- **It loses the thread.** A story that starts "ఒక రోజు ఒక రైతు" (one day a farmer) turns into the same sentence about
  a farmer and a cow, repeated with small changes.
- **It is a base model.** It continues text; it does not answer questions or follow requests, and it has no safety
  tuning. Its context is 1,024 tokens, about 570 Telugu words.

## What is next

The obvious next step is a chat version that answers questions and follows requests in Telugu. That needs Telugu
instruction data under a licence that allows training, and there is not much of it yet. Until then, Vemana is a base to
build on: fine-tune it, keep training it, or study its checkpoints.

## Try it

- In your browser: [huggingface.co/spaces/udgamlabs/Vemana](https://huggingface.co/spaces/udgamlabs/Vemana)
- The model page, with every result: [udgamlabs.com/models/vemana](https://udgamlabs.com/models/vemana)
- Weights: [huggingface.co/udgamlabs/Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base), and every checkpoint
  in [Vemana-Checkpoints](https://huggingface.co/udgamlabs/Vemana-Checkpoints)
- Code: [github.com/udgamlabs/vemana](https://github.com/udgamlabs/vemana)

Thank you to AI4Bharat, Hugging Face and the HPLT project for the open data Vemana stands on, and to the Telugu
Wikipedia editors whose new pages made a fair test possible.

---

## About Udgam Labs

Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Today that means open models such as Gargi, a 384M-parameter language model that comes within 2 points of Qwen2.5-0.5B on three reasoning benchmarks with about 1,000× less training data. Next come products, custom models for businesses, and AI in the physical world. Udgam (उद्गम) is Sanskrit for origin.

Udgam Labs · AI research from first principles · https://udgamlabs.com