Vemana is learning Telugu, live
Vemana is a 278-million-parameter Telugu base language model, trained from scratch by Udgam Labs on one GPU. This page follows the run while it happens. Every number comes from the training log and updates about once a minute.
Table of scores
Where the run is
What Vemana writes now
At every milestone the saved model continues the same six Telugu openings. The English line under each opening is our gloss of the prompt; Vemana's own text is shown exactly as it wrote it, mistakes included, with a literal English translation underneath. Pick two checkpoints to compare.
English under each answer: machine translation (Claude Sonnet 5.5), literal: broken Telugu stays broken. Translations appear a few minutes after each milestone.
Sampled: picks among the likelier next words at random (temperature 0.8, top 40, a fixed seed), closer to how people use a model. Greedy: always the single likeliest next word, which shows repetition more plainly.
The first answers arrive at the 10% milestone.
Is it on track?
These checks and their reference lines were fixed before training began. They are progress readings only: the run trains to the end whatever they show, and only the final score, after our release checks, decides any claim.
What has happened so far
How to read this page
What Vemana is
A base language model: it has read a large amount of Telugu text and learned to continue it. It has not been taught to follow instructions or chat; that would be a later step. "From scratch" means it starts from random numbers, not from another company's model.
What bits per byte means
As the model reads a page, it predicts what comes next at every point. Bits per byte counts how many yes-or-no questions it would need, on average, to pin down each byte of the real text. Lower means it predicted better. A Telugu letter takes 3 bytes, so a score of 0.40 is about 1.2 bits per letter.
Measuring per byte rather than per word piece lets models with different vocabularies be compared fairly.
Why "fresh" pages
Big models are trained on most of the web, so an old test page may already be in their memory. We test on 416 Telugu Wikipedia pages created after the rival models were trained, and we removed any overlap with these pages from our own training data. Nobody's model has read them.
What would count as a win
The aim is the best open Telugu base model under 300 million parameters. Today's best in that size scores 0.426 on these pages. To claim a clear win the final model must reach 0.406 or lower. Until the final score exists, this page makes no claim.
The 25% check
At 20% of the run a copy of the model is branched off and given a quick cooldown, so it finishes as a complete model at 25%. Scoring that copy shows how far along Vemana is: how a finished model at this point would score. Below 0.421 reads as on track; 0.426 is the bar, today's best open model under 300M. The run continues to the end regardless.
The halfway check
At 50% the model, still mid-training, is scored. The 25% check told us how much a cooldown lowers the score, so: projected final = score at 50% minus that cooldown drop. At 0.406 or lower the projection reads as on track. It is an estimate, and the run continues to the end regardless.
Why the score drops at the end
For most of the run the learning rate (how big each correction is) stays constant, so the model keeps hopping around a good solution. Over the last 20% it is lowered smoothly to zero and the model settles into the best spot nearby. Scores usually fall quickly in this cooldown, which is why the 25% check scores a cooled-down copy rather than the raw mid-run model.
How it is trained
- 22,888 steps of 524,288 tokens each (512 passages of 1,024 tokens): 12 billion tokens in all.
- Optimizer MuonAdamW: Muon for the weight matrices, AdamW for the embeddings and the small scalar weights.
- Learning rate: 500 warm-up steps, then constant, then a cooldown to zero over the last 20% (from step 18,310).
- 16-bit compute with full-precision weights; one GPU.
The model's architecture
- A Gemma-4-style decoder with 278.2 million parameters.
- 18 layers, each 1,024 wide.
- Attention with 8 query heads sharing 2 key-value heads, 128 dimensions each.
- A GeGLU feed-forward block 3,584 wide.
- A 32,000-piece Telugu vocabulary, with the input and output tables shared.
- It reads up to 1,024 tokens of context at once.
What it reads
Three open Telugu collections: Sangraha (its verified Telugu part), FineWeb-2 Telugu and HPLT 3.0 Telugu. We filtered them, removed near-duplicates and removed anything overlapping the test pages. That leaves 5.1 billion unique tokens (2.9B Sangraha, 1.3B HPLT, 0.9B FineWeb-2), so 12 billion tokens of training means each passage is read about 2.35 times.
Official scores and live estimates
Official: each milestone model is saved, converted to the standard Hugging Face format and scored in full precision on all 416 pages, with the same script used for the rivals. Live: every 2.5% the running model is scored in place at lower precision. Live points are close but approximate; decisions use official scores only.
Why the run comes in segments
The whole run takes about 17 to 22 hours on one H100 GPU. Each machine runs for at most 8 hours, then saves everything (weights, optimizer, the exact place in the data). A new machine picks up from that save. Between segments the page may pause for a few minutes.
What happens after training
If the final score is 0.406 or lower, the result goes through our release check: rivals re-scored with the same harness, paired statistics on the same pages, every number written into a facts file. Only then would we claim anything. If it falls short, we will say so and write up what we learned.