How useful can a small English language model be if you train it from nothing, on a budget you could put on a credit card? That was the question behind Udgam English 125M, our first model trained entirely from scratch.
The short answer from this first attempt: not useful enough. The model missed the performance target we set before training. We are releasing it anyway — base and conversation variants, the tokenizer, the training recipe and the full results — because a measured negative result is still a result, and others can inspect and improve on the work.
What we built
- 125,829,314 parameters, a 12-layer GPT built on a pinned version of nanochat, with a 2,048-token context.
- From random weights. No pretrained weights, embeddings or vocabulary from another model; even the 16,384-token tokenizer was fitted on our own training text.
- Two variants: a base model for text completion, and a conversation variant tuned on one pass of OASST2.
Training, and what it cost
Pretraining saw about 2.03 billion prediction targets from a mix of FineWeb-Edu (60%), synthetic FinePhrase tables and tutorials (15% each) and a Nemotron fact-seeking subset (10%). Conversation tuning added 7,237 OASST2 examples. Everything ran on a single H100 in BF16.
We set a project ceiling of US16.58 at our last observation, paid entirely from credits. As the model card puts it, "zero new cash is not zero training cost" — and that figure leaves out the Mac, the internet connection and engineering time.
Results
We scored every model on the same frozen 2,500-question zero-shot suite: 500 items each from ARC-Challenge, HellaSwag, PIQA, BoolQ and WinoGrande, averaged per task. These are our protocol's numbers, not official leaderboard scores.
| Model | ARC-C | HellaSwag | PIQA | BoolQ | WinoGrande | Average |
|---|---|---|---|---|---|---|
| Udgam English 125M · base | 25.80 | 33.60 | 58.60 | 59.60 | 51.00 | 45.72 |
| Udgam English 125M · conversation | 26.40 | 34.20 | 58.40 | 61.80 | 51.20 | 46.40 |
| SmolLM2-135M | 30.00 | 44.20 | 69.60 | 60.40 | 54.60 | 51.76 |
| SmolLM2-360M | 35.20 | 57.60 | 72.20 | 61.40 | 60.60 | 57.40 |
| Sarvam-1 | 51.20 | 67.60 | 75.20 | 83.40 | 63.00 | 68.08 |
The gap to Sarvam-1, a model about twenty times larger, is roughly 22 points. On 24 strict automatic instruction-following checks, both variants scored zero; some answers were relevant but in the wrong format.
One number looks good and should not be taken at face value: on our GPU the base model decoded 184 tokens a second against 85 for SmolLM2-135M. But our path used a faster attention kernel than the baselines, so this is not a portable speed claim, and our packages are also larger on disk.
What we take from it
Two billion training targets and a tokenizer fitted from scratch were not enough to reach a small open model of the same size. The value of this release is the record: the exact artifacts, the costs, the evaluation suite and the miss, all in one place, so the next experiment — ours or anyone's — starts from measured ground rather than a guess.
Try it
The weights are in the Hugging Face snapshot and the v0.1.0 release on GitHub. From the complete distribution, it runs offline on a CPU:
shasum -a 256 -c SHA256SUMS
python -m pip install -r models/base/requirements.txt
python models/base/infer.py --device cpu --prompt "A useful way to check an explanation is" --max-new-tokens 64
It is native PyTorch only — no Transformers, GGUF or mobile builds — and English only. Treat it as a research artifact, not an assistant: its answers may be incorrect, biased or inappropriate. Code, tokenizer and weights are MIT-licensed; the training datasets keep their own terms.