Most of what an app needs from a language model is not conversation. It is narrower: given the functions this app offers, turn the user's request into the right call — or recognise that none of them fits and ask or decline.
Udgam-384M is built for that one job. It is a 384M-parameter English model, trained from scratch, meant to be the language layer of an app on the device itself. Your app stays in charge: it checks each call, runs the function and confirms anything risky. The model takes no action by itself.
Four jobs
- App actions: turn a request into a call to one of the app's functions, with the right arguments.
- Documents to JSON: fill a given JSON schema from a document.
- Shell commands: propose a command for a person to review — never to run automatically.
- Knowing when not to act: ask for missing details, or decline when no tool fits.
What we built
- 384.4M parameters, 20 layers, a 32,768-token vocabulary, and a 4,096-token context for the chat model.
- Trained from scratch — no weights from any other model — on roughly 8 billion tokens of licensed public and synthetic text, then fine-tuned for chat and tool use, including examples of when to refuse.
- The Gemma 4 architecture, chosen so the same weights load without conversion code in Ollama, LM Studio, llama.cpp, MLX and transformers.
We held out evaluation tools, document types and shell tasks from training, so the held-out tests below use tools the model never saw.
Results
Every model was run the same way: greedy decoding in llama.cpp at Q8_0, thinking off, each with its own chat template, scored by the same code. The held-out sets (274 items) are our own and follow this model's prompt conventions, so they favour it. BFCL v3 and IFEval are neutral.
| Test | Udgam-384M | Granite 4.0 350M | SmolLM2 360M | Qwen3-0.6B |
|---|---|---|---|---|
| Held-out: right decision to act, ask or decline | 75 | 18.3 | 46.7 | 66.7 |
| Held-out: document → schema-valid JSON | 56 | 32 | 0 | 24 |
| Held-out: right shell behaviour | 85.9 | 59.4 | 39.1 | 67.2 |
| Held-out: exactly right app call | 31.4 | 45.7 | 1.4 | 44.3 |
| BFCL v3 simple | 60 | 90.7 | 4.7 | 80 |
| BFCL v3 irrelevance | 68 | 55.3 | 75.3 | 88 |
| IFEval, prompt strict | 26.1 | 47 | 31.4 | 58 |
| Generation speed, Q4_K_M, Apple M6 GPU (tokens/s) | 444 | 411 | 358 | 282 |
A few more measurements worth knowing:
- Quantization is cheap, temperature is not. Q8_0 and Q6_K stay within noise of full precision. Raising the temperature from greedy to 0.7 dropped BFCL simple from 60% to 46%.
- The 4K stage works. Continued pretraining at 4,096 tokens raised retrieval at 3,500 tokens from 1% to 83%.
- It is small in memory. At Q4_K_M on CPU it peaked at 323 MB of RAM.
Where it falls short
- About one call in three is exactly right on unseen tools. Validate every call before running it.
- Shell commands are almost always valid syntax, but only about 8% produced the correct output when executed (26 tasks). Never auto-execute them.
- It is not safety-hardened: on XSTest it declines only about 54–57% of harmful prompts and over-refuses about 15% of safe ones. That is below what we would want for an open chatbot. Use allowlists, validation and confirmation in your app.
- It is often factually wrong, English only, and not for medical, legal or financial advice.
Try it
Download Udgam-384M-Chat-Q8_0.gguf and the Modelfile from
Hugging Face, then:
ollama create udgam -f Modelfile
ollama run udgam "Explain what a hash map is in two sentences."
Send "think": false in API calls, since Ollama enables thinking by default. Q4_K_M is the build we recommend for
phones. The technical report covers
the architecture, every evaluation and the safety results; the GitHub repository
has runnable examples; and the walkthrough shows recorded examples in plain
language. Weights, tokenizer and docs are Apache-2.0.