Models
Models · Research release · 22 September 2026

Udgam Hinglish Commands 0.6B

Turns short English and Romanized Hinglish requests — Hindi and English mixed in Latin letters — into a structured intent and labelled details that an app can use.

Read the release post

The result

Complete-command exact match rose from 9.90% to 54.88%. 5,877 of 13,026 are still wrong.

Measured on the frozen Hinglish-TOP test against the original Qwen3-0.6B, both given the same prompt with four retrieved training examples. A command counts only if the whole structure matches the human reference: every label, value and nesting. The gain is 44.98 percentage points (paired 95% interval 43.93–46.03). It is one parsing task, not a measure of general Hindi or English ability.

What it is

Type
A LoRA adapter (fine-tune) on Qwen3-0.6B; not a newly pretrained model
Trained
About 10.1 million adapter parameters; the Qwen3-0.6B weights stay unchanged
Input
A short request in English or Romanized Hinglish (Hindi and English in Latin letters)
Output
An intent with labelled details, as a TOP tree; the included code turns it into nested JSON
Domains
Alarms, Events, Messaging, Music, Navigation, Reminders, Timers, Weather
Training data
Human-labelled pairs from Hinglish-TOP, which builds on TOPv2; no synthetic data
Licence
Adapter CC BY-SA 4.0; Udgam-written code Apache 2.0; Qwen materials keep Apache 2.0

A recorded example

bed time ka alarm kab hai? becomes the intent “Look up an alarm” with the detail alarm name: bed time. It proposes an interpretation and executes nothing; your app resolves dates and names and decides whether to act.

Measured results

Hinglish-TOP test · complete-tree exact match
CommandsRowsQwen3-0.6BAdapter
All13,0269.90%54.88%
Romanized Hinglish6,5137.54%46.84%
English6,51312.27%62.92%
Alarms2,39016.15%61.63%
Events9227.81%55.97%
Messaging1,0129.98%63.83%
Music1,4468.64%53.46%
Navigation1,9969.22%42.74%
Reminders1,9302.64%36.37%
Timers1,3869.38%54.55%
Weather1,94412.40%73.56%

The test has 6,513 entries, each in English and in Romanized Hinglish, so the rows are paired, not independent. All eight domains improved, but the spread is wide: reminders are weakest at 36.37% and weather strongest at 73.56%.

Read the full benchmark results

Run it

Install the Python wrapper from the GitHub repository, download the adapter package from Hugging Face, and call the parser on a request. It needs the pinned Qwen3-0.6B base. The package includes the training examples it retrieves from; loading the weights alone with another prompt does not reproduce the measured setup.

Intended for developers and researchers testing a small local command parser. Have the user confirm a proposed command before your app acts on it.

Scope and limitations

  • 5,877 of 13,026 complete interpretations are still wrong, and Hinglish stays below half at 46.84%.
  • Valid is not correct: 99.96% of outputs are well-formed trees, but a well-formed tree can still mean the wrong thing.
  • Not native-script (Devanagari) Hindi or other languages, not speech, not general chat, and not shown to be reliable in production or fast on low-end phones.
  • The results hold for this adapter with its four-retrieved-example prompt. Requests outside the eight domains can be forced into one of them.
  • Public benchmark examples may have been in Qwen3-0.6B’s pretraining data, and the dataset, with its international names and places, is not a representative sample of everyday requests.

The adapter and prepared data are CC BY-SA 4.0, following the Hinglish-TOP and TOPv2 data it comes from. The original model and dataset authors have not endorsed this adaptation.

Adapter, code and documentation