# PhoneGPT, a small language model trained live on a 5-year-old phone

Canonical: https://udgamlabs.com/blog/phonegpt · Updated: 2026-11-06

> We trained a 1.21M-parameter model on a 2021 phone's own chip, live, against nanoGPT's published score, and missed by 0.0047. How we did it, and how to try it.

Note · 2026-11-05 · By Jitesh K Prasad

This one was a fun experiment. We took a mid-range Samsung phone from 2021, about \$330 at launch, and trained a small
language model on it, live on YouTube and X. No cloud, flight mode on, every training step on the phone's own chip.

We also gave it something to chase: 1.4697, the best validation loss in the README of Andrej Karpathy's nanoGPT, which
took one A100 data-centre GPU about 3 minutes. Our phone took hours, with a nanoGPT-style model that has about nine
times fewer parameters.

> **Result.** It missed by 0.0047. The phone reached 1.4663 at one checkpoint, below 1.4697, but our rule was two checkpoints in a row, and the closest pair was 1.4744 and 1.4663. So no, it did not go below. It came very close, and we think that is still surprising for a phone.

You can watch [the recording](https://www.youtube.com/watch?v=gXQFqCBB91U), see [the finished page](https://udgamlabs.com/live/phone), or
train your own with the [short tutorial below](#train-your-own). The code is at
[github.com/udgamlabs/phonegpt](https://github.com/udgamlabs/phonegpt), and the trained model is on
[Hugging Face](https://huggingface.co/udgamlabs/PhoneGPT) as a record of the run.

PhoneGPT is only our project name. It has no link to the offline-AI iPhone app with the same name.

## What it learned to write

It starts from random numbers and reads only about 1 MB of Shakespeare, one character at a time. Nobody tells it what
a word or a play is. This is real output from the live run, cut for length and otherwise unedited.

Before any training:

```text
eXXYpw:Gq,'&CnnWdlgN;meym&ipZNYdL,lB.I.fEEEwVqOy'QJkZOOQ.nWQZF--zzmKemSL
```

After 1,000 steps, about 26 minutes:

```text
DUCHES:
Mut the make sould to say-mouse,
And you the beother sire no the will where his,
```

After 24,000 steps, at the end:

```text
DION:
Say, where I may be made what I add
And to my lord, like a mercy than to my father.

LORD WILLOUGHBY:
As I am, in king, and so follow my uncle?
```

It won't write good Shakespeare, but it picked up the names, the colons and the rhythm of a play by itself. Watching
that happen slowly on an old phone was the fun part.

## The race

![The stream's finish frame: the phone's screen on the left, and on the right the headline Missed by 0.0047, the phone's last score 1.4817, the goal 1.4697, a chart and the model's last text.](https://udgamlabs.com/blog/phonegpt/finish-frame.jpg)

*The end of the stream. The big number is the last score; the best was 1.4663.*

![Line chart of PhoneGPT's score from step 4,000 to 24,000. It falls towards nanoGPT's 1.4697, dips under it once at step 18,000 with 1.4663, then rises slowly to 1.4817.](https://udgamlabs.com/blog/phonegpt/scores.png)

*Lower is better. Scored on every window of the held-out 10% of the text.*

| Step | 16,000 | 17,000 | 18,000 | 19,000 | 20,000 | 24,000 |
|---|---:|---:|---:|---:|---:|---:|
| Hours in | 7.2 | 7.6 | 8.1 | 8.5 | 9.0 | 10.75 |
| Score | 1.4782 | 1.4744 | **1.4663** | 1.4748 | 1.4785 | 1.4817 |

This wasn't a like-for-like race, in both directions:

| | nanoGPT's README | PhoneGPT |
|---|---|---|
| Hardware | One A100 GPU | Galaxy F62, 2 of its CPU cores |
| Time | About 3 minutes | 10 h 45 min (best score at about 8 h) |
| Parameters | 10.65M | 1.21M |
| Context | 256 characters | 128 characters |
| Score | Best of its sampled checks | Every window of the same held-out split |

## How we did it

**The trainer.** One C file, adapted from Karpathy's [llm.c](https://github.com/karpathy/llm.c) (MIT licence). We
changed it to start from random weights, read characters, use the phone's fast CPU instructions, and save a checkpoint
every 1,000 steps. It is compiled on a computer into one static program and copied to the phone over USB. There is no
app to install.

**The model.** 6 layers, 4 attention heads, 128-wide, reading 128 characters at a time, in batches of 16. That's
1,214,976 parameters, and a checkpoint is about 5 MB.

**The phone.** A Samsung Galaxy F62 with an Exynos 9825 chip, plugged in, in flight mode. Steps took a steady 1.5 seconds or so from start to finish, with no slowdown over the 10 hours.

**The score.** The phone only trained. After each checkpoint, a script on a Mac mini copied it over USB and scored it
on the whole held-out text, which took 3 to 4 minutes. So training was all on the phone, and the scoring was not.

**The stream.** The phone's own screen was mirrored over USB with [scrcpy](https://github.com/Genymobile/scrcpy). A local
web page put that next to the score and the latest text, and FFmpeg on the Mac mini sent the page to YouTube and X. The
mirror cost about 1% of training speed.

We had rehearsed all of this privately the night before, with a different seed, and that run did go below by our rule
and ended at 1.460. The live run didn't. That was the risk of doing it live, and we said so before it started.

## Train your own

We have only tested this on our Galaxy F62, from a Mac. Other 64-bit Android phones should work the same way, with no
root and no app, but your speed and your fast cores will differ. Keep the phone plugged in, and somewhere it can lose heat.

**You need:** an Android phone with [USB debugging](https://developer.android.com/studio/debug/dev-options) on, a USB
cable, and a computer with [adb](https://developer.android.com/tools/releases/platform-tools) and
[Zig](https://ziglang.org/download/), which we use as the cross-compiler.

**1. Get the code and the text**

```bash
git clone https://github.com/udgamlabs/phonegpt
cd phonegpt
curl -sSL -o input.txt https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
```

**2. Build it for the phone**

```bash
zig cc -target aarch64-linux-musl -O3 -ffast-math -static -o phone_gpt phone_gpt.c -lpthread -lm
```

**3. Find the fast cores**

```bash
adb shell 'for c in /sys/devices/system/cpu/cpu[0-9]*; do echo "$(basename $c) $(cat $c/cpufreq/cpuinfo_max_freq)"; done'
```

On the F62, cpu6 and cpu7 are the fastest. Pinning to those two was 2.3 times faster than using all eight, because the
fast cores otherwise wait for the slow ones. `taskset` takes those cores as a hex mask: cpu6 and cpu7 are `c0`, cpu7
alone is `80`, and cpu4 to cpu7 are `f0`. Set `-threads` to the number of cores in the mask.

**4. Copy it over and start**

This trial stops itself after 30 minutes, which is about when real-looking words appeared on ours.

```bash
adb shell mkdir -p /data/local/tmp/phonegpt
adb push phone_gpt input.txt /data/local/tmp/phonegpt/
adb shell chmod +x /data/local/tmp/phonegpt/phone_gpt
adb shell svc power stayon usb
adb shell 'cd /data/local/tmp/phonegpt && (nohup taskset c0 ./phone_gpt -threads 2 -L 6 -H 4 -C 128 -T 128 -B 16 -iters 18000 -warm 100 -lr 2e-3 -minlr 2e-4 -dropout 0.1 -ei 250 -eb 16 -log 25 -save 1000 -sample_every 250 -minutes 30 -maxtemp 45 -gen 1000 -tag run </dev/null > out.txt 2>&1 &)'
```

For the full run, change `-minutes 30` to `-minutes 510`. On the F62, 18,000 steps took about 8 hours. The trainer
pauses by itself if the battery goes above 45 C. To replay our live run instead, use `-iters 24000 -decay 18000 -minutes 680
-seed 20261105`. A fresh build on the same phone matched the live run's log at every logged point over the first 277 steps, as far as we checked.

**5. Watch it learn**

```bash
adb shell tail -n 30 /data/local/tmp/phonegpt/run_samples.txt
adb shell tail -n 3 /data/local/tmp/phonegpt/run_log.txt
adb shell dumpsys thermalservice | grep -E 'SKIN|BAT'
```

**6. Score it properly and clean up**

The held-out number in the trainer's own log is a quick check on a small sample, and on our rehearsal it read about
0.03 lower than the true score. To compare with anything, score every window on your computer:

```bash
adb pull /data/local/tmp/phonegpt/run_final.bin
zig cc -O3 -ffast-math -o phone_score phone_score.c -lpthread -lm
./phone_score run_final.bin input.txt score 4 1 16
cat score_summary.txt
adb shell svc power stayon false
```

This took about 4 minutes on our Mac mini with 4 threads, and it gave exactly the live run's 1.4663 when we scored that
checkpoint from a fresh copy of this code. `run_final.bin` appears when a run ends by itself, and its last text is in
`run_sample.txt`. To stop a run early, use `adb shell pkill phone_gpt`; the earlier `run_step*.bin` checkpoints stay.

## What we learned

- **Run past the best point and the score goes up.** From step 18,000 to 24,000, the held-out score rose 0.0154 while
  the training score kept falling. That looks like overfitting, though we didn't test it with another run. For this
  model on this text, we'd stop at 18,000.
- **Keep every checkpoint.** The last one was not the best one.
- **Don't race with a thin margin.** The random seed moved the best score by more than the margin we were racing for.
  A live race on one seed can go either way.
- **Trust the whole-split score.** A quick check can look like a win long before the real score gets there.
- **Heat was fine, but only just.** Skin temperature peaked at 46.9 C against the 47 C limit we had set, and the battery
  at 40.6 C. In our tests the big cores dropped to about half of their top clock within a minute, then held there.
- **Start long commands in the background.** In the rehearsal a stuck prompt cost us 32 minutes of an empty stream.

## Where it falls short

- It missed. One checkpoint under the line doesn't meet the rule we set before the start.
- The model isn't useful for anything except this. It writes Shakespeare-shaped nonsense, from a tiny text.
- 1.4697 is a score from a small teaching run, not a measure of a good language model. The comparison is a fun one, and
  it doesn't mean that a phone can replace a GPU, which did this in minutes.
- It's one live run on one phone. We haven't tested other phones.

## Credit, and who did this first

The trainer is adapted from [llm.c](https://github.com/karpathy/llm.c) and the target comes from
[nanoGPT](https://github.com/karpathy/nanoGPT), both by Andrej Karpathy, who isn't involved in this. The text is his
[Tiny Shakespeare](https://github.com/karpathy/char-rnn).

Others have trained models on small devices before us. [Yuuki](https://huggingface.co/OpceanAI/Yuuki-best) was trained
on a Redmi 12 and is much larger, and [Qapla](https://github.com/Carloscodix/qapla) trained a character-level model on
an \$8 ESP32-S3. As far as we can find, nobody has streamed a language model training live on a phone before. If someone
has, please correct us.

---

## About Udgam Labs

Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Today that means open models such as Gargi, a 384M-parameter language model that comes within 2 points of Qwen2.5-0.5B on three reasoning benchmarks with about 1,000× less training data. Next come products, custom models for businesses, and AI in the physical world. Udgam (उद्गम) is Sanskrit for origin.

Udgam Labs · AI research from first principles · https://udgamlabs.com