This one was a fun experiment. We took a mid-range Samsung phone from 2021, about $330 at launch, and trained a small language model on it, live on YouTube and X. No cloud, flight mode on, every training step on the phone's own chip.
We also gave it something to chase: 1.4697, the best validation loss in the README of Andrej Karpathy's nanoGPT, which took one A100 data-centre GPU about 3 minutes. Our phone took hours, with a nanoGPT-style model that has about nine times fewer parameters.
You can watch the recording, see the finished page, or train your own with the short tutorial below. The code is at github.com/udgamlabs/phonegpt, and the trained model is on Hugging Face as a record of the run.
PhoneGPT is only our project name. It has no link to the offline-AI iPhone app with the same name.
What it learned to write
It starts from random numbers and reads only about 1 MB of Shakespeare, one character at a time. Nobody tells it what a word or a play is. This is real output from the live run, cut for length and otherwise unedited.
Before any training:
eXXYpw:Gq,'&CnnWdlgN;meym&ipZNYdL,lB.I.fEEEwVqOy'QJkZOOQ.nWQZF--zzmKemSL
After 1,000 steps, about 26 minutes:
DUCHES:
Mut the make sould to say-mouse,
And you the beother sire no the will where his,
After 24,000 steps, at the end:
DION:
Say, where I may be made what I add
And to my lord, like a mercy than to my father.
LORD WILLOUGHBY:
As I am, in king, and so follow my uncle?
It won't write good Shakespeare, but it picked up the names, the colons and the rhythm of a play by itself. Watching that happen slowly on an old phone was the fun part.
The race
| Step | 16,000 | 17,000 | 18,000 | 19,000 | 20,000 | 24,000 |
|---|---|---|---|---|---|---|
| Hours in | 7.2 | 7.6 | 8.1 | 8.5 | 9.0 | 10.75 |
| Score | 1.4782 | 1.4744 | 1.4663 | 1.4748 | 1.4785 | 1.4817 |
This wasn't a like-for-like race, in both directions:
| nanoGPT's README | PhoneGPT | |
|---|---|---|
| Hardware | One A100 GPU | Galaxy F62, 2 of its CPU cores |
| Time | About 3 minutes | 10 h 45 min (best score at about 8 h) |
| Parameters | 10.65M | 1.21M |
| Context | 256 characters | 128 characters |
| Score | Best of its sampled checks | Every window of the same held-out split |
How we did it
The trainer. One C file, adapted from Karpathy's llm.c (MIT licence). We changed it to start from random weights, read characters, use the phone's fast CPU instructions, and save a checkpoint every 1,000 steps. It is compiled on a computer into one static program and copied to the phone over USB. There is no app to install.
The model. 6 layers, 4 attention heads, 128-wide, reading 128 characters at a time, in batches of 16. That's 1,214,976 parameters, and a checkpoint is about 5 MB.
The phone. A Samsung Galaxy F62 with an Exynos 9825 chip, plugged in, in flight mode. Steps took a steady 1.5 seconds or so from start to finish, with no slowdown over the 10 hours.
The score. The phone only trained. After each checkpoint, a script on a Mac mini copied it over USB and scored it on the whole held-out text, which took 3 to 4 minutes. So training was all on the phone, and the scoring was not.
The stream. The phone's own screen was mirrored over USB with scrcpy. A local web page put that next to the score and the latest text, and FFmpeg on the Mac mini sent the page to YouTube and X. The mirror cost about 1% of training speed.
We had rehearsed all of this privately the night before, with a different seed, and that run did go below by our rule and ended at 1.460. The live run didn't. That was the risk of doing it live, and we said so before it started.
Train your own
We have only tested this on our Galaxy F62, from a Mac. Other 64-bit Android phones should work the same way, with no root and no app, but your speed and your fast cores will differ. Keep the phone plugged in, and somewhere it can lose heat.
You need: an Android phone with USB debugging on, a USB cable, and a computer with adb and Zig, which we use as the cross-compiler.
1. Get the code and the text
git clone https://github.com/udgamlabs/phonegpt
cd phonegpt
curl -sSL -o input.txt https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
2. Build it for the phone
zig cc -target aarch64-linux-musl -O3 -ffast-math -static -o phone_gpt phone_gpt.c -lpthread -lm
3. Find the fast cores
adb shell 'for c in /sys/devices/system/cpu/cpu[0-9]*; do echo "$(basename $c) $(cat $c/cpufreq/cpuinfo_max_freq)"; done'
On the F62, cpu6 and cpu7 are the fastest. Pinning to those two was 2.3 times faster than using all eight, because the
fast cores otherwise wait for the slow ones. taskset takes those cores as a hex mask: cpu6 and cpu7 are c0, cpu7
alone is 80, and cpu4 to cpu7 are f0. Set -threads to the number of cores in the mask.
4. Copy it over and start
This trial stops itself after 30 minutes, which is about when real-looking words appeared on ours.
adb shell mkdir -p /data/local/tmp/phonegpt
adb push phone_gpt input.txt /data/local/tmp/phonegpt/
adb shell chmod +x /data/local/tmp/phonegpt/phone_gpt
adb shell svc power stayon usb
adb shell 'cd /data/local/tmp/phonegpt && (nohup taskset c0 ./phone_gpt -threads 2 -L 6 -H 4 -C 128 -T 128 -B 16 -iters 18000 -warm 100 -lr 2e-3 -minlr 2e-4 -dropout 0.1 -ei 250 -eb 16 -log 25 -save 1000 -sample_every 250 -minutes 30 -maxtemp 45 -gen 1000 -tag run </dev/null > out.txt 2>&1 &)'
For the full run, change -minutes 30 to -minutes 510. On the F62, 18,000 steps took about 8 hours. The trainer
pauses by itself if the battery goes above 45 C. To replay our live run instead, use -iters 24000 -decay 18000 -minutes 680 -seed 20261105. A fresh build on the same phone matched the live run's log at every logged point over the first 277 steps, as far as we checked.
5. Watch it learn
adb shell tail -n 30 /data/local/tmp/phonegpt/run_samples.txt
adb shell tail -n 3 /data/local/tmp/phonegpt/run_log.txt
adb shell dumpsys thermalservice | grep -E 'SKIN|BAT'
6. Score it properly and clean up
The held-out number in the trainer's own log is a quick check on a small sample, and on our rehearsal it read about 0.03 lower than the true score. To compare with anything, score every window on your computer:
adb pull /data/local/tmp/phonegpt/run_final.bin
zig cc -O3 -ffast-math -o phone_score phone_score.c -lpthread -lm
./phone_score run_final.bin input.txt score 4 1 16
cat score_summary.txt
adb shell svc power stayon false
This took about 4 minutes on our Mac mini with 4 threads, and it gave exactly the live run's 1.4663 when we scored that
checkpoint from a fresh copy of this code. run_final.bin appears when a run ends by itself, and its last text is in
run_sample.txt. To stop a run early, use adb shell pkill phone_gpt; the earlier run_step*.bin checkpoints stay.
What we learned
- Run past the best point and the score goes up. From step 18,000 to 24,000, the held-out score rose 0.0154 while the training score kept falling. That looks like overfitting, though we didn't test it with another run. For this model on this text, we'd stop at 18,000.
- Keep every checkpoint. The last one was not the best one.
- Don't race with a thin margin. The random seed moved the best score by more than the margin we were racing for. A live race on one seed can go either way.
- Trust the whole-split score. A quick check can look like a win long before the real score gets there.
- Heat was fine, but only just. Skin temperature peaked at 46.9 C against the 47 C limit we had set, and the battery at 40.6 C. In our tests the big cores dropped to about half of their top clock within a minute, then held there.
- Start long commands in the background. In the rehearsal a stuck prompt cost us 32 minutes of an empty stream.
Where it falls short
- It missed. One checkpoint under the line doesn't meet the rule we set before the start.
- The model isn't useful for anything except this. It writes Shakespeare-shaped nonsense, from a tiny text.
- 1.4697 is a score from a small teaching run, not a measure of a good language model. The comparison is a fun one, and it doesn't mean that a phone can replace a GPU, which did this in minutes.
- It's one live run on one phone. We haven't tested other phones.
Credit, and who did this first
The trainer is adapted from llm.c and the target comes from nanoGPT, both by Andrej Karpathy, who isn't involved in this. The text is his Tiny Shakespeare.
Others have trained models on small devices before us. Yuuki was trained on a Redmi 12 and is much larger, and Qapla trained a character-level model on an $8 ESP32-S3. As far as we can find, nobody has streamed a language model training live on a phone before. If someone has, please correct us.