On 2 November, MIT CSAIL showed GPT-6 Astra rewriting the weights of a small open model. Qwen3.5-0.8B often loops when it thinks out loud. Astra changed 131,072 of its weights, and its MATH-500 score rose from 7.9% to 48.1% at a 32,768-token limit, mostly because answers that used to loop now finished. The authors released the edited model, their code and their logs. That is why anyone could check the result.
A wave of "weight edit" models followed on Hugging Face. We wanted to know what was in it, and how many asked the simplest question first: would a setting, a prompt or a decode rule do the same job, with no weights changed?
What we did
On 12 November we searched the public Hugging Face API for repos created since 25 October, using 24 terms such as "weight-space programming", "weight edit" and "loop fix". 245 repos matched. We read every model card twice, the second time against a fixed rubric, and checked each disputed label by hand. We ran no models, so the results below are what the cards report.
| What it is | Repos |
|---|---|
| MIT's checkpoint, and copies, rebuilds or derivatives of it | 24 |
| Format conversions of other people's edits | 26 |
| New edits for loops or overthinking | 38 |
| New edits for other faults | 92 |
| An edit claimed, but no result shown | 20 |
| In scope | 200 |
| Matched the search, but not a weight edit | 45 |
The 130 new edits came from 76 accounts and went well beyond maths. People edited Whisper, translation models, tool callers, classifiers and game policies. Six repos report that no working edit was found, and say so on the card.
The one-line question
Qwen's own card says Qwen3.5-0.8B "operates in non-thinking mode by default", and warns that it is prone to thinking loops. So for any loop edit on this model, the obvious comparison is the unedited model with thinking switched off, which is one line of configuration.
Across the 200 repos, 15 compared the edit with a fix that changes no weights:
| Outcome | Repos |
|---|---|
| The edit did better | 5 |
| A tie | 2 |
| The simpler fix did better | 3 |
| Mixed | 2 |
| Compared at different token limits | 3 |
The other 185 made no such comparison. 42 repos are Qwen3.5-0.8B loop edits or copies of one, and 2 of them tested thinking off. Both gave thinking off an 8,192-token cap and the edit 32,768, and about a third of the thinking-off answers were still cut off at that cap.
What the careful reruns found
The most careful comparisons came from outside model cards. Issue #4 on MIT's log repository holds a pre-registered rerun. Under Qwen's own non-thinking settings at 32,768 tokens, the unedited model scored between 44.0% and 51.7% across 4 samples of all 500 problems. Eastwood's "Rethinking Weight-Space Programming" ran a full grid with paired confidence intervals. It put MIT's edit 6.4 points ahead of the best no-edit setting on MATH-500 and 11.5 ahead on GSM8K, and 2.2 behind on IFEval.
Our own check was smaller: 300 MATH-500 problems, one sample each, at 32,768 tokens. Thinking off scored 49.7% and MIT's edit 49.0%, with thinking off using about 40% of the tokens. That is one sample, and it disagrees with Eastwood's larger run, so we treat it as one contested data point.
Taken together, MIT's edit is a real repair. It is far ahead of the model with thinking on, and level with or a few points ahead of switching thinking off. MIT's post makes the same point: fixing loops "is not the same as making a model better at math".
Edits that did something a setting couldn't
- Thinking in German (kaivoss/Qwen3-4B-German-Thinking): a prompt gave 0 German reasoning traces in 32 on the original model; the edit gave 40 of 40.
- Whisper copy loops (emillykkegrann/whisper-large-v3-copy-heads-off): zeroing 11 decoder heads cut long loops on 30 unseen recordings from 21 to 1. The card also notes that slicing the audio by voice activity avoids the loops without any edit.
- Tool calls (hunterbown/trainless-toolcall-2b): on 703 held-out cases the card reports a tie with a strong system prompt, and says so plainly.
Three other cards reported that a simpler fix beat their edit, and two traced most of their gain to something else. We think reporting that is the most useful thing a card can do.
What we'd ask of the next weight edit
- Compare it with the cheapest fix that changes no weights, at the same token limit.
- Report how many answers hit the limit, and accuracy where both versions finish.
- Add a random edit of the same size as a control. Only 19 of the 200 cards did.
- Test on items the edit was not built from.
Limits
- Keyword search: an edit whose card uses none of our terms is missing.
- Refusal ablation: this older genre is mostly not covered.
- Card claims: we did not reproduce them, and labels are judgement calls; the rubric is published with the data.
Credits
MIT CSAIL (Takehi and co-authors) for the post, checkpoint and logs; the Hugging Face rebuild that reached 51.0%; the contributors to issue #4 on RikiyaT/lm_understand-logs; Eastwood; and every author whose card reported a control, a tie or a loss.
Dataset: udgamlabs/weight-edit-census