もりし retweeted
Can one improvement help a model discover the next?
If a model becomes better at finding useful edits, it could help produce its next improvement. That’s the recursive self-improvement loop I want to test, starting from the Weight-Space Programming work by @rikiyatakehi and colleagues.
Astra rewrote parts of two Qwen3.5-0.8B weight matrices without gradient-based training. The authors reported MATH-500 accuracy rising from 7.9% to 48.1%.
My analysis connects three pieces around that edit: the steering rule it stores, the corrections shared weights can express, and the output changes Fisher geometry predicts locally. The aim is to turn an impressive repair into something we can reconstruct, measure and build on.
At a linear projection, the idealized rank-one edit follows a familiar identity:
ΔW = d kᵀ ⇒ Δh(x) = d [kᵀa(x)]
Read a scalar from the current activations, use it to scale a direction, and apply the correction. The weights stay fixed during inference; the push changes with the state. We reconstructed the saved edit as an activation intervention and confirmed numerical agreement in FP32 on 40 fixed prefixes.
Fisher geometry gives a local measure of how much the output distribution changes:
KL(pₕ ‖ pₕ₊ᵤ) ≈ ½ uᵀFₕu
Here, pₕ is the next-token distribution at a fixed prefix, u is a small activation change, and Fₕ is the Fisher metric with respect to h. Along the tested edit direction, measurements on 32 evaluation prefixes supported the prediction at small strengths, but it lost accuracy at the original strength.
For an engineer, this suggests a workflow: try a correction with an activation hook, check whether shared weights can express it across inputs, then evaluate generation on held-out tasks. The same design questions apply across models, even when useful directions and gates differ.
The next question is whether an edited model becomes a better editor. Compare it with its unedited version on the same frozen targets, with the same tools, evidence and budget. Then test successive rounds against a fixed editor with the same accumulated history and total budget, using independent evaluations.
The reconstruction and local measurements are completed work. Choosing better edits with geometry—and testing whether one improvement helps produce another—is the research plan.
The attached Kabuki watercolor offers the intuition: change the current, or push the boat. The article takes that picture inside the model, then asks how a useful correction could help improve the process that finds the next one.
hayashiyus.github.io/blog/we…
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
もりし retweeted
来週の月曜日 (2026-10-05) に神田明神 (?!) でおこなわれる研究会にて、「言語モデルの内部表現には何が転写されているのか」というトークをします。まだ参加申し込み可能なようです。
10月5日に研究集会「AIとメタ認知」を開催します。人工知能の認識能力について、哲学者とAI研究者の両側面から研究成果を報告していただきます。
sites.google.com/view/ai-met…




















