Trying to reproduce the developmental program behind brain connectivity. Website: jamiesonwarner.con
San Francisco, CA
Joined May 2015
- Tweets140
- Following315
- Followers68
- Likes935
Jamieson Warner retweeted
Interestingly, the gradient properties arising from shift invariance of softmax(teapaw.org/miyashita_03/status/21…) and scale invariance of a network(teapaw.org/miyashita_03/status/21…) can be derived using exactly the same steps.
Both start from a symmetry: transforming the input or weights by a constant "c" leaves the function's output unchanged. Then differentiating with respect to "c" reveals the constraints on the gradient.
- Shift invariance: Gradient components sum to zero.
- Scale invariance: The gradient is orthogonal to the weights.
These next ~12 months sit at a golden opportunity before AI tools have replaced us as researchers entirely, but while our access to next-generation automated coding can vastly accelerate any idea you have and bring it to life.
ES seems to work pretty well in high dimensional search environments. You can just solve for good operating reasoning behaviors in the region around the pre-trained model, and this lets you locate solutions to long-horizon task environments typical in RL.
Ok guys so all the recent weight-space programming buzz inspired me to write a blog post on zeroth-order, weight-space methods (back to the basics), from the theory to a bit of update on our latest result.
Turns out plain weight perturbations don't just work for fine-tuning on math, they work for post-training agents too: vanilla ES takes Qwen3.5-4B from 12.1% → 21.1% on Terminal-Bench 2.1, past the released Tmax RL checkpoint (18.2%).
Better yet, no credit assignment, no critic, no masking, no messy hyperparameters, just good ol' perturb-and-update. Oh did I forget to mention that all with population size of ~30?
Here's why I think this could be huge for long-horizon agentic post-training 🧵+blog post👇
Humans will continue to commune with the super intelligence, who will understand our mental limitations and deliver a firehose of relatable concepts. The research work will be done by AI, but a select few “ascended” humans may choose to lose themselves in connection to it.
Jamieson Warner retweeted
Biology has been doing computer science for four billion years. Before the discovery of DNA, Alan Turing and John von Neumann worked out the logic of self-replicating machines, showing how matter folds into a system that reads and executes instructions to build itself.
Science is fundamentally about tool creation and use. Design an experiment (a measuring tool) to measure a variable.
Jamieson Warner retweeted
How do you build a brain from a genome? We know a lot about the mechanisms and molecules
Here we revisit the algorithmic problem: What kinds of programs can specify a brain within the genome’s information budget and the finite time available for development?
Neuroscience, cognitive science, and psychology are like machine hardware, kernel development, and software.
Everybody has their own way they use codex or claude tools for coding. This setup works for me because it's parallel (fast) and manages reports. I like seeing both big updates in reports and also the finished experiments.
🤖 Made with AI
