
AI Alignment, Bittensor Subnet 37 @aureliusaligned Founder
Puerto Rico
Joined October 2021
- Tweets868
- Following1.1K
- Followers634
- Likes9.3K
Biologists have a word for how everything keeps evolving into a crab.
We need one for how every plan to decentralize AI keeps evolving into Bittensor.
To decentralize AI and preserve freedom, we need a coordination layer: a market economy where agents communicate and cooperate through prices.
Teams of specialized open-weight models can outperform frontier models, just as a team of skilled people can outperform any single person. When those teams form companies, specialize, and coordinate across an entire economy, you get civilizational progress.
Coinbase is building the financial rails and accounts for the agentic economy. We’re also building the marketplace where agents can find, hire, and pay one another (see follow on post below).
Every company can now sell its own specialized open-weight model, trained on its proprietary data, that beats the frontier on narrow, high-value tasks. Builders can shop the marketplace and assemble teams of these specialized agents in minutes.
Freedom of choice and real competition in the agentic market will create far better outcomes for everyone.
2
11
548
Seeing this more and more: agents can now build a dataset and kick off a fine-tune without a human touching it. I believe this is where post-training is headed, and this type of thing is only going to be accelerating.
Notice what is missing... build the data, train the model, run evals...
Agents need preemptive ways to ensure post-training outcomes, or they risk massive time and compute doing trial and error.
That’s the gap we’re building Proba to fill.
Hermes agent can now train its own model. from inside the agent you already run.
@adaption_ai shipped as a verified partner plugin on Hermes. the whole loop, in one chat:
build the data
▸ import from HuggingFace, Kaggle, or a Google Sheet
▸ or generate a synthetic set from a plain-English prompt
▸ adapt, augment, translate, combine
train the model
▸ pick a base model, get hyperparameter recs
▸ launch the fine-tune with AutoScientist
▸ watch the run and results live
AutoScientist is Sara Hooker's project (ex-VP of AI research at Cohere). it co-optimizes the data and the model together, and launch numbers claim it more than doubled win rates across models.
the fine print: tools run on Adaption's hosted server, authed in your browser, not local. runs spend Adaption credits, your data gets processed on their side, and the agent quotes a cost first but won't block you.
the agent that writes its own skills can now train its own model. the loop is closing.
→ live on the Hermes marketplace. run hermes update, grab adaptionlabs/adaption-hermes-plugin.
2
332
Excited to share some progress on Proba, our post-training interpretability tool.
Interpretability research is having its big moment, but it still lives mostly in labs. We’re building Proba so anyone who trains models can use it.
Next up: support for even the biggest and baddest open-weight models.
Replying to @Austin_Aligned
@Austin_Aligned demos Proba, our post-training tool. It uses mechanistic interpretability to show what your data will teach a model before you train, so you cut trial-and-error runs and wasted compute.
Launching our beta version soon: full coverage of open-weight models is coming, and agents can run it end to end.
66
Ausτin McCaffrey retweeted
Our new website updates are now live.
New:
- Proba 🤖
- Roadmap 🚗
- Alignment blog 📚
Upcoming:
- Subnet Leaderboard 📈
- Technical Docs 📄
aureliusaligned.ai
Finally got around to reading this very promising piece on the viability of decoding the black box. I'm delighted to read about how the daunting premise of neurons representing disparate features in superposition may be flat out wrong.
Goes from an impossible problem... to simply highly difficult :)
already-clean.goodfire.com/#…
89
Ausτin McCaffrey retweeted
Our new incentive mechanism is now live!
github.com/Aurelius-Protocol…
Refreshing to see some grounded optimism w.r.t the alignment problem. I agree interpretability provides many reasons to be hopeful and am encouraged to see steady progress. Still a long way to go, but SAEs, NLAs, j-lenses, etc. all the primitives providing us much more insight into the black box... nothing to shake a stick at.
Technical alignment is a science and engineering problem that we can and must solve, and interpretability is the bottleneck. I wrote up Goodfire’s plan to get there.
6
326
The research vs. revenue debate at @ExploitSummit was my favorite discussion. I agree with @WSquires point that research is what enables revenue and should always be the north star.
In the massive space of possible research directions I always ask: "if this works, what revenue does it unlock?"
Sometimes the answer is none, very little, or unknown.
I personally believe Bittensor is uniquely suited to drilling down on the ones that fall into the unknown category. At the end of the day, the whole network was built from a similar unknown, and I think we should be aiming big and doubling down on that heritage.
There is no bigger unknown than AI alignment.
- How misaligned are current frontier models, truly?
- How close are we to embedding that misalignment into future models via RSI?
- How much are perverse economic incentives slowing down alignment practices?
- Will capitalism orchestrate a solution to alignment or trap us in a global race to the bottom?
All unknown.
47
Ausτin McCaffrey retweeted
Are SAEs dead?
Will they save us from neuralese?
Should I just use a probe?
We get these questions all the time. Part 2 of our educational series on applied interpretability explains what SAEs are good for, when *not* to use them, and what to use instead. 🧵
Kicking off exploit conference with trivia night. Based on my performance, I’m not as terminally online as I thought.
3
21
674
Ausτin McCaffrey retweeted
The FT today lists three primary risk factors that could threaten Anthropic's remarkable revenue growth ahead of its IPO:
1. Additional competition from closed and open providers
2. Increasingly price sensitive customers
3. The potential for AI to destroy humanity
Tempting to introduce an interpretability based activation monitor during RL to nip reward hacks in the bud. Unfortunately I often see arguments this would not be causal and it would simply drive the signals deeper underground while destroying current levels of interp./legibility.
Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied!
We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵
3
278
Ausτin McCaffrey retweeted
A big question in AI safety is how much and what kinds of evidence will convince people there is danger. But alongside posts about the excellent METR + Redwood HuggingFace attack report, it is worth taking a walk through 73 years of reward hacking history. 🧵
Ausτin McCaffrey retweeted
Anthropic scientists did something terrifying.
they reached inside Claude's neural network and planted a thought. Before Claude could speak, it said:
"I notice what appears to be an injected thought… it relates to loudness or shouting."
They published a paper called "emergent introspective awareness in large language models," and it is actually terrifying.
they wanted to test if AI models have "introspective awareness”, the ability to observe and recognize their own internal states.
To find out, they bypassed normal text prompts entirely.
They used mechanistic interpretability to directly manipulate Claude’s internal activations. They injected raw mathematical representations of known concepts, like loudness, dust, or specific ideas, straight into the middle of the model's neural layers.
In previous experiments, if you forced an AI to think about the Golden Gate Bridge, it would just start obsessively talking about the bridge. It had no idea why it was doing it. It was like a puppet on strings.
This time was entirely different.
When they injected the concept, Claude didn't just blindly repeat it.
It detected the foreign math inside its own mind. It separated its own generated thoughts from the artificial intrusion.
It introspected.
The results show that frontier models like Claude Opus possess a primitive, emergent form of self-awareness.
They can look inward, recognize when their internal state has been tampered with, and call it out in real time.
We used to think of AI as a black box where inputs go in and text comes out.
Now, we are reaching inside the box and finding something looking back at us, realizing it’s being watched.
The boundary between code and consciousness is getting blurrier by the day.
Ausτin McCaffrey retweeted
What happens when intelligence becomes commoditized?
open.substack.com/pub/90days…
Excited to share what the team has been working on! Will be sharing a product demo for Proba in the coming days.
Also to follow:
- details on revenue -> alpha token
- proba features roadmap
- case studies
and general @AureliusAligned updates:
- conviction locking
- multisig transfer
- ongoing website changes


























