stochastic grief descent mechanistic interpretability @ LORIA
France
Joined June 2026
- Tweets206
- Following163
- Followers68
- Likes2.4K
kind of funny that normal people never understood how programming works re: explicitly needing to write every single behavior and a couple decades later they finally do then come LLMs which are designed specifically to oneshot that framing
3
229
shayan retweeted
honestly degrading the idea of “existential safety” — the biggest problem humanity has ever faced — by lumping it together with things like spam, “toxicity”, political unwanted content, deepfakes etc has been a huge disaster for ai safety and polarized a lot of people
shayan retweeted
we must know, we will know
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
exact thoughts! mind racing so many different things to do and achieve this is really the best time to be alive
I feel like my understanding of what a neural net is has been catapulted into the future by five years or something. it has brought back a lot of the sheer joy I had for studying the mystery of deep learning. neurons are already clean! they form a wonderful alien ontology!
1
82
oh wow this is insane
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
1
2
197
shayan retweeted
holy shit. i think this post is the single best piece of news i have seen all year
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
shayan retweeted
when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even eliezer will have to lower his pdoom. we can start making alignment into a 'normal science'
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
shayan retweeted
Pacing is out the window. For it to work, every American lab needed to be on board, Mark Zuckerberg has repeatedly made it clear that he is not interested. One defection among US labs breaks it all, as the others will be forced to accelerate or get passed. Mark has a staggering amount of compute and has been all in on compute capex for years; even in the face of some investor opposition he never stopped building his infra once. He said on the the earnings call a couple of years ago that he'd rather overspend now, and potentially even be wrong, than not have the compute when he needed it for the next 10x. He's finally hitting his stride, and he is not going to slow down.
For Team Pacing to solve a defection like this, they would have needed a supportive, pro-Pacing administration that was passing capability regulations and was willing to use the power of the federal government to enforce them and bring noncompliant labs into line. They don't have any of this. In fact, it's the opposite! Mark has Trump's ear, and President Trump, Mark Zuckerberg, and Jensen are completely aligned on policy; It's full speed ahead, and if you want to pace yourself, go ahead, but don't tell anyone else what to do. This breaks Pacing.
None of this changes until 2029 at the earliest, and to me, 2029 is past the ASI window. The last election, as it turns out, really was the most important one in history.
