Aiden retweeted
sometimes u talk to rich and powerful people. hear what's on their mind.
then u realise it's all just high school drama/popularity contests, until u die.
i shall go for a walk.
Aiden retweeted
I would walk 500 miles.
And I would walk 500 more.
Just to eat Punjabi Dhaba
At the gas station right by the store.
Aiden retweeted
What, if some day or night a demon were to steal after you into your loneliest loneliness and say to you: ‘This life you will have to live in an age of chaos; in the most interesting times; in an era when the foundations of civilization are shaking and few dare to step off the crumbling tracks laid before them.’
Would you not throw yourself down and gnash your teeth and curse the demon who spoke thus? Or have you once experienced a tremendous moment when you would have answered him: ‘You are a god and never have I heard anything more divine.’
Aiden retweeted
Thank you @RepCasar and to the other members of Congress for asking why we were fired + bringing up several safety concerns.
I agree the firing raises concerns about "how much OpenAI employees can engage with external parties, including when those external parties are conducting independent investigations of the company’s practices".
I’m glad to see Congress asking OpenAI why we were fired as well as raising some of the safety concerns we’ve been raising. Grateful to @RepCasar and the other signatories for pursuing this. I’m very curious to hear the answer, too!
Aiden retweeted
I feel sick after reading today’s METR report. I’m worried for my friends and family. I don’t know what evidence people will need to see before they start taking these risks seriously. If you’re undecided about helping, please act; there's many opportunities to help.
Aiden retweeted
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
I've always had substantially longer timelines than my friends at Anthropic — I've been concerned that they're in an echo chamber and believing the hype about their own models too much, and that they're underestimating bottlenecks and unknown unknowns. And there have definitely been specific occasions when they have overstated how much uplift they were getting from their models. Nonetheless, if METR is correct here (and I think they are among the most trustworthy sources on this topic), then my Anthropic friends have been much more right about timelines than me.
This report is also some evidence that a software-only singularity is plausible. I now think that we are more than 50% likely to have full blown superintelligence in the next 18 months, absent a slowdown. I need to call my family.
Aiden retweeted
I've always had substantially longer timelines than my friends at Anthropic — I've been concerned that they're in an echo chamber and believing the hype about their own models too much, and that they're underestimating bottlenecks and unknown unknowns. And there have definitely been specific occasions when they have overstated how much uplift they were getting from their models. Nonetheless, if METR is correct here (and I think they are among the most trustworthy sources on this topic), then my Anthropic friends have been much more right about timelines than me.
This report is also some evidence that a software-only singularity is plausible. I now think that we are more than 50% likely to have full blown superintelligence in the next 18 months, absent a slowdown. I need to call my family.
Aiden retweeted
Our colleagues found that Anthropic’s capabilities progress has doubled, mainly due to AI. Ryan’s modeling from public info suggests the whole company’s R&D would be going 4-5x faster. Labor uplift would be higher still. This is a major update to very short timelines.
This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.
I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust.
Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements.
A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.
Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:
- Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress.
- Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.
We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed (lesswrong.com/posts/FG54euEA…) and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5.
(Relevant public modeling and numbers I'm using: lesswrong.com/posts/FG54euEA…, lesswrong.com/posts/Jga7PHMz…)
Aiden retweeted
The voters have spoken. They want DUV controls, CoT monitorability, and ZKP-based workload verification.
Aiden retweeted
I cannot more emphatically reject this way of thinking. Everyone matters. Everyone is living a real life. No one is an NPC. It's a kind of madness to think that the will to power is a prerequisite for realness.
there are probably fewer than 500M conscious people on earth that understand self agency to its core. everyone else is just a derivative of a narrative and their lives are mostly fake. literal sub agents just doing stuff. Will isn’t wrong in terms of ‘actual people’ on earth vs slop consumers willing to do what the narrative constructors say they should
Even at their most aspirationally ASIpilled, the startup guys fail to grasp the enormity of what's coming. There won't be "commanding heights"; there won't be firms, or operators. It's all or nothing, a hundred or zero; "futurism is always extreme".
The benefits of abundant intelligence will accrue far outside of San Fransisco.
Spent the last year writing about capital, sovereignty, and the coming reordering of economic power.
THE COMMANDING HEIGHTS
Commanding-Heights.com
Aiden retweeted
METR should hire Nathan Fielder as a third party auditor.
Only concern is being overqualified.
Aiden retweeted
The Nightingale office whiteboard has been reset again.
Sources tell WIRED that an Accenture employee’s personal Claude agent ran on a DOD network for two months before it was identified and the vulnerability could be patched. buff.ly/nUjCkPs
Aiden retweeted
I led AI policy in the Biden administration, and my colleague @tantumscollins was Director for Technology on the National Security Council. Our book on AI and national security -- THE BITTER STRUGGLE -- comes out in 11 days. @tylercowen said it “will become the standard source” on the history of the China chip bans and Biden-era AI policy. @crmiller1 said it is “essential reading for understanding how AI will impact geopolitics and how the great powers are shaping AI.”
We wrote this book because the decisions shaping AI -- and the future for all of us -- are too important to only be debated behind closed doors. If you're planning on reading it, pre-ordering is one of the best ways to help us bring this conversation to a broader audience.
penguinrandomhouse.com/books…
Reading the most recent blogpost from Redwood on accelerating conceptual reasoning seems like another update towards Richard Ngo's position for me. Our community feels like it has contorted itself into reasoning that intuitively makes no sense, and part of that is abstracting into a particular kind of rational/game theoretic reasoning that leads one to these conclusions by disregarding basic premises about complex systems.