Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI Previously - AI safety researcher @ConjectureAI, UIUC Physics DMs open!
Joined May 2024
- Tweets1.7K
- Following676
- Followers2.1K
- Likes11.5K
Max Winga retweeted
It is pretty rich of you to cite "others could go faster" as a reason to redact so much important information when you yourself do not appear to be slowing down much at all. Nor are you throwing yourself behind efforts to pace the frontier with nearly the gusto that you would if it was your top priority. Not to mention "commercial sensitivity" you yourself have said human extinction is a plausible outcome why are you even talking about commercial factors (well, I guess honesty is good, and since it IS apparently a factor in your decision...) To all my friends at Anthropic: Your company appears to be recklessly gambling with our lives. You fancy yourselves the more responsible alternative to OpenAI (as they fancied themselves compared to Google) but judging by actions rather than words the similarities between the three of you seem stronger than the differences.
Max Winga retweeted
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
Max Winga retweeted
This is not what it looks like when there's just one or two more alignment issues to fix before it's okay to begin a recursive self-improvement loop. This is the sort of emergent surprise that signals that you have no idea what's going on in there, and that it's time to back off.
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
Max Winga retweeted
This is really bad and people should be freaking out about it.
We're probably all literally going to die in the coming years. We shouldn't just accept that and move on.
We at AIFP are spooked by this information. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027. We are thinking harder and longer about this now and will hopefully have a lengthier update on our thinking to share in the next week.
Max Winga retweeted
The cyber stuff is baked in at this point. A lot of damage will be done. But it's not too late to stop the bio and the RSI pieces.
The way to stop is to halt frontier AI development. This will be hugely costly. But the cost of continuing will be greater. That much isn't obvious just yet. It will be over the next year or so.
Max Winga retweeted
There's a decent chance the internet ceases to exist as such, at least for a time.
AI-enabled bioweapons are also coming, and there's no "shut down the websites" option for that.
We're approaching the point where frontier AI R&D becomes politically untenable.
Vulnerabilities in a programming language that supports much of the internet are so severe that European and British authorities recommend websites temporarily go offline until they can be fixed. nbcnews.com/tech/security/cr…
Max Winga retweeted
I'm using the critical PHP vulnerability as an opportunity to issue my periodic reminder that AI is not only becoming more capable in offensive cyber capabilities but also in offensive bioscience capabilities. It's way easier to patch software than humans.
Max Winga retweeted
And the problem is that current third party auditors are those who took this proverbial deal with the devil.
This means we should trust them much less, even if audits became mandatory and no longer voluntary.
People worry a lot about third-party auditors being biased because of where they get funding or who their employees are friends with. IMO by far the biggest source of bias is that if they upset an AI company, that company will stop letting them audit.
When I was at METR this dynamic was a way bigger effect than everything else combined. Of course no auditor can avoid this until audits are no longer voluntary.
Max Winga retweeted
Thanks! We've applied. Can't fit all the wins in 280 characters, forgive me :)
We actually have over 200 lawmakers on board between the UK (180) AND Canada (38) campaigns as a result of over 400 direct briefings with lawmakers themselves in the US, UK, Canada, and Germany (+ just started EU-focused outreach this week). We've also met with over 250 congressional offices.
Our bill to UK ban ASI was the first such bill to be introduced anywhere in the world in September. We also consulted on the second ever bill (Sanders-Casar). You may notice similarities between their bill and our discussion draft we published:
drafts.controlai.org/asi-ban…
We've also been involved with killswitch legislation in the US and UK (not a solution to x-risk though).
On top of creator partnerships getting millions of views. Connor Leahy has gotten 10M+ views across podcast appearances in the last few months, and our media team has helped get over 125 tier 1 and 2 articles covering AI extinction risk / ban ASI in the last year.
Everything we have is built and proven to scale across countries. We're entirely bottlenecked by funding and good talent.
When we started everyone said this was impossible. You can't talk to lawmakers about this issue. Banning ASI is impossible.
We've proved them wrong.
All it requires is laser focus on banning ASI (it's the only thing we care about) and a relentless drive to iterate, optimize, and scale bigger.
Max Winga retweeted
🗣️ 'We are kind of running out of time.'
CEO of ControlAI, Andrea Miotti, issues a stark warning about artificial intelligence, telling @cathynewman that 'more needs to happen' in government to implement safety measures quicker than it is currently moving.
Max Winga retweeted
This is HUGE! In an open letter to the Royal Society, a group of 42 extremely distinguished mathematicians who have greatly contributed to to their field, presents a stark warning about the capabilities and risks of frontier AI.
Hundreds more have later addet their name to the letter.
1) confirms frontier AI now at level of leading mathematicians in many fields of maths, and that this capability progress happened over the last few months. Warns of plausible superhuman level of mathematics capability in relatively short time.
2) warns if similar capability leaps in other fields that could pose more imminent risks.
3) states that warnings of imminent existential risk is not to be dismissed as hype.
Open Letter to Sir Paul Nurse, President of the Royal Society: proofsandprompts.com/2026/09…
Max Winga retweeted
What should AI oversight look like?
“First and foremost, we must stop the development of systems that we cannot control and that risk humanity's extinction."
ControlAI's US Executive Director Connor Leahy (@NPCollapse) on ABC News:
Max Winga retweeted
Former Chief Scientist of the UK's AI Security Institute @geoffreyirving says the chance of human extinction from superintelligent AI is about a "coin flip", in his view.
Irving previously led safety teams at OpenAI and Google DeepMind.
He says this estimate is after taking account of the possibility that we take action to prevent the threat.
In other words, if we don't act, our chances are even worse.
Max Winga retweeted
"If you care about whether human beings continue to exist, then you should care about banning superintelligence."
Professor Stuart Russell, author of the authoritative AI textbook, on the eve of @alexsobel's introduction of our bill to ban superintelligence to Parliament.
Max Winga retweeted
Want to do something about the threat posed by superintelligent AI? Apply to join ControlAI's team!
We're hiring a Policy Advisor to work closely with me briefing members of Congress. This role is DC-based and ideal for someone with national security or Hill experience...
Max Winga retweeted
Here's a writeup of everything we know about the attack as of now, including new evidence it may have been part of a rogue AI swarm.
blog.controlai.org/p/rogue-a…
Max Winga retweeted
You all realize that the Hugging Face attack and this happened months ago, there's probably even more happening right now that we're completely in the dark about.
This will only going to get worse if the models get more autonomous, capable, and resistant to oversight.
Exclusive: A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research reut.rs/4gJ7FPG
