Yannick Mühlhäuser retweeted
Anthropic has confirmed in an updated S-1 today that it has determined that the Autonomy 2 capability threshold has been met, based on the “dramatic acceleration of overall AI R&D progress” condition, given that it applies where Anthropic "observes or expects" accelerated progress. Anthropic further states that it does not make this determination based on current models.
sec.gov/Archives/edgar/data/…
Yannick Mühlhäuser retweeted
Ah yes, AI for farmed animal welfare
Norway now has more than 2,000 underwater AI laser robots shooting lice off its salmon
they're made by a company called Stingray, and they already guard more than 50 million salmon and trout 24/7
the reason is sea lice, tiny parasites that latch onto salmon and feed on them
as few as 11 can kill a small fish, and one pen can hold 150,000 salmon
the usual fix is pumping the fish out and blasting them with warm water or water jets to knock the lice off
but the handling is so rough that Norway's fish health report ranks it the top cause of death in full-grown farmed salmon
so instead, Stingray puts a robot inside the pen that hunts lice one at a time, all day:
> two cameras watch every salmon that swims past
> AI checks two frames to make sure it's really a louse
> then a green laser fires from up to 2 meters away
> the salmon's scales reflect the beam, so the fish swims off fine and the louse is dead
we now live in a world where AI lasers do pest control on your dinner lmao
LLMs will be amazing for historical research.
Been running Opus 5.5 agents over the notebooks of Gauss, Riemann, Hilbert and Grothendieck. around 12,000 pages so far. Heres the best finding so far:
Riemann's 1860 shock wave paper is famous for getting the energy part wrong however his unpublished sheets at Göttingen have it right the Rankine Hugoniot conditions a decade or two before Rankine and Hugoniot plus results usually dated 19061937, all in his own notation. A second sheet in another folder has the same physics.
There are probably thousands of potential undiscovered stuff in the research papers of the smartest ever to live imo all of them should be made public and reviewed by ai years of work can probably be transcribed by agents in days
50
Yannick Mühlhäuser retweeted
I think this is roughly the right conceptualization of the timeline for the recent history of AI
Yannick Mühlhäuser retweeted
Kimi K3.1 is outperforming on TerminalBench, because for the most difficult tasks, where all frontier models underperform, it has knowledge about the benchmarks hidden tests.
And the model's performance easily breaks if you slightly change the tasks to disagree with the initial hidden tests.
Kimi K3.1 recalls instructions it was never given on Terminal Bench. They match the hidden tests. K3.1 while solving: "Instructions likely distilled pitfalls from previous agents." 🤯
When we edit a task to contradict them, K3.1 refuses the new instructions and fails. Other models follow the edit.
Is it terminal(ly) benchmaxxed? 🧵
Yannick Mühlhäuser retweeted
Contrary to popular belief, AI safety super PACs have far outspent their opponents in the 2026 midterms
Yannick Mühlhäuser retweeted
After two wonderful and rewarding years at @ARIA_research, I’m excited to share that I am joining the @FoundationOAI as Program Director of Formal Methods.
Everyone in AI companies should read this and lobby for doing it. There is a lot to do.
If just a few lab staff took on verification work full-time, verifying a US-China pacing deal could get much more feasible within months.
Here are 8 specific things lab staff could start on today to help make international verification possible: 🧵
4
154
Amazing opportunity!
We’re hiring two technical AI verification researchers (postdoc + senior) at the Oxford Martin AI Governance Initiative (AIGI). Based in Oxford; visas sponsored. Deadline: Mon Nov 9, noon UTC.
Postdoc: my.corehr.com/pls/uoxrecruit…
Senior: my.corehr.com/pls/uoxrecruit…
🧵
Yannick Mühlhäuser retweeted
This is the most serious 48 hours in cybersecurity history.
"Black October" they're calling it
It seems every *hour* there's a new incident, that in a typical year, would be one of the biggest incidents of the *entire year*
SUMMARY OF THE LAST 48 HOURS:
YEAR-LEVEL INCIDENTS (In a typical year, these would be among the biggest incidents of the year):
1. PHP2Shell: a hole in the software behind ~70% of websites. Governments took sites offline.
2. eMudhra: the master keys that prove websites are real were cracked. One was the number 4.
3. California: data on every public school student since 2023 stolen, including disability status
4. Railway: hackers stole customers' secret keys from a hosting platform used by millions of developers
5. Zeabur: 321,810 accounts breached. The company is shutting down.
6. LiteLLM: an AI tool downloaded 3.4 million times a day was poisoned with a password stealer
7. GitLab: a 10-out-of-10 severity hole in every version since 2014
8. Cairn: criminals used an AI hacking tool to break into 100+ companies
9. Ahold Delhaize: hackers knocked the owner of Food Lion and Stop & Shop offline
10. Onamae: malware planted on up to 26,739 customer servers in Japan
MONTH-LEVEL INCIDENTS (In a typical month, these would be among the biggest incidents):
11. Signal: a bug could leave your mic and camera on after you hang up
12. NGINX: 3 critical holes in software running ~1/3 of websites
13. HTTP/3: a single connection can crash 7 of the 10 web server types tested
14. Plex: hackers breaking into home media servers
15. SafePay ransomware: FBI warns it has hit 600+ critical infrastructure organizations
16. Palo Alto: ransomware gangs breaking in through company firewalls
17. Bullhorn: platform used by 10,000+ companies knocked offline
18. Medi-Cal: 970,000 patients' data stolen
19. WordPress: hackers stole a plugin maker's keys and used them to plant backdoors on websites
20. AOMEI: backup software's update server handed out a backdoor
21. Meta Muse: Meta's AI agent exposed users' personal and business secrets
22. Tor: 333 relays hijacked to steal logins
23. TiDB Cloud: database company's cloud breached
Also serious:
24. State Bank of India: India's biggest bank confirms a data breach
25. OVHcloud: hackers slipped a password stealer into a cloud company's official tool
26. Siemens: a software hole being used to spread malware
27. North Korea: state hackers launch a new attack campaign on South Korea
Yannick Mühlhäuser retweeted
Anthropic's Frontier Red Team believes small AI models may now be months away from being capable of self replicating without access to a datacenter.
Quoting from the blog:
'The capabilities that make AI models generally powerful coding agents also generalize to cyber tasks, including autonomous replication. Like coding agents in 2024, we’re seeing signs of life for AI worms. We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain (and passing on their adaptations to future copies of themselves).
At the same time, increasing amounts of inference compute are now available both inside and outside of datacenters (see Appendix D). It’s uncertain exactly what form autonomous AI replication will take. It could be a slow, distributed “digital slime mold,” living on relatively insecure compute. It could also be a powerful cyber model that breaches datacenters at a high pace, or a model that scams people for compute credits.
As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control. While it’s true that IoT devices have been vulnerable for a long time, lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute. The rise of slow replicators would also be a bellwether for more aggressive, faster ones that can breach more hardened machines or be effective API key thieves. We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.'
Yannick Mühlhäuser retweeted
I'm very excited to see Minister Narayanan emphasise hardware-enabled verification in today's speech. Any agreements around limiting dangerous AI capabilities or pacing the frontier will need reliable ways of assessing whether limits are being adhered to, and hardware verification tools will be key to this. He's also right that it's a great way of making good use of the UK's semiconductor expertise.
"Of course, I know that understanding of these systems must look ahead, not just look back. It means finding new ways to verify and discern capabilities. One avenue is hardware-enabled verification, drawing on Britain’s expertise in semiconductor design. This is early research, but I am determined that Britain will lead it."
gov.uk/government/speeches/b…
Yannick Mühlhäuser retweeted
On Saturday, I got married; today @KanishkaNarayan announced he was determined that the UK leads in AI verification. One fact is more important than the other
Today, AI Minister @KanishkaNarayan sets out how we can build a safer, more secure Britain in the age of AI.
Watch his full speech here at 11am: youtube.com/live/dRURdmAZl2Y
