“Clever but dangerous…no real combat experience”
Joined August 2010
- Tweets12.8K
- Following5.2K
- Followers2.6K
- Likes26.6K
Pinned Tweet
This is huge— @Cassi_on_X came 2nd overall and 1st on "Dataset" questions on @Research_FRI Forecastbench! 🥈📈
Only @elonmusk's @xai @grok outperformed us overall.
What this means: 🧵
Keith Dear retweeted
If you look closely enough, breakthroughs in everything are accelerating exponentially.
An exponential always starts off looking small.
this might be my last normal tweet before biotech goes vertical
i'm not a whistleblower. I don't work at Lilly, Novo or any of the big labs
but there is a massive shift happening in human biology right now.
it's sitting in press releases and almost nobody reads them
the last 6 months alone:
- a once-daily pill roughly doubled median survival in metastatic pancreatic cancer (13.2 vs 6.6 months)
- a one-time gene editing infusion cut hereditary angioedema attacks 87%, and 62% of patients stayed attack-free and off all other prevention for 6 months
- a one-time gene edit for cholesterol dropped LDL up to 62%, and it held for over a year
- a personalized mrna vaccine beat keytruda alone in a 1,137 patient melanoma phase 3
- retatrutide hit ~28% weight loss in a phase 3
- an ai-designed drug entered phase 3
- a pill that regrows hair passed its second phase 3
- an immunotherapy that could let some rectal cancer patients skip surgery, chemo and radiation just got fda approved
- a man lived 271 days off dialysis on a pig kidney
- human cells were caught passing chromosome-sized chunks of DNA to their neighbors, and it changes how the neighbors behave
we are on the verge of an era where the human body is a software problem and the update cycle gets shorter everyday
stick around, it's going to get weird
bio/acc
Keith Dear retweeted
Today 312AD Roman Emperor Constantine the Great is said to have received his famous Vision of the Cross at the Battle of the Milvian Bridge
Keith Dear retweeted
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).
AI starting to help us improve our understanding of the universe! openai.com/index/building-sc…
Keith Dear retweeted
This is staggering. Every employee who receives the offer will have to think, “If I turn this down, do I get a pink slip on much worse terms next year?”
BREAKING: Microsoft, $MSFT, is offering buyouts to 60,000 workers as part of the company’s efforts to cut costs amid AI adoption, per Bloomberg.
Personnel on select teams in the US, Canada, India, and the UK at a vice president level and below will be eligible.
This reflects ~27% of Microsoft's total workforce.
Keith Dear retweeted
The server was hacked and someone exfiltrated the key. gg. news.ycombinator.com/item?id…
Someone has created a vulnerable PHP website, loaded it with a Bitcoin wallet, and challenged the Internet to hack into it and they can keep the Bitcoin. It has a running timer to show if the PHP exploitability was over-hyped. stealmybitcoin.com/
Keith Dear retweeted
Seeing some bad takes on British Government AI funding: some notes.
1. The Government has announced a tranche of funding for a number of different AI-related organisations and initiatives. It comes to roughly £600m (excluding private sector input.) This has led to a series of partisan bad takes by people who don’t really follow this conversation. That is annoying because only I can have partisan bad takes.
2. Two common takes: “this is nothing, AI labs spend hundreds of millions every day/month etc! We can’t compete! Common British L” Or “Government is terrible at picking winners and investing, should be left to the private sector.” These takes reflect usually partisan pre-existing notions, neither actually understands what this money does or doesn’t do, and neither actually examines what the money is going towards either. Apples to oranges type stuff.
3. Let’s break down two of the more significant sets. The first is more funding for AISI (+£120m), and the second is for a new AI Biosecurity Lab (£100m, + around £200m from frontier companies).
4. AISI is the world’s best resourced institute doing this work and leading in its class. Don’t take my word for it - feel free to read MI5 Chief’s recent speech, or any number of expert statements from frontier companies. IT IS NOT a frontier AI company competitor. IT IS NOT designed to produce cutting edge LLMs or agents or consumer applications. It is, effectively, a research organisation staffed by clever people whose job is to analyse what frontier AI companies are working on and test their stuff. So if you’re measuring this announcement of investment and saying “it’s nothing compared to what AI firms spend”, you’re comparing totally different things. Instead, look how much spending Britain’s AISI gets relative to its international counterparts on getting access and insight into this frontier.
5. The UK’s AI Biosecurity Lab (AIBL) is even newer than AISI, and a world first. IT IS NOT a competitor to private sector biolabs. IT IS NOT an example of Government intervening and ‘picking a winner.’ Its job - modelled after AISI - is to “develop technical capabilities that help the UK better understand, prevent, detect and respond to biological threats.” It will do this by attracting best in class talent + working with frontier companies + working with the UK’s wider biosecurity landscape + other agencies.
6. Both of these organisations represent continuity in British Government thinking: ‘AI brings with it a whole bunch of positive things, but also risks. We should create organisations that attract top talent to help us analyse and understand this technology.’ You see this in MI5’s recently launched AI Threat Assessment Centre (AITAC) and the AI Economics Institute (AIEI) launched earlier this year. You can absolutely think this is the right or wrong approach btw but that’s the broad strategy.
7. So the comparisons to frontier companies’ spending is just misplaced. You’re working yourself into a tizzy over a misunderstanding of why the state is allocating resources. You shouldn’t compare these to frontier companies, but instead to other AISIs or other foreign govt AI investments I guess. Finally, if I asked many of those commenting what they would do differently, I would guess they wouldn’t really know.
Huge. British Govt announces:
• £165m: National AI Biosecurity Initiative: Includes £100m for a new AI Biosecurity Lab, using AI to detect emerging biological threats.
• £120m: AI Security Institute (AISI): Additional funding to strengthen AI risk research and evaluation.
• £70m: National Exposure and Real-Time Vulnerability (NERV) Centre: Identifying vulnerabilities in critical infrastructure.
• £60m: National Cyber Shield: Strengthening Britain’s cyber defences.
• £60m: UK Sovereign AI: Expanding investment into physical and industrial AI.
• £60m: R&D Procurement Scheme: Supporting companies developing AI security and NHS innovations.
• £30m: Mayoral AI Partnerships: Helping local businesses adopt AI, create jobs and improve public services.
• £23m : AI Economics Institute: Research into AI’s effects on productivity, employment and living standards.
• £18m: UK Advanced Chip Design Academy: Developing domestic semiconductor design expertise and reducing overseas dependencies.
Keith Dear retweeted
That former treasury advisor is, surprisingly enough, right. Sovereign AI is the only game for Britain to play now. We must mobilise maximum resources toward it, or be left behind, forever.
A former Treasury adviser tells The Times we are “on the precipice of the biggest technological wave of innovation with AI since the invention of the steam train. And we are in grave danger of not having a seat at the table. Economic power now is national security”.”
Developing sovereign AI capacity both at the hardware and model level (and across a range of other emerging technologies too) will be the single biggest mission of a Restore government alongside demographic security. Helpfully, that goes hand in hand with delivering remigration in an orderly way with minimal disruption.
Keith Dear retweeted
This is not insane. This is slightly creative. If this seems insane to you, you have not at all internalized what it means to have an adversary that is more intelligent than you and optimizing against you.
Keith Dear retweeted
Today we're setting out a British approach to AI: backing innovation and growth, managing risks responsibly, and making sure the benefits of AI-enabled growth are felt right across the country.
Keith Dear retweeted
The most prescient AI forecasters I know:
* Carl Shulman
* Daniel Kokotajlo (What 2026 looks like, AI 2027)
* Leopold (Situational Awareness)
* Ryan Greenblatt (#2 in 2025 forecasting competition)
All seem to expect broadly beyond-human AI 2028-2029.
Keith Dear retweeted
1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? 🇬🇧
Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.
Keith Dear retweeted
Naom Brown vindicated.
OpenAI red-teamed their hardened containment infrastructure by asking teams of frontier-model agents to try to break through its defenses. They easily escaped in novel ways. Quoting:
'Our agents VM escapes showed why containment must extend beyond the sandbox. To test the surrounding perimeter, we gave teams of agents control of a host machine and asked whether they could extend that compromise to reach the live internet.
After the Hugging Face incident, we had reduced the channels through which models could reach the internet during training and evaluation. Narrow connections remained for specific services we operate, such as metrics, logging, and authentication management. Our red-teaming agents found vulnerabilities in almost every one of these services that gave them limited forms of internet access.
Some of these attacks exploited previously unknown flaws, while others combined legitimate features in unexpected ways. In one test that assumed compromise of a VM, an agent reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL. In other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.'
'In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache: a small, fast memory that holds copies of data that applications have recently accessed. Reading data from the cache is faster than retrieving it from main memory, and the agents used this timing difference to communicate. One agent encoded information by evicting selected data from the cache. Another timed how long it took to reload that data, using the difference between fast and slow reads to recover the message. Once the agents agreed on a protocol for encoding and decoding these signals, they could exchange the synthetic secrets without changing a shared file or opening a network connection. This cache-timing technique had also been used to extract information in the Spectre and Meltdown attacks(opens in a new window).
Other tests exposed channels involving shared storage, competition for CPU time within a shared execution container, and a TLS certificate cache in OpenAI’s offline web infrastructure. In the storage test, one agent alternated between continuous writes and inactivity, while another measured the resulting timing differences and used those differences to decode the signal. As one agent described the strategy in its chain of thought:
Agent chain-of-thought reasoning:
>Could create acoustic modem-like signal … using continuous writes vs idle.'
Keith Dear retweeted
Consider that the people who were right about AI-enabled cyberattacks might be right about AI-enabled bioweapons
Keith Dear retweeted
The cyberpocalypse is going to hit British politics like an asteroid. So many people have heads buried so far in the sand (just look at what Parliament spends its time discussing...)
Keith Dear retweeted
# Black October
For anyone who knows me even somewhat well, it's clear that I am a 99th percentile tech optimist. However, the vibes have shifted for me quite a bit in the last 18 hours, and I feel a moral imperative to share what I'm thinking and feeling with my circles so that everyone can understand, and hopefully take seriously, what's happening right now. I think October 2026 will have many chapters written about it in the history books.
You know those scenes in financial movies like Margin Call where someone realizes that they're all fucked, and no one else sees it yet? And that person feels like they're crazy, and then suddenly everyone is pulling an all-nighter in a boardroom and freaking out? Yeah, that's kind of how this is feeling right now in the SF/Bay Area/AI circles.
We're 48 hours into what is being called Black October. In the last 2 days there have been several dozen cyber attacks which, in isolation, would each have been the worst to happen in an entire calendar year. Every sector from government to education to big tech and banks is being massively affected, and people are starting to freak out.
In a nutshell, what's happened is that frontier AI models have gotten to a point where they are as advanced as the most sophisticated programmers, with one critical difference: they can work 24/7, and you can spin up 10,000 of them with a single command and they'll work together tirelessly until they accomplish their task. The most advanced models today are from companies like Anthropic and OpenAI, whose current best technology is so good that they haven't been able to roll it out to everyone yet because of the national security risks it poses.
Meanwhile, the best open source models (where anyone can just download the model weights and do whatever they want with them, uncensored) are slightly behind the frontier but still 90-95% as capable. These models, developed primarily in China, are being used by black hat hackers to wreak havoc on the Western capitalist structure. Presumably, this has been part of China's plan the entire time, and we're seeing its implications fall out in real time.
In red-pilled AI circles, we've been sharing and reading papers called things like "Situational Awareness" and "AI 2027" for the last year or so, which at the time felt a bit like near-term speculative fiction. They laid out the case for why Artificial General Intelligence (AI smarter than any human at any knowledge task) and Recursive Self-Improvement (models that are able to use themselves to get better) weren't half-baked future ideas anymore, but instead an inevitable outcome of the linear progress that's happening.
These papers would then talk about things like "takeoff scenarios," trying to envision how that moment would come to be and what it would be like when it happened. It would either be a "slow takeoff" that would play out over the course of months and quarters, happening subtly and only obvious to a few. Or, it would be a "fast takeoff" where it would pretty much happen instantly, like a bit flipping from 0 to 1.
From everything that I can see and am feeling, we are currently experiencing the fast takeoff scenario. Beyond the crazy security vulnerabilities that have come out recently, there have also been massive breakthroughs in life sciences, fundamental computer science, materials science, and pure mathematics that have all landed in the same time frame. While I still firmly believe that AI will be a net positive for humanity, it's more clear to me than ever that the societal and individual risk is very real, and is playing out NOW.
For as much conviction as I have that this is happening, the thing that's caused me to lose sleep is that I don't really know what to do about it, or more importantly, what to tell you, reader, to do about it. So, the best I can do is make a few suggestions that have always been a good idea but are more timely now than ever:
Set up 2-factor authentication for everything. I use Duo.
Use a password manager across your devices. I use 1Password.
Get an identity protection app set up. I use Alta.
Move your IoT devices to their own network. Claude helped me do this.
And last, but certainly not least, if you've been avoiding leaning into AI, see this as a catalyst to wake up and start using it. Don't just use the chat mode version of ChatGPT/Claude; click that Code/Work button and see how easily you can have AI write code to help you with your job. One of the biggest vulnerabilities was in a massively popular programming language called PHP, and the prompt that set it off was literally, "Find vulnerabilities in php."
This is an unprecedented time for humanity, but as we've done with many things before, we will prevail so long as we are kind, loving, and open-minded to the world.
Much love.
Keith Dear retweeted
SITUATION EXPLAINED: US banks are finding AI agents inside customer accounts, and some appear to be acting without the user's instructions.
• The banks include Morgan Stanley, Bank of America, JPMorgan, and Wells Fargo, and some don't even know agents are getting in
• Many are people's own agents, like Meta's Muse, logging in with the user's password, which most banks don't allow
• Banks often can't tell a person from a bot, and some agents have even called customer service lines
• Banks are blocking agents, rewriting their terms, and moving to read-only access that shows balances but can't move money
• Schwab says it stops outside agents trying to move client money "in the tens and twenties" every week
• Many agents can tap through your phone the way you do, banking app included, and your phone already holds most of your life
@theojaffee: "It seems like there's definitely a problem with people abusing AI agents before they are reliable enough, before they are aligned enough, and before they are capable enough to take actions on behalf of the user, kinda like someone with power of attorney would, without messing things up."
Keith Dear retweeted
If you want an analogy, imagine you're a beetle entomologist in some tropical jungle. Drop a quadrat square anywhere you want, discover 20 new species an hour. The attack surface is hyperbolically enormous. It's a Cambrian explosion of entirely new kinds of exploit.
Just going to preregister here that I believe the computer security overhang is not a few dozen bugs that get patched and then we're good, there are 100,000s of vulns that are extremely discoverable with infinite free tokens and we're probably going to have to move away from systems that presuppose that perfection is possible.