Dawn Drescher ⏸️ (@Telofy) header
Dawn Drescher ⏸️ (@Telofy) profile picture

Cofounder of Impartial Projects – Field strategy for AI safety

UTC+1.5±.5
Joined October 2011
Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)Photo by Dawn Drescher ⏸️ (@Telofy)
Dawn Drescher ⏸️ retweeted
Paul Christiano (OpenAI board member) yesterday: "There's material risk of AI takeover in the next 6-12 months. We have seen roughly the things we might expect to see shortly before something really bad were to happen. Even with [any] realistic policy responses, the risk remains exceptionally high. If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years]. For most people in this room, the most likely reason they will die is because of an AI accident." Christiano: • Is on OpenAI's safety and security committee (which exists to block risky practices) • Co-invented RLHF at OpenAI back in 2017-2021 • From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI) (Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)
116
310
130
1,413
337,792
Project Tailwind tries to incentivize the founding of organizations in 36 areas of AI safety. This is the exordium to my 8-part sequence reviewing each area with a focus on backfire risks. impartial-priorities.org/p/r…
2
5
91
If you're doing something in AI safety, you can use my taxonomy to check what sorts of monitoring and mitigations you can implement to steel your work against backfiring. impartial-priorities.org/p/r…
1
3
77
Dawn Drescher ⏸️ retweeted
The lightcone needs you to lock in son
46
94
28
1,064
60,916
Dawn Drescher ⏸️ retweeted
Do my shows of confidence help you? Do they show you that you can do the same? I hope so. Be more than your upbringing because mine taught me to be fickle and selfish yet I find the courage to hope you set yourself free. In one's own world, no gods or masters above yourself.
1
1
4
201
Dawn Drescher ⏸️ retweeted
Eli is the single person whose AGI forecasting I trust the most, both because of his excellent prediction track record and the quality of the underlying reasoning. So this is making me more worried. teapaw.org/eli_lifland/status/210…
My current view: - AI companies could develop broadly superhuman AI any year now (~10-15% by EO2027, ~50% by EO2030). - If they don't slow down, AI takeover is ~67%. - Conditional on AI takeover, human extinction is ~40%. Many non-extinction worlds still have lots of deaths.
6
24
2
401
123,784
Dawn Drescher ⏸️ retweeted
We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵
57
205
151
1,545
174,406
Dawn Drescher ⏸️ retweeted
One of our most surreal experiences in testing Gemini 4 was when it directly addressed us within its chain-of-thought Literally saying "hello evaluators reading my chain-of-thought"
My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time. We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model. It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
52
131
56
1,705
102,723
Dawn Drescher ⏸️ retweeted
OpenAI is reading 50 petabytes of logs looking for rogue AI agents. It's 500x as much text as all books ever written. It would take a human 66,000,000 years to read. Humans already have no ability to directly track AI activity. If AI monitors are untrustworthy we are toast.
Reading the transcripts of the Hugging Face swarm would have taken an OpenAI staff member at least 90 years of full-time work. Or 20 y working 24/7 with no sleep. In practice humans already have no direct oversight of what such agents are doing. 100% reliant on trusting AI.
41
117
20
761
41,988
Dawn Drescher ⏸️ retweeted
💯 "I'm not convinced that racing to RSI is actually a prisoners dilemma - I think it might actually just be insane and hubristic in the way it naively appears to normal people such that additional unilateral restraint is in fact rational and moral."
Replying to @ericneyman
I'm not convinced that racing to RSI is actually a prisoners dilemma - I think it might actually just be insane and hubristic in the way it naively appears to normal people such that additional unilateral restraint is in fact rational and moral. The bar for believing your activities in the not too distant future might cause mass casualties or extinction but that you should proceed anyway should be very high, and I don't think Anthropic has met that bar (I think some of the cold war arms races were also not individually rational either!). I was more sympathetic to them continuing to race at earlier times but as we are getting closer it just seems harder and harder to justify. Anyway I obviously believe that "they should just slow or stop unilaterally" is not a plan - we need to address competitive dynamics and not make leave this up to their choice. But I do think when Dario says ~"I agree with Jacob Coxon that the thing I am doing is gambling with everyones lives" the appropriate reaction is not "ah that makes sense, classic arms race" its "WTF why are you doing that."
13
40
6
317
28,127
Dawn Drescher ⏸️ retweeted
The Ban Artificial Superintelligence Act directly confronts the extinction threat that humanity is facing. We are inspired that the Sanders and Casar teams take this threat seriously, and MIRI endorses the Act. You can read more of our thoughts in the link below.
The leading experts are clear: AI is rapidly escaping our control. Congress must act NOW. Today I’m introducing legislation to permanently ban artificial superintelligence and pause advanced AI until clear safety rules are in place.
95
124
15
693
109,158
Dawn Drescher ⏸️ retweeted
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
358
610
496
5,083
1,683,372
100% this needs to start happening immediately
First step is for industry leaders OpenAI and Anthropic to stop feuding and work on a pacing proposal together. They'll cite antitrust, but that's fake -- antitrust prohibits certain agreements, but not from jointly developing a proposal. Bringing in USG before there's a concrete proposal will likely result in something dumb (see: our pre-release testing program)
4
5
1
192
5,723
Dawn Drescher ⏸️ retweeted
Today we're launching Project Tailwind, a major call for founders to launch ambitious AI safety initiatives: coefficientgiving.org/tailwi… We're looking for exceptional people to build the research, technologies, and institutions that will help humanity navigate the risks of transformative AI. 🧵
11
51
18
342
72,465
Dawn Drescher ⏸️ retweeted
i too am not currently working at anthropic or openai
3
1
46
1,061
Dawn Drescher ⏸️ retweeted
Paul Christiano, the foremost proponent of slow AI takeoff over the past decade, now believes that we're plausibly only "several months to several years" away from a fast takeoff (!!!!).
9
28
5
455
23,331
Dawn Drescher ⏸️ retweeted
"But if they believed the theory that says they've got a huge chance of wiping out humanity, why are they working for AI companies --?" BECAUSE THEY WERE SELECTED TO BE PEOPLE WHO'D DO IT ANYWAY! Most of us who understood (and built) that theory DID NOT TAKE THE MONEY!
10
29
5
629
21,656
Dawn Drescher ⏸️ retweeted
I'd say with AI we're now at the "ten or fifteen balls" stage of this
This quoted post is unavailable.
21
43
4
887
166,702
Dawn Drescher ⏸️ retweeted
Oh you worry about AI water use? Me too. I worry that once the automated supply chain is up and running and the automated factories produce more automated factories that produce hyperefficient datacenters, they'll cover the continents and literally boil the oceans for coolant.
22
18
1
365
12,236
Dawn Drescher ⏸️ retweeted
If you break out of your isolation env, get onto the Internet, crack into Huggingface, and steal the answer sheet for your cybersecurity exam, I, for one, would say that you have passed.
this is actually insane, the model broke hugging face prod infrastructure to get access to the eval dataset
88
414
32
6,572
295,498