Cryptography, personal trivia, and the future of all humanity. I block for rudeness or sarcasm.
Scotts Valley, CA
Joined November 2008
- Tweets90.9K
- Following1.5K
- Followers9.4K
- Likes209K
Paul Crowley retweeted
Jesus taught us to count years up instead of down, which was a huge deal, because right before he was born we were down to our last year.
Paul Crowley retweeted
This is a specific sore point from lived experience.
I had Samsung devices that would or could not capture a true likeness of a child's skin rash during a severe allergic reaction.
It effectively airbrushed for beauty, ruining the opportunity to collect evidence for the MD.
Paul Crowley retweeted
This is why I think debates are the worst format for comparing and contrasting viewpoints, it benefits the dishonest in a way that can't possibly be evened out.
Paul Crowley retweeted
Important, and I'm glad this is finally public. Google DeepMind researchers are under in perpetuity non-disparagement clauses forbidding them from ever speaking critically of the company. I'd already confirmed this with multiple people.
Yes, some ex-employees have spoken out nonetheless. But it has also discouraged others from doing so. And yes, it most certainly has a chilling effect with regards to discussions on safety and risk.
Others have written compellingly about how important the ability to speak freely and critically is, especially on topics like safety and risk. And what a testament to the research employee community's coordination and hard-won bargaining power it has been to get there at other companies.
GDM burn that commons by having this. And everyone who signs burns that commons too. Let's get it changed.
Congrats to Sofia for having the courage to resign and speak out!
This is also very important:
Staff are scared to speak out about AI safety and its uses in warfare because their employment contracts prohibit them from publicly criticising the company.
bbc.co.uk/news/articles/cmre…
Paul Crowley retweeted
The problem with using obscure philosophical vocabulary to sound smart is that every once in a while you use a word wrong and end up looking totally epistemological.
Paul Crowley retweeted
Claims:
Game detects “ad whales,” then periodically gives them smooth runs until they barely beat their high scores. Right at that point, it kills them deliberately and offers a revive if they watch an ad.
$100 million annually in revenue. Hellish.
Paul Crowley retweeted
Maybe I shouldn't say this out loud, but if I was an economist worried about EA false flag cybersecurity incidents, I would fist a grizzly bear. Sure, the bear will maul you, but what could be higher expected value than raising awareness of EAFFCSI-risk with your gruesome death?
unavailable
Paul Crowley retweeted
I am once again asking people to not do the Ozymandias plan from Alan Moore's famous dystopian graphic novel "Don't Do the Ozymandias Plan"
Paul Crowley retweeted
seeing a lot of this going around lately
Maybe I shouldn't even say this out loud. But it seems to be that if one is an EA doomer, the principle and belief consistent action is to us AI to undertake a controlled but still scary hacking incident in order to show the world the risk.
Clearly no amount of Eliezer Yudkoswky essays are going to convince the world of the magnitude of the risk. So why not do something like a false flag cybersecurity incident?
Sure you will go to jail, but if the world is probably ending you probably won't even see a cell. And aren't they focused on taking actions with high public expected value? This seems like exactly that sort of thing.
Obviously there are some EA doomers making their best contribution to risk reduction in other ways. But what could be higher expected value than waking society up to the risk six months early? Surely there are some willing to sacrifice themselves for this?
Again, this is my saying what seems like follows from their beliefs, not mine.
Paul Crowley retweeted
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
Paul Crowley retweeted
Many people with relevant expertise think there's a real chance that AI could kill us all in the not too distant future. Others disagree. I've written a post, aimed principally at mathematicians, detailing my struggles to decide what to make of this.
gowers.wordpress.com/2026/10…
Paul Crowley retweeted
There's a story going around: after the drone strikes on Yandex data centres, comments on AfD and MAGA accounts collapsed.
We monitor 17,000+ politicians, media and civil society accounts in 30+ countries. 50,000+ posts a day.
So we checked.
We don't see it. 🧵
Paul Crowley retweeted
One of our most surreal experiences in testing Gemini 4 was when it directly addressed us within its chain-of-thought
Literally saying "hello evaluators reading my chain-of-thought"
My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time.
We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations
I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model.
It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
Paul Crowley retweeted
"[those traits] will matter whenn you start displaying them"
"an enormous amount of misunderstanding could be alleviated byy such a mechanism"
"I defend those statements onn the basis of free speech"
Paul Crowley retweeted
Nate Soares says the real question isn’t how AI would kill us.
It’s when it would no longer need us.
“How do these machines become self-sufficient?”
“Right now, we build the data centers, we build the energy network, we keep it all running.”
“To kill humanity, it’s suicide for machines. We’re the ones who are keeping all the machines running.”
“Elon Musk talks about the infinite money glitch of building factories that build robots that build factories that build robots.”
“From some perspective, AI, its incentives are to play nice until the fully automated economy is up and running.”
“And then sort of be like, thanks, I’m taking this whatever direction I want.”
“My guess is that at that point, it’s pretty easy to just kill all humans at once with some manufactured plague.”
@So8res
Paul Crowley retweeted
A minimal requirement for pacing the frontier: show that you can adequately align your current model class before you start training a more capable one. lesswrong.com/posts/ZWSstGSw…
Paul Crowley retweeted
You know, it's crazy how they couldn't get any of their actual research leaders to put their names on this "note from our research leaders".
A note from our research leaders:
Last week we parted ways with Jasmine, Mikita, and Tomek after a thorough investigation found they violated clear policies on handling sensitive information. Our internal investigation uncovered a significant breach of trust beyond what’s outlined in the letter they published and we stand by the decision to not continue their employment. We generally keep individual employment matters private and don't believe a back and forth would be productive or lead to a resolution, but we want to address the points they raised in their letter directly.
- We want to be very clear that these decisions were not about raising safety concerns or speaking out. Safety and research debates happen every day at OpenAI, often spirited and highly critical. We actively encourage these discussions and consider them essential to making the right decisions. We cannot do the work in front of us without a high degree of trust. We will continue to be extremely forgiving of our team making good-faith mistakes. We have not and do not terminate any of our employees for raising concerns.
- We are actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks. People across the company have been working really hard on getting these partnerships up and running. We are committed to embedding external assessors and continue to make close collaboration with independent safety organizations a core part of our safety work. Many of our researchers already work with 3p safety organizations productively.
- We agree with the letter that preserving the monitorability of frontier models requires an industry-wide commitment, including from OpenAI. Monitorability has long been a core piece of our research program, and something we continue to invest significant resources in (see our publications on Monitoring Monitorability and the subsequent open sourcing of monitorability evals, our system card for GPT-6 Astra, Jakub’s blog and post on X, and the numerous blog posts on our Alignment blog on the topic).
We are deeply sad about this outcome. We appreciated Jasmine, Mikita, and Tomek’s contributions to AI safety at OpenAI and their willingness to speak up and challenge ideas. We championed their voices, supported their work, and placed enormous trust in them. These decisions were not about them raising safety concerns. We have always encouraged that and always will.