@Butanium_

Astra fellow w/ Owain Evans ex MATS 7/7.1 Scholar w/ Neel Nanda and intern at DLAB (EPFL) AI safety research / improv theater

butanium.github.io/
Joined December 2018
  • Tweets1.3K
  • Following761
  • Followers1.2K
Replying to @Karim_abdelll
showing OPSD where the feedback is an extra turn confused me a lot Had to ask claude about this as I didn't see this when skimming through the paper
1
1
43
Replying to @maxsloef
this model is very intense
7
582
I can't even talk about this with opus 😔
1
4
111
another example: taking the Mythos incident CoT and sanitizing it with Kimi until it doesn't trigger the cyber classifier then triggers the reasoning extraction one when asked to classify it as concerning or not!
1
4
150
This is so easy to trigger (literally first try): 1) Ask kimi a question and record the CoT 2) Ask claude to analyze the CoT and give the main arguments 3) Classifier! Anthropic classifier can't even distinguish between thinking in the prompt vs thinking from the model
Clément Dumas
@Butanium_Oct 13
How much false positives on the internal AI safety traffic? How much false positive on transcript analysis? >As a result, our safeguards are now roughly as robust as those that prevent Claude from helping build bioweapons, and they have a false positive rate of less than 0.1%.
3
24
1,054
💀💀💀, basically if the model would do a novel concerning thing on my current eval, we wouldn't have caught because Opus tried to work around the classifier
1
1
98
Someone should try character training but you only train the early layers and see if the model verbalizes more who it is 👀 More on that coming soon but it's interesting to track self report across training step when doing character training!
David Atkinson
@diatkinsonOct 6
Replying to @diatkinson
We can test this further: trained on all 40 layers, Qwen3-14B is a terrible self-reporter. But if we train only its first 20 layers, faithfulness reaches 0.74. Once training reaches layer 25 or beyond, faithfulness plummets, although the decisions are ~just as good.
1
1
20
704
After "you can't do cyber defense with claude", @AnthropicAI presents "you can't do monitoring with claude" love to see that from the most AI safety aligned company!
Jan Betley@BetleyJanOct 5
Had the same today, when analyzing traces from GPT-5.6-sol. Annoying. Literally my agent says "I can't make you a report but here are some scripts you can run yourself".
6
8
2
78
12,733
Replying to @voooooogel
mine is a disney movie ahah
4
79
Load more