Ensuring that tech companies don't have a monopoly on being able to do research on cutting edge AI @AiEleuther. She/her
Joined May 2019
- Tweets15.1K
- Following928
- Followers19.3K
- Likes13.8K
Pinned Tweet
If you follow me and haven’t listened to this talk in its entirety, doing so is more important than whatever it is you are currently doing. There’s a lot of cybersec jargon at times but I don’t think you need to know anything about security to understand what’s important.
Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more.
youtube.com/watch?v=87DyyMV0…
I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time!
Stella Biderman retweeted
Live now at board 211 :)
Excited to share that I'll be in Budapest for EMNLP 2026 (October 24–31) to present our paper, “Fractional Rotation, Full Potential? Investigating Performance and Convergence of Partial RoPE”!
This is joint work with Krishna Gummadi, Manish Gupta and @lasha_nlp!
Stella Biderman retweeted
Pangram is good, but has some well-known failure modes, like:
- A time you heard it didn't work
- A school essay you wrote years ago that it flagged as AI (you don't have a link or the essay text handy)
- A different AI detector got it wrong, and they're all the same, right?
Analysis by @KelseyTuoc is consistently ahead of the curve.
theargumentmag.com/p/the-cor…
I’m still working through it, but this work is incredibly cool and (unless there’s some kind of serious mistake) the lineage of work including this and “Why Larger Models Learn More” is exactly the kind of work that I think is necessary to develop a science of DL. I’m a huge fan
The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem.
A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:
ALT Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.
for me to take a serious stab at solving when I got the bandwidth to do so (probably January tbh). Now I can spend that time on something else :)
I can’t wait for the full write-up!
Stella Biderman retweeted
Based on this article, OpenAI has made false accusations about me. The three researchers fired from OpenAI were told I facilitated architecture leaks to The Information. This is false.
A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.
The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.
As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.
I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.
New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.
Then, OpenAI asked to see their Signal chats. A week later, they were let go.
More below from @keachhagey, @berber_jin1, and me.
wsj.com/tech/ai/the-signal-c…
THIS IS DISINFORMATION
The PHP exploit Nate is talking about was found by AI-enabled defensive operations. There is no evidence of large scale exploitation. This is a huge win for defense running ahead of offensive.
Roon wrote about this as well xitter.cf/tszzl/status/211311768…
And it's important to remember that Fable was shut down by the USG just days after the it was released. Convos I've had with people at A\ indicate that they were concerned that this might happen and (although nobody has told me this) it seems very plausible that one reason for