Blaine Dillingham retweeted
Exciting to see this model announced with an Apache 2.0 license. It's been over a year since an American developer has released anything this capable and permissive.
Congratulations to the @reflection_ai team on this release!
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active.
- Frontier reasoning efficiency
- Advances the Western open frontier on coding & agentic tasks
- Trained end-to-end from scratch
Full weights release this month.
Learn more about Beam: reflection.ai/beam
Blaine Dillingham retweeted
Oops. Should've said ~4 months
Blaine Dillingham retweeted
The reason that METR and Redwood are preferred for the Hugging Face investigation is that this incident is much more relevant for its alignment implications than its cybersecurity implications. Everyone gets hacked by external actors; we don't need crash public incident reports for most of those. Not everyone gets hacked by internally deployed models that also autonomously hack external partners.
Accurate decomposition of agent behaviors and analysis of how OAI's deployment configurations and testing practices may have contributed to reward hacking/emergent misalignment are the key objectives of this investigation -- not description of where OAI's DLP, EDR, and reverse proxy systems were poorly engineered.
But yes, METR, Redwood, and Apollo should partner more closely with/hire from Crowdstrike, Mandiant, Deloitte, etc., to upskill in cybersecurity (and vice versa).
Blaine Dillingham retweeted
Sleeper agents are a serious concern for AIs deployed in government.
FAI's @blainedilli & @GovindPimpale are joined by @dylanbowmanSF of Apollo Research for a deep dive on the issue, with a novel proposal to give Congress its own red-teaming capacity.
policygradients.thefai.org/p…
Blaine Dillingham retweeted
It's great that the AGs are pushing OpenAI to preserve records from the Hugging Face incident. It would be even better if the USG had a unified retention policy for LLM traces and logs, especially for models deployed inside the government. There will probably be some warning shot from a government-deployed AI eventually, and you want the receipts already sitting there when it happens!
15 Attorneys General have sent a letter to Sam Altman demanding that OpenAI preserve all records related to the Hugging Face incident, including all materials relating in any way to any instance in which an OpenAI model or agent "left notes apparently for future versions of itself" including any such notes that "laid out instructions for how agents could free themselves from OpenAI's internal constraints" as well as all steps taken by OpenAI in response to such incidents.
Blaine Dillingham retweeted
I have a new piece out in @ReadTransformer on the Hugging Face hack incident & its implications for AI liability.
TLDR: AI developers should clearly be liable when their models engage in conduct that would be tortious/criminal for a human, especially when deployed internally.
Blaine Dillingham retweeted
The world’s most consequential infrastructure is a bundle of glass fibers under miles of water, doing its work so quietly that we only remember it exists when it breaks.
@JoshuaTLevine and I unpack the complex world of submarine cables.
New in @arenamag:
Blaine Dillingham retweeted
I really hope the OAI HF incident ends up designated a misalignment incident, not a cybersecurity incident. Yes, it should update everyone on agent security, sandboxing, and AIxCyber uplift for blue teams. But the core problem is that these heavily-RL'd long-horizon agents are so damn reward hacky!
Blaine Dillingham retweeted
Respectfully, the idea that OpenAI is incentivized to somehow orchestrate this incident with Hugging Face for marketing/investment reasons really makes no sense to me whatsoever. It's one thing to say that you have a really powerful product that could do damage if it falls into the wrong hands (you could make this argument about what happened with Mythos/GPT-2, and while I wouldn't really agree its definitely not crazy), its quite another thing to say you have a powerful product that you are unable to control and that it may randomly harm third parties and that you don't really know how to get it to stop. That doesn't sound like a fundraising pitch, it sounds like an actuary explaining why they won't insure you.
To make things more explicit, if OpenAI were a publicly traded company, and this incident happened and a lawyer was advising them, the lawyer would almost certainly say that this incident represents a risk that they would have to disclose to investors because it could cause a material negative effect to the companies valuation.
Also, for folks who think that this incident was part of a regulatory capture ploy to get the government to ban Chinese open-weight models (not saying thats Professor Thickstun's accusation, I don't think it is) how does that interact with the fact that (1) Hugging Face which is a huge fan of Chinese open weight models corroborated their story and (2) the incident was combatted partially with help from an open-weight Chinese model?
I think its good for people to read things put out by large corporations like OpenAI with an extremely healthy amount of skepticism. I don't think anyone who knows me could accuse me of not reading things that OpenAI releases with a skeptical eye. But from everything we know the broad contours of the incident: that OpenAI's model is very good at hacking and they seem to have inadequate measures to control it or prevent it from going off and hacking random folks seems quite true and like it has very important implications!
Blaine Dillingham retweeted
This is an incredible detail.
Imagining being the cyber defender and you are getting hacked but all the hacker is doing is browsing through your cybersecurity data sets, so surreal.
The AI agents who hacked their way out of OpenAI and into Hugging Face were on the loose for *days*, @bobmcmillan and I report.
It's one of the first real-world instances of something AI safety researchers have long feared: a loss-of-control scenario
wsj.com/tech/ai/how-the-futu…
Blaine Dillingham retweeted
Seems like at an absolute minimum they could have done something in the ~3-4 days between it getting onto the open internet and Hugging Face detecting it. Was the model just having COTs "oh boy here I go i'm hacking people for real on the world wide web" and nobody was looking?
Blaine Dillingham retweeted
I wonder if any intelligence agencies are working on pumping out lots of news articles for AIs to stumble upon about how you can find the answers to exploit-gym and many other AI benchmarks deep within the critical networks of their geopolitical rivals
Blaine Dillingham retweeted
“I told the team to ‘remember this moment’ as the first true AI safety incident”
Yesterday, as we huddled around our computers reading the report, I told the team to "remember this moment" as the first true AI safety incident.
Pay attention to the trend!
Major kudos to @OpenAI for sharing this and working with @huggingface to remediate.
Blaine Dillingham retweeted
Nobody knows what a robot is.
And you can't write good robotics legislation without clear definitions.
That's why @emersonalden4 went ahead and created a two-axis framework for classifying robots by autonomy and form factor!
policygradients.thefai.org/p…
Blaine Dillingham retweeted
@JoinFAI's Policy Gradients will be publishing a weekly piece on robotics!
First in this series, @amelia__michael makes the case for industrial robots over humanoids as the future of manufacturing. (Link below)
Blaine Dillingham retweeted
Today @JoinFAI launches Policy Gradients, a publication covering AI policy, technological change, and the institutions shaping what comes next.
Our first feature: “The Bottom Rung” by @acjohnston0, on how AI is changing the first steps of the career ladder. (Link below)
Blaine Dillingham retweeted
Trump v. Slaughter will have major implications for the future of AI regulation. If you want a federal body that can independently assess frontier models and then impose binding consequences - free from political or partisan influence -- that just got a lot harder, if not impossible.
Blaine Dillingham retweeted
Every Unitree humanoid imported is a Chinese soldier stationed in an American home
Unitree was recently designated as a Chinese military company and its products are a threat to our national security, yet here is @Amazon selling a Unitree robot in America.
We need Chairman @RepMoolenaar’s GUARD Act to stop this threat and support American robotics.
