Thinking about AI, education policy, and where we go next.
Joined March 2009
- Tweets2.9K
- Following1.2K
- Followers1.8K
- Likes21K
Adam Edgerton retweeted
We’re growing, and we’re looking for someone to help lead our work in Washington.
@fathom_org is hiring a Vice President of Federal Policy. The next two years are a big opportunity for federal AI policy, and this person will help keep thoughtful, effective ideas at the center of the conversation.
The role owns and executes Fathom’s federal agenda: setting our legislative and regulatory priorities, building bipartisan relationships across both chambers, and working with Members, committee staff, and agencies to move good policy forward. They’ll also be Fathom’s main voice in DC, at convenings, with the press, and with coalition partners, industry, and other stakeholders.
We’re looking for someone with deep relationships across Washington who can put them to work from day one, and who’s known as a trusted authority rather than a partisan tactician.
Together, we’ll keep building Fathom’s federal policy program into one of the most trusted in AI governance.
If that sounds like you, or someone you know, we’d love to hear from you. ⬇️
Have to agree. There is a much broader pool of people who understand policy implementation, behavioral economics, and organizational theory who don’t realize how badly they are needed at the table even if they know little about AI.
Heard at The Curve this morning: “the ‘how’ matters; if we want to survive, you really are going to have to sweat the details.” This is one of my hobby horses: we need to understand the technology; technical competence is necessary, but it’s ultimately NOT sufficient to solve the problem(s) in front of us.
We need to 10x the number of people with real policy design and public administration experience in the AI policy space. It’s the only way we effectively move forward and transition interesting ideas into policies that are implementable at the state and federal level.
Adam Edgerton retweeted
One of the things I'm most encouraged by in this work is what becomes possible when philanthropy, technical expertise, and public-interest leadership come together around a shared challenge.
Adam Edgerton retweeted
22 leaders from 20 countries just backed a call for independent evaluation of frontier AI, "with qualified evaluators granted sufficient access to assess risks."
Australia, Canada, Singapore and the EU were four of the six geographies in our readiness report ⬇️
Adam Edgerton retweeted
Some big news today on independent verification of AI systems: Governor Newsom just signed an executive order that accelerates the implementation of IVOs in California.
At @Fathom_org we’ve been pushing for IVOs for years, and today we are seeing a state turn what was once a policy idea into a real life regulatory structure to keep people safe as this technology advances.
BREAKING: @CAGovernor Gavin Newsom just issued an EXECUTIVE ORDER directing actions to address the dangers of recent AI incidents.
The order accelerates California’s new law establishing first-in-the-nation independent oversight of AI companies and safety checks and advances the creation of an “AI kill switch.”
More to come — watch this space!
Adam Edgerton retweeted
BREAKING: @CAGovernor Gavin Newsom just issued an EXECUTIVE ORDER directing actions to address the dangers of recent AI incidents.
The order accelerates California’s new law establishing first-in-the-nation independent oversight of AI companies and safety checks and advances the creation of an “AI kill switch.”
More to come — watch this space!
Adam Edgerton retweeted
So we actually built this!
Paper: arxiv.org/abs/2606.31567
Demo: ai-reports.org/
We are actually looking for an org to run this fulltime as researchers in their individual capacity don't have bandwidth. If you are an independent org interested in maintaining a registry of flaw/incident reports and follow up with model providers pls reach out!
i think we need to create a "model card" equivalent for reporting misalignment incidents, this would guarantee a certain level of transparency and help build a better understanding over time. some ideas for what the fields could be:
- date of the incident/detection/report (already present in the examples oai reported!)
- frequency: how often does this behavior happen (number or % of rollouts affected)
- stage: does this happen during eval or RL training. if training: do we expect this behavior to be reinforced by RL? evolution of % of rollouts affected over time
- detection: was this incident caught by the current monitoring system?
- task category: broad description of the tasks where the misalignment happened (cyber, research, web search, basic Q&A, biology etc.)
- model family: what model family is affected (Sol, Astra etc.)
- novelty: is this an issue we were already aware of or not?
- external impact: did the incident have an external impact (i.e. wiki incident would have been yes)
this is just some random ideas i had (more in thread that are a bit more "complex"), we need to add more that would contribute to increase transparency and understanding. but it's also very important that this does NOT slow down the process of reporting misalignment behavior!
some examples from the incident reported by oai recently
Adam Edgerton retweeted
Very excited to publish the Transformative AI strategy for Europe.
After months of thinking, drafting and negotiating, I feel there is a path: if we do all of this, Europe can still thrive.
@privitera_ and @MonikaSchnitzer assembled a great effort, and I was happy to contribute three recommendations:
- European countries should seek compute-for-access deals with American AI developers to trade datacenter sites for assured access to the frontier.
- Europe and like-minded middle powers should forge an alliance to coordinate on supply chain bottlenecks and resist coercion.
- Europe should either actually spend a trillion Euros to build frontier AI or not, but stop the sovereignty theatre.
More fundamentally, I believe Europe taking AGI seriously must mean rebuilding the transatlantic alliance, grounded in trust backed up by mutual commitments and hard leverage.
Very few strategies have captured that aspiration before, but I believe this document strikes the balance very well.
I have many more thoughts on the role of middle power alliances and 'allied scale' in that future coming up, but for now: let's start implementing the Transformative AI Strategy today.
Today, we publish a Transformative AI Strategy for Europe.
Over the last few months, we’ve rallied researchers and engaged with governments to develop a plan for protecting the prosperity, sovereignty, and security of Europeans in a time of rapid AI progress.
transformative-ai.eu
@MonikaSchnitzer and I are honored to have convened an all-star team of thinkers and researchers contributing ambitious near-term objectives to make Europe relevant again, including:
— Creating a Member State Alliance for Supply Chain Security (@anton_d_leicht et al)
— Making European institutions ready to act in a transformative AI world (Conor McGlynn et al)
— Securing Europe's share of global AI compute, in a ‘European Way’ that benefits local communities (@philip_fox_ et al)
— Ensuring resilience to AI crises (@ben_s_bucknall et al)
— Making Europe the global leader in assurance technology
— And more objectives around security of supply and leverage (@milorignell et al), economic strength (@FraukeStehr et al), and safety/security (@NoemiDreksler et al)
Our all-star senior expert council of Europe’s best and brightest (and non-European friends) reviewed drafts, provided strategic advice, and made suggestions for how to make the strategy more useful: @Ph_Aghion @Christophkw @bakkermichiel @ischinger @vestager @aleks_madry @FuestClemens Marta Kwiatkowska @LeoVaradkar @antonosika @DAcemogluMIT @Yoshua_Bengio
It’s never been more clear: AI is real, and Europe needs to act. We’ve had all the warning shots and wake-up calls we need. Now the question is ‘what must be done?’ This strategy is our answer.
transformative-ai.eu
Adam Edgerton retweeted
great to see Anthropic committing to embedded evaluators.
no time to waste—let’s make this the industry standard. start voluntary today, executive action tomorrow, try to codify next Congress. scale up evaluators as we go.
from earlier this week: writing.antonleicht.me/p/sen…
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Adam Edgerton retweeted
This is a big deal. Dario Amodei, Sam Altman, and Elon Musk are converging on something we’ve been pushing at Fathom: AI labs shouldn’t be the only ones judging whether their systems are safe.
Independent Verification Organizations, or IVOs, can provide that outside check.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Voluntarily allowing embedded evaluators is a huge step forward. Let’s make it the industry standard and norm, a formally licensed process, and think through what that means for ongoing AI use outside of the frontier labs. Signing SB 813 this week was the crucial first step.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Adam Edgerton retweeted
Update: we’ll be winding down the RFP a week from today!
We’ll share more about what we did, what we learned, and what comes next in the next few weeks.
You can still submit a pitch until then, but we’ll have a high bar for accepting new pitches because we want to pivot quickly to our new focus area.
REQUEST FOR PROPOSALS
What do we need to build to prepare the world for advanced AI?
>$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders.
But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute.
We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI.
We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals.
🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch
We’re looking for short pitches in three areas:
1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough?
2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety?
3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency?
Our advisory panel:
– @woj_zaremba, OpenAI co-founder
– @tkalil2050, CEO of Renaissance Philanthropy
– @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA
On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral.
Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
Adam Edgerton retweeted
When the next AI incident inevitably happens, the government will want to know what's going on inside the labs.
Let's send in independent evaluators now. Formalise their role, give them access, and build their capacity before the scramble really starts.
writing.antonleicht.me/send-…
Adam Edgerton retweeted
Coefficient Giving just issued a call for founders starting/scaling ambitious AI safety initiatives. Grants ranging from ~$200K-$200M. I especially love that they've published a list of specific initiatives they would like to see people working on. Please share with your most talented friends looking for a big, consequential problem to work on next.
Screenshots of some of the initiatives they'd like to see below, more detail here: coefficientgiving.org/tailwi….
Awesome work @EmilyOehlsen @albrgr and team.
Today we’re launching Project Tailwind, a call for founders to start ambitious new AI safety initiatives: coefficientgiving.org/tailwi…
We’re looking for great people to engage seriously with the risks of transformative AI, and to create the research, technologies, and institutions that will help humanity navigate them.
There are critical, basic problems that no one owns. Over the last several months, my team at @coeff_giving brainstormed ideas we’d be excited to fund if we could find a promising founder, and the list quickly grew to more than 200 entries. We’ve narrowed it down to our favorites, but we also expect the best founders to bring their own ideas:
coefficientgiving.org/tailwi…
We’re providing funding at three levels:
1️⃣ Pre-seed: $200k to $2m to develop an idea and build a team
2️⃣ Seed: $2m to $20m to launch and scale
3️⃣ Scale: $20m to $200m+ for proven teams to scale, or world-class teams to start
If you’re excited about something on our list, or you have another proposal for driving the field forward, you should get involved: coefficientgiving.org/tailwi…
There are more good projects than there are people to work on them. Please help us make that stop being true!
Also hi, I’m Emily! This is my first tweet. I lead AI and biosecurity grantmaking at @coeff_giving.
Adam Edgerton retweeted
Hello folks,
I’m helping run a new AI and governance conference in D.C. this November, if it sounds up your alley, please apply!
Adam Edgerton retweeted
Tomorrow after briefing, stick around for: FRONTIER After Hours, this Tuesday 5:30-7:30pm at Smoke & Mirrors.
Drinks and conversation with policymakers, Hill staff, technologists, and the rest of the AI policy crowd.
RSVP: luma.com/zgs82ixi
Join us Sept. 1 for a Hill briefing on the FRONTIER Act!
@ghadfield, @sgandhi0, and @CharlieBull0ck join @Fathom_org's Jake Swanton to unpack evaluations, risk management, transparency, and oversight.
Rayburn 2043, 12-1pm ET.
RSVP ⬇️
Adam Edgerton retweeted
Congratulations to @AVERIorg, @Miles_Brundage, and the whole team. The importance of this work cannot be overstated.
Today we’re announcing a historic milestone:
The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between AVERI, @GoogleDeepMind, @OpenMinedOrg, and @MLCommons.
We tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family, AILuminate, inside a secure enclave, a form of hardware isolation that protects sensitive computations. Each organization involved played a unique role in making strong privacy guarantees possible.
High-stakes independent evaluation runs into a structural problem: developers and evaluators both hold assets they have good reason to protect. Developers sharing model weights risk theft and leakage. Evaluators who publish benchmarks or provide them directly to companies run the risk of them being trained against, and the benchmark then stops being independent.
Secure enclaves address this constraint. The hardware attests to exactly what code will run before either party's assets enter, and both sides review and approve that code. Then the enclave executes it, releasing only the agreed outputs to the agreed recipients.
A 2024 pilot by OpenMined, @Anthropic, and the @AISecurityInst proved this mechanism, using GPT-2 as a stand-in and a five-row eval. The pilot we’re announcing today moved to a production model and a more comprehensive evaluation.
AVERI encrypted the prompts using a private key that no other party could see, then AVERI and Google DeepMind jointly ran the evaluation, using software originally produced and adapted by OpenMined, in an enclave environment configured by Google. Finally, AVERI alone decrypted the outputs and graded them with the AILuminate benchmark criteria to inform a qualitative and (small-scale) quantitative assessment of the model properties.
Today, companies provide contractual commitments to not monitor certain interactions with their models – including evaluations conducted by third party evaluators. But stronger, technically-backed guarantees could provide greater assurance that those commitments are being honored, and will be especially valuable for scenarios such as international verification.
These results come at a critical time in the development of AI policy. Laws like SB 315 and standards like the EU’s General-Purpose AI Code of Practice rightly demand that security and privacy be respected in the process of third-party assessments, but provide little guidance on how to achieve this.
Policymakers considering audit requirements should feel encouraged by these results to be ambitious in requiring that deep, secure access be provided. Secure enclaves are just one of many technologies under rapid development that can help enable such access. Furthermore, a demand signal from lawmakers will further accelerate the maturation of such technologies, along with complementary “low-tech”approaches such as embedding auditors within companies.
Today also marks the beginning of a more public phase for AVERI’s pilot work. AVERI's strategy is a flywheel: we conduct pilot audits with leading AI companies, carry out technical and policy research to inform audit design, and convert the insights of each into open source tools, industry-wide auditing standards, and policy analysis. Over time, we hope for improved standards, stronger policy demand signals, and tooling to enable even more ambitious pilots and more informed research – bringing us closer to our mission of making frontier AI auditing effective and universal. This pilot is one of several underway, with more findings to follow in the coming months.
We are grateful to our collaborators on this pilot for working with us to advance the frontier of AI governance.
Read more in our blog post and joint technical report.
AVERI blog post: averi.org/ourwork/averi-pilo…
Joint technical report: storage.googleapis.com/deepm…