Gaurav Sett (@GauravSett) header
Gaurav Sett (@GauravSett) profile picture

AI Policy. PhD Student, @RANDCorporation. Non-Resident Fellow, @IFP.

US-East-1
Joined March 2022
Photo by Gaurav Sett (@GauravSett)Photo by Gaurav Sett (@GauravSett)Photo by Gaurav Sett (@GauravSett)Photo by Gaurav Sett (@GauravSett)Photo by Gaurav Sett (@GauravSett)
Made another task tracker for NPSM-11 on "Artificial Intelligence in the National Security Enterprise"
Task tracker for the new EO on "Promoting Advanced Artificial Intelligence Innovation and Security"
2
22
2,682
Gaurav Sett retweeted
I’m hiring a Member of Technical Staff at IFP. Your job will be to build software that allows us to operate with the effective capacity of a think tank 10 times our size. If you care about reforming US policy, I think this is one of the highest leverage roles available: you’ll be directly amplifying the work of our extremely cracked and growing team of 35. Given how important this job is, we will pay a $5,000 reward for any referral that results in a hire. More on the role: I expect this will be a hard role to fill, for a few reasons: 1. You have to believe US policy is the most important lever for impact There are plenty of opportunities in the private sector for engineers to have a significant impact. But technical talent in public policy is far more neglected, and often much higher leverage. IFP’s mission is to reform US policy to drive breakthrough discoveries, attract top talent, and expand our nation’s capacity to build. We measure our efficacy by our counterfactual policy impact: whether we create good policy outcomes that would not have otherwise happened. The software you build will directly affect the rate at which we can do this. The work you help us do faster and better will be read in the Oval Office, will shape the direction of billions in federal $, and will directly influence geopolitical strategy. We need someone in this role who internalizes how important this is. 2. You have to want to deeply understand what effective policy work looks like Many tasks in policy research and advocacy are time-consuming but basic text-to-text translation tasks, like simplifying results from academic papers, summarizing public comments on new regulations, and recording information about contacts from meeting notes. This work is necessary to produce good policy outcomes, but laborious. One way to understand this role is by looking at the way progress in AI capabilities has shifted the job of a programmer away from writing code and more towards specifying requirements and verifying the work of AI agents. In this role, you’ll do the same for policy work, moving the focus of our team away from basic natural language manipulation tasks and towards strategy, deep research, and policy engagement. A crucial part of this work is identifying which parts of policy research and advocacy can be effectively automated with frontier models without compromising quality, and continually retooling to adapt to the latest state of the art. The ideal candidate is someone who cares about building tools that people love to use and who can work effectively with our policy teams to understand their goals, processes, and constraints. 3. You have to be able to build a wide variety of great software Your projects will vary widely, from building agent scaffolds to custom CRM tooling to deployment pipelines for maintainable, performant microsites (check out the full job description for more). The right person will have led the design and delivery of complex user-facing products across the full stack, know how to ship quickly while building maintainable tech, and understand the trade-offs inherent in your design choices. – This role is open to both remote work and people able to work out of our office in Washington, DC, with the latter being preferred. The salary range for this role is $165,000 to $245,000. Here’s the full JD and link to apply: ifp.org/opportunity/member-o… Applications close May 31st.
7
38
17
169
1,837,254
It’s time to build for AI resilience. Billions are flowing into AI resilience. Whether it gets spent well or bottlenecked by lack of proposals depends on people with real ideas stepping up to scope and build them. That's what The Launch Sequence is trying to make easier: we help authors turn vague ideas into concrete projects and connect them with funders ready to back serious work. Our RFP is still open: ifp.org/rfp-launch
AI will help discover new science, such as cures for diseases, which is perhaps the most important way to increase quality of life long-term. AI will also present new threats to society that we have to address. No company can sufficiently mitigate these on their own; we will need a society-wide response to things like novel bio threats, a massive and fast change to the economy, extremely capable models causing complex emergent effects across society, and more. These are the areas the OpenAI Foundation will initially focus on, and in my opinion are some of the most important ones for us to get right. The Foundation will spend at least $1 billion over the next year. @woj_zaremba, co-founder of OpenAI, will transition to Head of AI Resilience. I believe that shifting how the world thinks about safety to include a Resilience-style approach is critical, and I am extremely grateful to Wojciech for taking on this role. Wojciech has been my cofounder for the last decade; anyone who knows him will understand what I mean when I say he is one of a kind. He has a lot of ideas about how we build a new kind of AI safety. @JacobTref is joining as Head of Life Sciences and Curing Diseases. @annaadeola, our VP of Global Impact, will transition to Head of AI for Civil Society and Philanthropy. @robert_kaiden is joining as Chief Financial Officer. @jeffarnold is joining as Director of Operations.
1
8
37
8,121
Gaurav Sett retweeted
Super excited about the OAI Foundation and Wojciech's announcement! >$1B will flow over the next year, with $25B committed to curing diseases and AI resilience. These are exactly the kinds of investments we’re hoping to generate concrete ideas for with The Launch Sequence’s RFP (ifp.org/rfp-launch). We’re collaborating with over 20 organizations to find and vet promising ideas, develop concrete proposals, find promising teams, and seek funding for them. We’re also super excited about our advisory panel: – @woj_zaremba , OpenAI co-founder leading the OpenAI Foundation’s AI Resilience portfolio – @tkalil2050 CEO of Renaissance Philanthropy, and previous OSTP deputy director for tech and innovation. – Kathleen Fisher CEO of ARIA and former DARPA I2O Director. – @matthewclifford , Co-founder and Chair, Entrepreneurs First, Chair of ARIA, and previous UK prime minister's AI advisor. – @geochurch , Director of Church Lab and co-founder of dozens of biotech startups. Our RFP is still open. If you have truly ambitious ideas for how philanthropies like the OAI Foundation can help prepare the world for advanced AI, send us your pitch: ifp.org/rfp-launch
Life update — I’m moving to the OpenAI Foundation to lead AI resilience. AGI will bring tremendous benefits and potential disruptions, such as impacts on children and youth, model malfunctions, emergent bio-risks, and more. AI resilience is about minimizing these disruptions so society can fully realize the benefits. openaifoundation.org/news/up…
4
18
2
120
29,933
Policy researchers—it is time to start transforming your work into verifiable tasks and delegating to agents.
The linear scale looks insane
3
137
The future belongs to those who, in a long sequence of small decisions, dare to reach for better branches. Let them be the many.
How much control do we have over technological development? Usually, not much. I think the evidence suggests we live in a strongly technologically deterministic world. But sometimes, we can make small changes to tech development that have massive, lasting consequences: Nuclear proliferation, COVID vaccines, permissive action links — what do these successful examples have in common? Accelerating the development of critical technologies can alter the relative sequence in which they arrive. Small changes at the right time can make all the difference. I think we’re in this critical window of opportunity for AI right now. Where, how, and in what order technologies are developed will likely decide whether we reach a good or a bad future. There is no fundamental reason why everything should go well; no law of the universe conspiring to make things better. We can acknowledge the strong constraints of tech determinism and still assume responsibility, take charge, and design a future that lives up to our optimism. My first Substack post, in MACROSCIENCE → macroscience.org/p/do-not-su…
1
2
11
900
Gaurav Sett retweeted
If you have ideas in this vein, you should submit to The Launch Sequence! $10k for published proposals, $1k for new ideas we greenlight And we're working with funders to make sure there's follow-on capital if you want to build what you've written up 🏗️ ifp.org/rfp-launch/
wake up, people. we were always going to need to harden literally all software on earth, our biology, and physical infrastructure as a function of ai progress one way to think about the high level goal here is that we should seek to reliably engineer and calibrate the exchange rate between ai capability and ai power in different domains now is the time to build some ambitious security companies in software, bio, and infra. the business will be big. if you need a sign, let this silly little lobster thing be it. the agents will only get more capable from here
11
1
41
16,928
Looking for ambitious, concrete projects to prepare the world for advanced AI. 🧠 🚀
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
29
28
1
167
43,918
Gaurav Sett retweeted
IFP is a rare combination: technically literate, genuinely bipartisan, and actually competent. If you're tired of policy slop and want to work on things that matter, you should pitch. Ideas matter!
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
5
25
112
21,231
It's fashionable these days to say you're neither a doomer who cares only about the risks, nor an accelerationist who skips over them, but some secret middle category of reasonable people who want to maximize the benefits of AI and minimize the risks. But how do we actually do that? What does society concretely need to build to make sure defensive capabilities don't get dangerously outpaced? Concerningly, it is not clear. For all the noise around AI progress, it's hard to put together a list of the defensive technologies we need to build, and even harder to find people actually executing on these ideas. There are billions soon to be pointed at this question and it's not clear what to do. Do you have an idea for a technology, tool, or institution that needs to exist for us to positively navigate the transition to a world advanced AI? We want to know.
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
3
13
1
78
9,418
Gaurav Sett retweeted
There's no shovel-ready list of the essential projects to build in a world of advanced AI. At IFP, we're hoping to change that, and we need your help. We're offering a $10k honorarium for proposals we select.
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
2
20
1
77
22,285
Gaurav Sett retweeted
The private sector will ensure we have an abundance of chatbots and agentic coding tools. It's up to the public and philanthropic sectors to deliver faster science, stronger security, and adaptive institutions. But these public interest funders need well-scoped projects to fund! Pitch us your best ideas and we'll help you develop them into fundable opportunities.
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
2
14
78
23,502
I recently joined @IFP as a Non-Resident Fellow to work on The Launch Sequence, an initiative to develop ambitious projects to prepare the world for advanced AI. Today, we opened our Request for Proposals!
REQUEST FOR PROPOSALS What do we need to build to prepare the world for advanced AI? >$25 billion is about to flow into AI resilience and AI-for-science from the OpenAI Foundation, the Chan Zuckerberg Initiative, and other major funders. But there's no shovel-ready list of the essential projects to build, and no critical mass of builders ready to execute. We're trying to fix that with The Launch Sequence, a collection of concrete projects to accelerate science, strengthen security, and adapt institutions to future advanced AI. We're opening up The Launch Sequence for new pitches. We'll help you develop your idea, connect you with funders, and get it built. $10,000 honorarium for published proposals. 🔗 Read the RFP and submit a short pitch (200–400 words): ifp.org/rfp-launch We’re looking for short pitches in three areas: 1. Accelerating science: What do researchers need to make breakthroughs with AI that markets and conventional grants won’t fund fast enough? 2. Strengthening security: What tools, technologies, or orgs can help ensure rapid AI advances don't undermine national security and public safety? 3. Adapting institutions: How do our institutions need to evolve to help society adapt to AI-driven change while preserving human agency? Our advisory panel: – @woj_zaremba, OpenAI co-founder – @tkalil2050, CEO of Renaissance Philanthropy – @matthewclifford, Co-founder & Chair of Entrepreneurs First, founding Chair of ARIA On top of the $10,000 honorarium, we offer a $1,000 bounty for pitching an idea we hadn’t heard before (if we publish it), and for every successful referral. Submissions are rolling, but we'll prioritize early pitches (within the next few weeks) and start reviewing immediately.
1
2
1
27
2,323
Internalize this information and adjust your ideas about global AI competition accordingly.
1
6
2,845
An easy example: Start an "AI for personal uplift" group chat with your colleagues/friends, and run an "Experiment with Claude Code" session (template agenda linked below)
This is likely going to be one of the most important decades in human history, and even ordinary actions like “putting an essay on the internet” or “starting a groupchat” can be extremely high leverage. Think hard about what you’re doing, & post more, write more, participate!
1
2
133
Incredible use of online RL! Does this show part of why OpenAI wants its own browser? If OpenAI can track when users accept agent actions (e.g., user books flight suggested by the agent), it can get great reward signal!
We've trained a new Tab model that is now the default in Cursor. This model makes 21% fewer suggestions than the previous model while having a 28% higher accept rate for the suggestions it makes. Learn more about how we improved Tab with online RL.
2
262