Research Scientist at @ILI_NYU and @ICSIatBerkeley. A dad. Tweets are reflex, noise, and error. 🦋https://xitter.cf/t.co/t1n5c7A8Tp
An NYC suburb
Joined April 2008
- Tweets16.1K
- Following655
- Followers1.1K
- Likes13.7K
I'm building something new. digifesto.com/2025/05/28/im-…
What about the loyalty of AI agents in government function? digifesto.com/2025/02/06/wha…
Complex systems and ethics according to contextual integrity digifesto.com/2024/12/08/com…
a brief reflection on where I'm at as a scientist digifesto.com/2024/11/26/how…
Ok, but what is the anti-incumbency wave about? Is there an explanation of that yet? Is it just 'inflation'?
Sebastian Benthall retweeted
This is a great paper. It points out:
1. Humans do not even approximately behave according to rational choice theory
2. There is no reason to think advanced AI will "inevitably" maximize some utility function
3a. Human preferences are derivative / constructed, so aligning AI by matching its behavior to our stated preferences is wrongheaded
3b. We can align AIs directly to some normative ideal of a "good assistant / programmer / driver / etc." instead
4. Aggregating preferences across people is fraught with philosophical and mathematical difficulties. We should not aim to align AI to the "collective will of humanity."
Should AI be aligned with human preferences, rewards, or utility functions?
Excited to finally share a preprint that @MicahCarroll @FranklinMatija @hal_ashton & I have worked on for almost 2 years, arguing that AI alignment has to move beyond the preference-reward-utility nexus!
Direct, concise answers to coding questions in Google AI Overview is a real improvement, saving a few clicks every half hour or so.
I had a great time at the 6th Annual Symposium on Applications of Contextual Integrity!
It's a great community that's always growing intellectually, with a sincere and friendly vibe.
Thanks to everyone who organized it.
Sebastian Benthall retweeted
My colleague Amy Greenwald just sent me this talk slide. It's from Jesús Fernández-Villaverde, an *economist*.
We have so won.
(I mean, I have *thoughts* about this content and his course slides, but also… We have so won.)
Sebastian Benthall retweeted
Next Session @TheOSSci Economics Interest Group on the history of software markup languages in economic modeling w/ @sbenthall Research Scientist at @ILI_NYU @ICSIatBerkeley @econ_ark
Next Monday 12 August 1200 ET
Join Us! bit.ly/3w6dOlK
I've been awarded a grant from the Future of Life Foundation! It will support my research on Designing Fiduciary AI. Beyond performing foundational research on what fiduciary standards for AI should be, the plan is to bootstrap a standardizing process through research workshops.
In the coming months I will be reaching out to interested members of the research community to put the workshop together. Please be in touch if this interests you.
Sebastian Benthall retweeted
𝗝𝗮𝗺𝗲𝘀 𝗖. 𝗦𝗰𝗼𝘁𝘁 passed away on July 19. He was a political scientist and specialist in Southeast Asia. He was a dissident within Political Science. Yet his books opened up lines of research on central issues, such as the exercise of power and resistance to power.👇
There was nothing at all surprising about the presidential debate. What's surprising is that there are people pretending to be surprised.
Envisioning the Future of Computer and Information Science Research: Some Ideas digifesto.com/2024/06/09/env…