Cognitive scientist, previously at @Cambridge_Uni 's Winton Centre for Risk and Evidence Communication, now working on LLM capability evaluation & alignment
Joined August 2010
- Tweets524
- Following317
- Followers285
- Likes624
Pinned Tweet
Can weaker models verify solutions generated by more capable models? Announcing FindTheFlaws--5 new datasets of “solutions” to hard problems, with expert-annotated errors, to support empirical evaluations of debate, critique, and other oversight paradigms. They are: 🧵
Proud to have contributed to this study that cleanly demonstrates the persuasive capabilities of LLMs. (No secret Reddit posting involved!!)
Gabriel Recchia retweeted
"If you cannot measure it, you cannot improve it" - Lord Kelvin
The science of AI safety evaluation is still nascent, but is advancing and we know much more today than two years ago.
CeSIA tried to make this knowledge accessible by publishing a SotA literature review!
Gabriel Recchia retweeted
Humans are often very wrong.
This is a big problem if you want to use human judgment to oversee super-smart AI systems.
In our new post, @geoffreyirving argues that we might be able to deal with this issue – not by fixing the humans, but by redesigning oversight protocols.
Gabriel Recchia retweeted
Advanced AI systems require complex evaluations to measure abilities, but conventional analysis techniques often fall short.
Introducing HiBayES: a flexible, robust statistical modelling framework that accounts for the nuances & hierarchical structure of advanced evaluations.
ALT An example of hierarchically nested evaluation data.
This is an excellent and, I think, very important piece that I hope gets the attention it deserves within the AI safety community. Many congratulations to @JoshAEngels, @dbaek__, @thesubhashk, @tegmark
FindTheFlaws is available now to help researchers build more robust oversight protocols as AI capabilities continue to advance.
paper: arxiv.org/abs/2503.22989
data: github.com/modulo-research/f…
Collaborative work with @ChatrikMangat , @Art3mis0707 , @issacli0821
Gabriel Recchia retweeted
AISI's research agenda is out! We cover a variety of topics in the evaluation and mitigations of risks from frontier LLMs, including both work happening at AISI and work we are excited to see others tackle.
Gabriel Recchia retweeted
This has been accepted at ICML 2025! See you all in Vancouver.
Credit to @TingchenFu for leading this work and to my wonderful collaborators! @MrinankSharma @philiptorr Shay B.Cohen and @DavidSKrueger
Gabriel Recchia retweeted
Responsible Reviewing #NeurIPS2025 — TL;DR
1- If you/ your co-author skip your assigned reviews
→ you wont see your own paper’s reviews.
2- Submit a poor quality review → your paper may be desk‑rejected.
👏 Nice one, NeurIPS!
🔗 blog.neurips.cc/2025/05/02/r…