PhD candidate @MIT | AI for social science, multi-agent AI | formerly @AmazonScience, @CarnegieMellon
Cambridge, MA
Joined March 2019
- Tweets51
- Following278
- Followers162
- Likes436
Robin Na retweeted
Data-driven economic ABMs are very interesting.
Poledna et al. (2023) is a compelling example: a national-scale economic simulation that rivals traditional VAR and DSGE models in out-of-sample forecasting.
doi.org/10.1016/j.euroecorev…
Exciting potential for ABMs × AI!
This is true and I have always thought it was a big shame. Part of the reason (as Andrey notes in the replies) is that historically ABMs had not been taken to the data. But this is changing. Here is my colleague @RMaria_drc’s recent work bridging this gap:
arxiv.org/abs/2412.16591
ABMs x AI hold a lot of promise for economics, and it’s an active area of research for our team.
Robin Na retweeted
Jobs do far more than deliver a paycheck—they are a core institution in modern society for agency, purpose, recognition, and community
In our new DeepMind Institute essay with @scychan_brains and @JulianDJacobs, we explore research on how work relates to wellbeing. As AI potentially changes the nature of work, we must design policies and institutions that protect its non-monetary value
bit.ly/wellbeing-work-agi
Robin Na retweeted
Out today in Nature Human Behaviour: our review of large language models in computational social science, with @xu_fengli, @yongli and colleagues at @Tsinghua_Uni, @hkust & @MSFTResearch Asia. A thread on what we found, including where the field is fooling itself.
Read @ doi.org/10.1038/s41562-026-0…
Robin Na retweeted
A new very creative method for measuring ambiguity with LLMs co-led by @dsallydt and @LukeThorburn_ (EMNLP Oral 2026).
When I worked with @mhtessler @summerfieldlab et al on the Habermas Machine (a project in which we use LLMs to find statements that maximize agreement within groups of people), we often found that LLMs maximize consensus by simply making statements more ambiguous.
Eg a statement like "peaceful protests should be allowed" will likely lead to high agreement but that's because a lot of disagreement hides in different interpretations of what's "peaceful".
One way to measure ambiguity is to ask lots of people how they would interpret a statement and see if those people disagree. But this is obviously impossible to do at scale. In this work we show that we can use LLMs to help approximate this. Read Sally's thread or the paper for more details.
I think work like this is super important, for example to stress tests AI constitutions and model specs to see which rules are ambigious and in which situations.
If you are in Budapest, please stop by Sally's presentation.
🚨 New paper: how to measure ambiguity in laws, public proposals, and even AI constitutions! with @LukeThorburn_ @Werdnamai @bakkermichiel 🧵
Robin Na retweeted
What should we actually ask an LLM simulation to get right?
Much of the field focuses on reproducing human distributions or specific moments.
In our reworked paper, we argue that another target may be just as important: preserving the ordering across environments.
🧵1/6
Robin Na retweeted
What is a minimal set of assumptions that can explain the majority of human behavior?
We investigated this question with large-scale cognitive models🚀
CORE writes prediction errors in hierarchical episodic memories with context controling encoding, retrieval, and forgetting.
Robin Na retweeted
🚨New preprint!
We find evidence of LLMs enabling people to file lawsuits without lawyers (filing "pro se") at historically unprecedented rates in federal courts.👇
1/n
Robin Na retweeted
🚨 Out in Science this week, with @DG_Rand @duncanjwatts and Abdullah Almaatouq 🚨
We apply an integrative approach to a classic question in behavioral econ and cooperation research: *when* does peer punishment help or hinder collective welfare?
Robin Na retweeted
🚨📄 New preprint! We find the “boiling the frog” equivalent of AI use. In a series of RCTs, we show that after just 10 min of AI assistance people perform worse and give up more often than those who never used AI.
w Grace Liu @brianchristian Mira Dumbalska and Rachit Dubey 🧵
Robin Na retweeted
Nature meta-research project puts claims in social-science papers. I'm interested in Econ and Psych so I focused on that:
Econ had about the same rate of "not reproducible" analyses as Psych and a worse rate then Political Science.
nature.com/articles/d41586-0…
Robin Na retweeted
I'm recruiting a PhD student to join the Human AI Collaboration lab at @KelloggSchool @northwesterncs @NICOatNU
If you're excited about computational social science, LLMs, digital experiments, real-world problem solving, this could be a great fit
Please reshare!
Deets 👇
Robin Na retweeted
Join us Wednesday, November 20 for the @mit_ide lunch seminar with @CaoHancheng for his talk "Evaluating and Designing Computing Systems for the Future of Work."
Join in person or online - free and open to all!
In-person registration: eventbrite.com/e/fall-2024-i…
Online registration: eventbrite.com/e/fall-2024-i…
Robin Na retweeted
Is encouraging LLMs to reason through a task always beneficial?🤔
NO🛑- inspired by when verbal thinking makes humans worse at tasks, we predict when CoT impairs LLMs & find 3 types of failure cases.
In one OpenAI o1 preview accuracy drops 36.3% compared to GPT-4o zero-shot!😱
Robin Na retweeted
📢 I’m recruiting PhD students for Fall'25 in @Berkeley_EECS or @UCJointCPH! Please apply if you have related interests in AI for social good / networks / public health / social science
See my website for more on my research and how to apply: serinachang5.github.io/
Please RT!
Robin Na retweeted
🚨🚨🚨 VERY excited to share our new paper on how AI can facilitate democratic deliberation, published today in @ScienceMagazine! Together with @mhtessler, @summerfieldlab, and other amazing collaborators at @GoogleDeepMind we've been building the "Habermas Machine"
Robin Na retweeted
🚨New WP: Can LLMs predict results of social science experiments?🚨
Prior work uses LLMs to simulate survey responses, but can they predict results of social science experiments?
Across 70 studies, we find striking alignment (r = .85) between simulated and observed effects 🧵👇
Robin Na retweeted
Our new paper explains why some life outcomes are hard to predict. We combined machine learning, a mathematical decomposition, and in-depth qualitative interviews to understand the origins of unpredictability: pnas.org/doi/10.1073/pnas.23…
Robin Na retweeted
Join us for the final @mit_ide seminar of the semester w @m_sendhil & @asheshrambachan for their talk "Do LLMs Understand? How Would We Know if They Did? Uses and Misuses of Large Language Models."
💻Join online: bit.ly/ideseminar515
📍Join in-person: bit.ly/ideseminarvirtual515
Robin Na retweeted
Next Wed., April 10 join us for the @mit_ide lunch seminar with guest, @MinaLee__ on "Writing with Language Models" at 12pm ET.
💻Anyone can join online: bit.ly/ideseminarvirtual410
📍@MIT + @mit_ide members join in-person: bit.ly/ideseminar410
Robin Na retweeted
How are you using ChatGPT or Claude 3? We don't just throw a query at ChatGPT once; we do it sequentially, and ChatGPT makes sequential decisions. A natural question here is, does an LLM have the ability to make good sequential decisions? If so, why are pertained models good at sequential decision-making? If not, what methods should be used for training?
Our paper provides an answer to these questions.
arxiv.org/abs/2403.16843
Answers to these questions and more in this paper with Xiangyu Liu, Asuman Ozdaglar, and @KaiqingZhang
