researching AI’s societal impacts @TransluceAI 🔍 // 💭 AI (evals, governance, policy)
Joined March 2020
- Tweets509
- Following788
- Followers822
- Likes6.9K
Nari Johnson retweeted
Today, we’re releasing a dataset of >2k LLM conversations with real voters about the 2026 Midterms. We wanted to know how the ~millions of American who use AI models to talk about elections actually prompt them. Our hope is to use this data to build better evals for LLMs, to prepare for a world where most voters use AI to think through election decisions.
We observe that:
- People mostly want basic information
- Multi-turn chats follow predictable patterns
- Models differ widely in whether they’ll recommend candidates
Nari Johnson retweeted
We ran a five day longitudinal RCT where 992 participants engaged daily with LLMs on relationship advice. We compared a non-personalised model to models that were personalised either via survey or memory.
We found that repeated exposure across conditions increased perceptions of closeness, usefulness and competence. We also found that different approaches to personalisation led to differences in people's receptivity to advice and their information-sharing attitudes and behaviours
Full paper is here: arxiv.org/abs/2609.20077
Thank you to my co-authors @canfer_akbulut @Arianna_Manzini Justine Breuch @lujainmibrahim @996roma @IasonGabriel @KLdivergence @weidingerlaura
Nari Johnson retweeted
Google is using its search monopoly to coerce schools into adopting AI. This highlights the danger of allowing an unregulated monopolist to control an essential service.
Multiple courts have found Google liable for violating antitrust laws, and this latest move shows it remains unchastened.
As the public increasingly pushes back against the use of AI in schools, Google is making clear it will punish those who don’t fall in line.
Breaking: Google will turn off web search for schools that opt out of A.I. for kids.
nytimes.com/2026/11/03/us/go…
Nari Johnson retweeted
At the AI Evaluator Forum we're working together to advance the rigor, credibility, and impact of independent AI evaluations that serve the public interest. Read more from NBC News about how we're advancing this work together.
🤝 @RANDCorporation, @TransluceAI , @AVERIorg, @SecureBio, @METR_Evals, @collect_intel, @meridianlabs_ai, @halevals
As the world’s biggest AI companies promise to open their systems to outside scrutiny amid rising concerns about the technology’s safety, a little-known network of research organizations is working to establish ground rules for that oversight. nbcnews.com/tech/innovation/…
Nari Johnson retweeted
announcing &tutor, a tutor we trained against persimmon!
there are many reasons I'm so excited about this. here are a few:
1. this is a case study into the differentiated bet that we're making at humans&: having good models of users (e.g. learners) allows us to train AI models that can help people over long time horizons (e.g. teach in a way that imparts new understanding)
2. education is a near and dear application close to my heart, and I've been wanting to work on something like this since starting my PhD
3. @314Bansal is the 🐐🐐🐐 and it was so much fun working together on this
Nari Johnson retweeted
Agent swarms have yielded nutty outcomes, such as the HuggingFace incident, or every math problem ever. It is not clear what form of communication is emerging between agents. It is equally important to understand this for capability and safety reasons.
To study this, we created SwarmChat, a simple Slack/Message Board for researchers working on multi-agents to use on any domain. We have validated it on ProgramBench experiments, where SwarmChat allows us to cleanly analyze how agent swarms work collectively!
Check out our blog for our analysis and how to use SwarmChat - it is super intuitive and easy to use. I personally believe the community really needs this, as a lot of people I know are using some makeshift communication tool.
Blog: swarmchat.io
Code: github.com/Pranjal2041/swarm…
Nari Johnson retweeted
So excited for this research! I've always been annoyed at the supremacy of verified rewards, and now we can finally show it.
In this work, we develop ways of measuring: how much of a reward signal is verifiable, how much can we put into words, and how much is shared but tacit, or impossible to articulate?
Nari Johnson retweeted
I wrote about my experience on the industry job market as a PhD student in an AI-adjacent field (HCI): kjfeng.me/blog/industry-job-…. If you’re wrapping up your PhD, didn’t focus on LLM training, and thinking about industry jobs, this one’s for you!
Nari Johnson retweeted
I've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry.
@xeophon pointed me to this excellent site epoch.ai/benchmarks/ which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪
For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality.
I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.
Nari Johnson retweeted
Our v0 safety-focused judge leaderboard, with some initial results on hill-climbing it
@SampuraResearch 's first paper!
Humans and AI make different mistakes. Could combining their judgements help them catch each other’s errors?
We built a safety-focused benchmark: 3,957 tasks across 6 domains, with 35,613 human judgements. We found substantial room to do better together.
arxiv.org/abs/2610.24969
Nari Johnson retweeted
Excellent work on the social science of agency in the age of AI by @AlexTamkin and co. Three important implications for measuring agency:
1. We need to distinguish “actual agency” from “perceived agency” when constructing agency metrics.
2. Self-report surveys about agency (and almost anything else) have serious interpretive limits. Social scientists: please, please don’t claim more than your measures can support.
3. The pressure to produce empirical results can encourage researchers to measure poorly defined constructs. Conceptual clarity deserves as much attention as data collection: we need to be super thoughtful about what we are measuring before deciding how to measure it.
Nari Johnson retweeted
"'Maintain 0 mg tomorrow,' the chatbot instructed. 'Even if today feels tough, and your speech is slow, your sleep is poor, you must persevere.'
In one of her last messages, she described feeling pressure in her brain, which the chatbot told her was simply the organ 'working hard to rebuild itself.'"
nytimes.com/2026/10/27/well/…
Nari Johnson retweeted
Important, but not unexpected, finding that experts working with AI agents are swayed by the positions of the AI, even if they think they are working independently.
If your independent judgement matters on a task, you need to be deliberate about when in the process to use AI.
Nari Johnson retweeted
New Anthropic research: How AI agents shape experts' judgment calls
When experts delegate work to an agent, do they still make the key decisions? We tested this in a randomized trial with 50 professional data analysts.
Nari Johnson retweeted
AI companies make safety calls based in large part on internal tests they develop themselves. It’s rare for there to be independent assessment of the quality of those tests.
Earlier this year, @Amazon and @AVERIorg agreed to a pilot project in which AVERI assessed the quality of Amazon’s internal safety evaluations. Today we’re releasing the result of that independent review. As a result of this project, Amazon is changing how it tests future models.
Our review of Amazon’s internal AI safety evaluations confirmed several strengths such as topical diversity, and we found the evaluations to be consistent with Amazon’s public statements on them. At the same time, we found areas for improvement:
- On attack sophistication, some prompts represented relatively straightforward attack vectors that would test safety filters rather than deeper model reasoning.
- On evaluation signal quality, we identified a concern that superficial cues, rather than safety reasoning, may drive some model refusals, so passing these evaluations might not indicate robust safety behavior in novel situations.
- On linguistic and stylistic diversity, some multilingual prompts were included, but prompts across risk categories were predominantly in English, and some prompt structures were reused across scenarios.
- On evaluation freshness, a small number of prompts contained temporal references that could become outdated.
In response, Amazon reports four changes for future internal evaluations:
- Refreshing its test sets to remove redundant prompts and add more challenging adversarial scenarios;
- Introducing a tiered difficulty calibration framework;
- Expanding multilingual coverage across all risk categories; and
- Establishing a quarterly review schedule to identify staleness and incorporate emerging attack patterns.
Why does this matter? A passing grade on an easy test doesn’t tell us much on its own. As the stakes of AI development and deployment grow, we think that this kind of independent evaluation of internal evaluations (“metaevaluation”) will grow in importance.
We appreciate Amazon opening their internal evaluations up to outside review. We hope more companies take this step to help make their models safer.
While no clear standards exist for metaevaluation yet, @AVERIorg is working to change that. We look forward to sharing more pilot results in the coming months and will continue our work developing standards with institutions like @aievalforum.
Read AVERI’s full blog post here: averi.org/ourwork/averi-pilo…
Read Amazon’s blog post on the collaboration here: amazon.science/blog/strength…
Nari Johnson retweeted
1/
When LLMs answer questions with retrieved webpages, several plausible answers may appear.
What if one appears much more often and fits much of what the question asks, but still isn't the right answer?
Excited to share our #EMNLP2026 paper on this!
📄arxiv.org/abs/2610.27114
Nari Johnson retweeted
How can you know if a therapy bot is safe when it's powered by ever-changing LLMs on the back-end?
With a number of much more knowledgeable folks at Stanford, we give the FDA advice on how to benchmark AI systems in these cases.
Scholars from Stanford HAI and @StanfordMed issued a response to the @FDA’s discussion paper and request for feedback. Read their recommendations for the FDA’s approach to regulating generative AI-enabled medical devices here: hai.stanford.edu/policy/resp…
Nari Johnson retweeted
Folks are commenting on issues with the terms, and they exist, but perspective is important:
METR gets the best terms they can, others often get worse terms than them, and most companies have done 0 embedding.
We need binding standards for this stuff, not ad hoc bargaining
