@joabaum

Postdoc @StanfordNLP @StanfordAILab / Prev: @MilaNLProc @UZH_en @MPI_IS @CarnegieMellon. CompSocSci, LLMs, algorithmic fairness.

 Stanford
joe-baumann.com/
Joined February 2021
  • Tweets257
  • Following1.7K
  • Followers1K
21Media
We present SWE-chat: the first large-scale dataset of coding agent interactions from real users in the wild. In 40% of real coding sessions, the agent writes ~all the code. Users push back 39% of the time – agents almost never stop to check. Data, paper, & findings in the 🧵👇
Overview of SWE-chat. Left: a data collection pipeline diagram. Open-source developers install the Entire.io CLI tool, which logs their coding agent sessions and pushes the logs to a dedicated branch on their public GitHub repository. We discover and aggregate these logs into the SWE-chat dataset, with line-level attribution of which lines of code were written by the human versus the agent. Right: a growth chart showing cumulative logged events over time, rising steeply through early 2026. As of April 2026, the dataset contains 2.7 million logged events from over 200 repositories, including 63,000+ user prompts and 355,000+ agent tool calls across nearly 6,000 sessions.

ALT Overview of SWE-chat. Left: a data collection pipeline diagram. Open-source developers install the Entire.io CLI tool, which logs their coding agent sessions and pushes the logs to a dedicated branch on their public GitHub repository. We discover and aggregate these logs into the SWE-chat dataset, with line-level attribution of which lines of code were written by the human versus the agent. Right: a growth chart showing cumulative logged events over time, rising steeply through early 2026. As of April 2026, the dataset contains 2.7 million logged events from over 200 repositories, including 63,000+ user prompts and 355,000+ agent tool calls across nearly 6,000 sessions.

14
77
13
484
72,515
Time flies and I am looking for full time research positions! I am finishing my PhD at Cambridge early 2027. My research focuses on simulating human behavior with LLMs, LLM personalization, evaluating and shaping model behavior, and understanding human–AI interaction. Please get in touch if you are hiring in relavant areas!
2
17
1
73
4,785
Joachim Baumann retweeted
MilaNLP@MilaNLProcOct 20
Our lab @MilaNLProc has 6 papers accepted at EMNLP and workshops! 🎉 See you in Budapest 🇭🇺
6
4
2
17
1,217
Joachim Baumann retweeted
Debora Nozza@debora_nozzaOct 19
📢 I’m hiring! Excited about NLP & Computational Social Science? I have an open 2-year postdoc at @MilaNLProc on personalized & subjective NLP, focusing on everyday language technology use and their social & ethical impact 📅 Deadline: Nov 15, 2026 🔗 jobmarket.unibocconi.eu/?id=…
12
1
31
1,991
Joachim Baumann retweeted
Ricardo Olmedo@rdolmedo_Oct 12
With only ~$1,000 of training compute, we trained 𝗳𝗿𝗼𝗺 𝘀𝗰𝗿𝗮𝘁𝗰𝗵 a 1.7B coding agent that beats Claude 3 Opus and GPT-4o on SWE-bench Verified 🤯 At 25.5% pass@1, it sets a new record for nanoswe, our open SWE-bench speedrun. How is this even possible? 🧵👇
45
48
10
900
55,850
Joachim Baumann retweeted
clem 🤗
@ClementDelangueOct 8
We urgently need more public traces of AI agents attacking and defending systems. Defenders can’t learn from what they can’t see. If you have traces and are being pressured to keep them private, my DMs are open. Let’s level the playing field and fight the asymmetry and lack of transparency in AI!
136
138
24
1,286
64,439
Joachim Baumann retweeted
Weixian Xu
@WeixianXuOct 7
Language models already excel at math, but human growth remains essential: we need AI to teach people, not replace them! Teaching is different from solving. As the Chinese principle “因材施教” suggests, preferences differ from person to person, and a good teacher should adapt their teaching to each learner. We introduce Sherpa, a framework for training LLMs to teach adaptively.
1
20
5
113
15,507
Joachim Baumann retweeted
Hayoung Jung
@hayounggjungOct 6
👀 Today, we release our live evaluation dashboard for SciConBench: sciconbench.cs.princeton.edu… Frontier models keep getting released every 1-2 weeks with claims of huge capability gains. But are they actually getting better at synthesizing scientific conclusions in health? So far: surprisingly little. We’re going to keep measuring the performance of the *latest* frontier models on this high-stakes, real-world task over the next 1-2 years. Check out the thread here↓

SciConBench

A live agentic benchmark for open-domain scientific conclusion synthesis. Continuously evaluating frontier models on newly published systematic reviews.

sciconbench.cs.princeton.edu
Manoel@manoelribeiroOct 6
🚨SciConBench accepted at #NeurIPS2026! Paper: arxiv.org/abs/2606.11337 Can AI *synthesize* scientific conclusions in health? We find that, on medical conclusions, we are far from it! Check out our live dashboard: sciconbench.cs.princeton.edu… w/@hayounggjung and @korolova
1
5
1
31
1,403
Joachim Baumann retweeted
Caleb Ziems@cjziemsOct 6
I'm on the academic job market this year! Recently I've been working on scaling laws and human-centered interventions for building more socially grounded AI. If you're attending #COLM2026, please drop me a DM or email. I'd love to chat!
32
1
109
10,239
Joachim Baumann retweeted
Vishakh Padmakumar
@vishakh_pkOct 6
I'm on the job market this year! I work on measuring the societal impacts (overreliance, homogenization etc.) of AI/LLMs and proposing ways to mitigate these. I'm at #COLM this week, and #NeurIPS later this year, and would love to chat more!
6
53
7
184
13,239
Joachim Baumann retweeted
Manoel@manoelribeiroOct 6
🚨SciConBench accepted at #NeurIPS2026! Paper: arxiv.org/abs/2606.11337 Can AI *synthesize* scientific conclusions in health? We find that, on medical conclusions, we are far from it! Check out our live dashboard: sciconbench.cs.princeton.edu… w/@hayounggjung and @korolova
3
18
2
57
5,345
Joachim Baumann retweeted
Joachim Baumann
@joabaumOct 2
SWE-chat is a living dataset and v2 is 3.5x larger than v1. We will keep releasing more data in the future. Here are some of the amazing works enabled by SWE-chat v1: huggingface.co/datasets/SALT…

SALT-NLP/SWE-chat · Datasets at Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co
1
1
14
688
Joachim Baumann retweeted
Joachim Baumann
@joabaumOct 2
SWE-chat v2 is here! The largest dataset of coding agent interactions from real users in the wild has gotten even larger: 230K prompts from 18K sessions. SWE-chat has enabled incredible research (🧵) – excited to see what v2 unlocks for the community. Come find us at @COLM_conf!
2
24
4
76
19,841
Joachim Baumann retweeted
Joachim Baumann
@joabaumOct 6
I'm so excited to attend @COLM_conf and the @SnorkelAI Frontier Data Summit this week! Please reach out if you want to chat about coding agent data, human oversight, or AI evals! #COLM #COLM2026 frontierdatasummit.ai
4
11
1
63
5,513
I'm so excited to attend @COLM_conf and the @SnorkelAI Frontier Data Summit this week! Please reach out if you want to chat about coding agent data, human oversight, or AI evals! #COLM #COLM2026 frontierdatasummit.ai
4
11
1
63
5,513
xitter.cf/jennjwang/status/20999…
Jennifer Wang@jennjwangSep 15
Third-party auditors often audit models via the API, but do those findings transfer to chatbot interfaces? We audited seven systems across ChatGPT, Claude, and Gemini and found that they don’t. (EMNLP 2026)
2
6
177
Replying to @joabaum @COLM_conf @SnorkelAI @stai_research @stanfordnlp @StanfordAILab @StanfordHAI
xitter.cf/joabaum/status/2048805…
Joachim Baumann
@joabaumApr 27
We present SWE-chat: the first large-scale dataset of coding agent interactions from real users in the wild. In 40% of real coding sessions, the agent writes ~all the code. Users push back 39% of the time – agents almost never stop to check. Data, paper, & findings in the 🧵👇
Overview of SWE-chat. Left: a data collection pipeline diagram. Open-source developers install the Entire.io CLI tool, which logs their coding agent sessions and pushes the logs to a dedicated branch on their public GitHub repository. We discover and aggregate these logs into the SWE-chat dataset, with line-level attribution of which lines of code were written by the human versus the agent. Right: a growth chart showing cumulative logged events over time, rising steeply through early 2026. As of April 2026, the dataset contains 2.7 million logged events from over 200 repositories, including 63,000+ user prompts and 355,000+ agent tool calls across nearly 6,000 sessions.

ALT Overview of SWE-chat. Left: a data collection pipeline diagram. Open-source developers install the Entire.io CLI tool, which logs their coding agent sessions and pushes the logs to a dedicated branch on their public GitHub repository. We discover and aggregate these logs into the SWE-chat dataset, with line-level attribution of which lines of code were written by the human versus the agent. Right: a growth chart showing cumulative logged events over time, rising steeply through early 2026. As of April 2026, the dataset contains 2.7 million logged events from over 200 repositories, including 63,000+ user prompts and 355,000+ agent tool calls across nearly 6,000 sessions.

1
4
283
Joachim Baumann retweeted
Vishakh Padmakumar
@vishakh_pkOct 5
Come hear about SWE-Chat v2 at COLM (@COLM_conf) this week! More traces, more features and a tonne of interesting ways that the community has already adopted this dataset 🔥🔥
Joachim Baumann
@joabaumOct 2
SWE-chat v2 is here! The largest dataset of coding agent interactions from real users in the wild has gotten even larger: 230K prompts from 18K sessions. SWE-chat has enabled incredible research (🧵) – excited to see what v2 unlocks for the community. Come find us at @COLM_conf!
4
36
3,207
Joachim Baumann retweeted
Steven Dillmann
@StevenDillmannOct 2
We love SWE-chat by @joabaum !
Joachim Baumann
@joabaumOct 2
SWE-chat v2 is here! The largest dataset of coding agent interactions from real users in the wild has gotten even larger: 230K prompts from 18K sessions. SWE-chat has enabled incredible research (🧵) – excited to see what v2 unlocks for the community. Come find us at @COLM_conf!
1
2
8
799
Researchers from Stanford’s STAIR and SALT keep releasing real-world coding agent interaction data! 230K prompts and more than 2 million tool calls now available on 🤗: huggingface.co/datasets/SALT…

SALT-NLP/SWE-chat · Datasets at Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co
Joachim Baumann
@joabaumOct 2
SWE-chat v2 is here! The largest dataset of coding agent interactions from real users in the wild has gotten even larger: 230K prompts from 18K sessions. SWE-chat has enabled incredible research (🧵) – excited to see what v2 unlocks for the community. Come find us at @COLM_conf!
2
7
614
SWE-chat v2 is here! The largest dataset of coding agent interactions from real users in the wild has gotten even larger: 230K prompts from 18K sessions. SWE-chat has enabled incredible research (🧵) – excited to see what v2 unlocks for the community. Come find us at @COLM_conf!
2
24
4
76
19,841
more replies
Thank you so much to the team (@vishakh_pk @kevin_x_li & @jyangballin & @Diyi_Yang & @sanmikoyejo) and to everyone who has already used SWE-chat and given us helpful feedback! ❤️ @stanfordnlp @StanfordAILab @StanfordHAI @Stanford
1
9
303
We'll present SWE-chat next week at COLM. Find us on Wednesday, Oct 7, 2026 – 4:30 PM – 6:30 PM PDT in Grand Ballroom #111 Paper: arxiv.org/abs/2604.20779 #COLM2026

SWE-chat: Coding Agent Interactions From Real Users in the Wild

AI coding agents are being adopted at scale, yet we lack empirical evidence on how people actually use them and how much of their output is useful in practice. We present SWE-chat, the first...

arxiv.org
8
269
Load more