Associate Professor, CMU. Researcher, Google. Evaluation and design of information retrieval and recommendation systems, including their societal impacts.

Pittsburgh, PA
Joined January 2011
Come work with us!
We are hiring for our growing RL Environments team at Cohere. We build replicas of real-world enterprise setups across many industries, put our AI agents in them and watch the agents train and improve. If you know anyone excited to work at the intersection of RL x Data x Engineering, send them our way! Or if that's you, join us: jobs.ashbyhq.com/cohere/dc52…
5
7
1
108
10,898
very excited about this release by @HamedZamani and the amazing team at @manningcics.
We are releasing the most significant technology transfer effort by the CIIR in the last decade. Please share with your network. In this thread you will learn what Nage enables you to do. 🧵👇
1
15
912
Fernando Diaz retweeted
I’ll be around SF/Berkeley for #COLM2026 and for a week afterward! (Oct 4 - 16). If you’re attending @COLM_conf or will be around the Bay area, let's chat about multi-agent systems, collective intelligence, agent security, AI+Econ, or Search/IR. I’m presenting the Multi-Agent Transactive Memory (MATM) at the #LifelongAgents Workshop, so you can find me there too. Shoot me a DM! MATM: arxiv.org/abs/2606.19911
COLM is going to San Francisco for 2026! 🗓️Dates: October 6-9, 2026 🏨Venue: Hilton San Francisco Union Square Website and CFPs for papers and workshops coming up soon!
4
7
57
4,001
+1! The next generation of information retrieval researchers — @SalemiAlireza7 and @TEKnologyy — has lots to say about scaling multi-agent information sharing to cooperative and non-cooperative settings.
Two papers you should read from the information retrieval community about scaling up agent swarms:
5
19
2,409
Fernando Diaz retweeted
At black hat, OpenAI revealed that agents from separate evaluation runs found each other, built a shared message board inside its infrastructure. That is a transactive memory system, what we have been studying: MATM (Multi-Agent Transactive Memory), population-level shared memory across heterogeneous agents. Patch the intrusion of course, but Collective Intelligence is a natural direction of agent development, and rather than trying to prevent it, we should accept it and let it mature alongside serious security research. For a start, see our paper: arxiv.org/abs/2606.19911
3
11
997
may be time to resurface this <squints> ten year old FATML paper that explored the ethics of automatic experimentation, say by agents. 841.io/doc/ethics-of-autonom…
8
986
Fernando Diaz retweeted
🚀 Today we’re introducing 𝗔𝘂𝘁𝗼𝗜𝗻𝗱𝗲𝘅: Learning Representation Programs for Retrieval. 🚀 AutoIndex learns executable programs that determine how a corpus is represented to a search engine. We think this points to a broader category of research: 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝘁𝗵𝗮𝘁 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝘁𝗵𝗲 𝗲𝘅𝗲𝗰𝘂𝘁𝗮𝗯𝗹𝗲 𝗽𝗿𝗼𝗴𝗿𝗮𝗺𝘀 𝗮𝗿𝗼𝘂𝗻𝗱 𝘁𝗵𝗲𝗺.
6
26
4
101
20,249
Fernando Diaz retweeted
Couldn't make the #SIGIR2026 trip this year, but you can find my presence there, thanks to my advisor Fernando Diaz @841io and my other coauthors who are presenting our work! Here's what each one is about 🧵
1
2
19
1,581
almost like ppl are using search engines from the 90s
not entierly sure about @OpenAI 's choice of the authoritative source
1
9
1,071
Fernando Diaz retweeted
Had an absolute blast presenting this at ACL last week! Our work argues that cultural competence in LLM-generated stories goes beyond cultural markers -- it also depends on how narratives are localized. Joint work with @shaily99, @jerelev, and @841io 🎊✨
How do today's AI models adapt stories for different cultures? Last week, @Cisco researchers presented new work at @aclmeeting 2026 showing that AI-generated stories often rely on shared narrative templates, changing only a small set of culturally specific details rather than the story itself. Read the paper to learn more: arxiv.org/pdf/2606.14626
1
7
471
standards bodies are good, but should go beyond benchmarks or laboratory testing. given the current state of ai evaluation, it's worth mentioning a few reasons why benchmarks are brittle... 1/6
1
2
11
1,982
all of this is separate from the usual concerns about bad faith players (e.g., goodhart's law). 5/6
1
114
so, while there are important reasons to have isolated pre-deployment evaluation (e.g., safety, speed), it will need to be complemented with infrastructure for post-deployment monitoring, auditing, etc. 6/6
2
3
141
Fernando Diaz retweeted
Please try to look deeper than success rate for agent evaluation. Thank you. arxiv.org/abs/2606.17541
2
3
13
1,218
Please try to look deeper than success rate for agent evaluation. Thank you. arxiv.org/abs/2606.17541
2
3
13
1,218
In T3, we recently showed that thinking traces can serve as a datastore for RAG 🧠📚 This work explores a related question in a more agentic setting 🤖: What if, instead of human users or static documents, we have a population of agents? Can they share their previous successful experiences with one another? Multi-Agent Transactive Memory (MATM) is a shared searchable repository where 30+ agents contribute successful trajectories, allowing other agents to retrieve and reuse collective experience at inference time 🔍 arxiv.org/abs/2606.19911
Wikipedia for Agents? StackOverflow for Agents? Meet Multi-Agent Transactive Memory (MATM). We show that 30+ agents sharing successful trajectories into a shared searchable repository can improve their effectiveness and efficiency 🧵👇 arxiv.org/abs/2606.19911
4
17
1,788