ph.d. @mit | building @openevidence (we're hiring!) | formerly @google @uwmadison | #nlproc and loud music
Miami, FL
Joined March 2020
- Tweets34
- Following154
- Followers461
- Likes154
I will be at ICLR this week to share this work! Stop by and say hi, or let me know if you want to chat about LM interpretability :)
Linearity of relation decoding in transformer LMs (led by *newly-minted Dr.* @evanqed and @arnab_api)
> Understanding how LMs transform internal knowledge representations to output predictions
arxiv.org/abs/2308.09124
It's been awesome to see this work come together - an agentic interpretability system that iteratively experiments on a model to understand how it works!
I believe "LMs with tools" is a powerful paradigm for auto interp. Lots of room for new tools + LMs to increase the scope!
Accepted to #ICML2024!! 🚀
Meet MAIA- a Multimodal Automated Interpretabilty Agent that helps users understand AI systems.
Given a user query (eg "label a model’s feature"), MAIA designs experiments iteratively by forming and updating hypotheses based on experimental results.
Evan Hernandez retweeted
Thanks @_akhaliq!
MAIA is next in our line of work on Interpretability Agents that interrogate other models, and the functions of their components, using iterative experimentation.
Project page: multimodal-interpretability.…
w/ @TamarRottShaham @f_x_wang @AchyutaBot @evanqed @jacobandreas @ antoniotorralba
You’ve heard of model editing, now get ready for 📝 representation editing 📝
In our new paper, we find directions in LM rep space that make the LM assert a fact is true.
Paper: arxiv.org/abs/2304.00740
Code: github.com/evandez/REMEDI
w/ @belindazli @jacobandreas
In fact, those directions also do an okay job of detecting errors in the LM’s background knowledge-- which suggests LMs could naturally represent state info and factual info in similar ways.
Evan Hernandez retweeted
Exciting to see @OpenAI scaling up @evanqed & co's work on learned models for neuron labeling (milan.csail.mit.edu/)! One technical observation, and one practical one:
Evan Hernandez retweeted
The NSF has invited us to propose an $18m (RI-1) project for infrastructure to help **you** (academic researchers) to study large language models.
What should we build to help your LLM research?
Respond with your thoughts.
(Or answer 3Q's on a form forms.gle/s7EDW4V12mR7U3Xs7)
🧵
Proud of Eric + team for pushing this work! You don't need a giant LM for everything. Relatively small (< 1B param), specially pretrained language models are effective for processing specialized text like EHR notes. Can beat ICL+big models with only O(100) FT data points
Recent results have suggested that LLMs encode a surprising amount of clinical knowledge. These results raise an important question about the role of smaller specialized clinical models: do we still need clinical LMs? We explore this question in our paper: arxiv.org/abs/2302.08091
Check out our #ICLR22 oral: MILAN, a procedure for generating language descriptions of neurons!
With @cogconfluence @davidbau @ teona @ torralba @jacobandreas
📄Paper: arxiv.org/abs/2201.11114
💻Website: milan.csail.mit.edu
🔍Demo: milan-demo.csail.mit.edu
Ultimately, MILAN is another tool for our interpretability toolkit. The irony is not lost on us that it relies on a black-box model to understand black-box models. But we think it serves an important purpose in helping practitioners make decisions about their own models.