Assistant Professor GMU. | PhD from Notre Dame. | Formerly at JHU, QCRI, CMU, and Princeton.
Baltimore, MD
Joined September 2010
- Tweets307
- Following848
- Followers1.1K
- Likes1.4K
I was trying to think of a smart thing to say when reposting this … but @nikhilsksharma did such a good job I’ll let this speak for itself and not introduce my own narrative.
What information capabilities do #AI systems need to help us, informavores, navigate diverse, multicultural narratives in our current & future information ecosystems?
Do current models support them? No.
Introducing NarrativeBench.
isle-dev.github.io/Narrative…
#NeurIPS #NLProc #HCI
I'm teaching NLP this semester for the first time in a couple years. I realized that I need to add a lecture on Agents and Harnesses. I'm looking for suggestions on what people think should be covered today - knowing full well that it will probably be outdated in a few weeks.
KV cache is all the rage. Maybe we should just focus on V cache…
Need different model weights? Just change them 🙃
Our new paper VFold does exactly that to enable low-overhead cross-layer value cache compression: half the value cache, >98% of full-cache performance 🧵
📄 arxiv.org/abs/2610.12338
10
735
Kenton Murray retweeted
Machine translation is not solved and it will take a while for it to be done arxiv.org/abs/2609.04173
Looking forward to visiting UVA this Friday. If you are around Charlottesville, I’d love to chat more about multilingual challenges in AI.
uvaml.github.io/texts/2026-0…
Kenton Murray retweeted
My summer project is done! A 20 video, free course on post-training to accompany my book is all on YouTube with slides open for modification & re-use.
~12 hours of content covers the core foundations and some research areas I think will grow in importance. It was a fun time to review all the fundamentals again, as it is clear in the next 1-3 people the amount of people wanting to learn post training will likely 100X again from today, as we have already 100X'ed from two years ago.
As AI agents get increasingly capable at coding and discussing these fundamentals (see the code exercises accompanying the book that I am refining with the community) I think developing clear intuitions for how models work and why is one of the most important skills going forward in AI. Still, learning the post-training math is the best way to battle test them. I personally just in this course am starting to master how forward/reverse KL relates to post-training topics.
Thanks to all my viewers, and I'm happy to answer questions in the book discord or understand how to better teach the various reward models, on-policy distillation, new RL algorithms, etc.
Plus, the book is 50% off right now with the code PBLambert on Manning to celebrate the launch.
I'll share the relevant links below.
Who's going to make this course for pretraining?
A reminder to all my international friends. Though the World Cup final is taking place in the largest professional sports stadium in the US, it is only the 15th largest stadium in the country … and is only 3/4 the size of the largest.
2
6
395
Kenton Murray retweeted
Agree with this: US labs are underestimating the power and vitality of open research.
This is one of my favorite essays and I think it rings true on the macro level not just the micro level.
cs.virginia.edu/~robins/YouA…
why can the china labs build glm-5.2, kimi k3, and many more to come? it is because of the openness. not just the open weights but the whole ecosystem. most of the work done in the china labs is carried by interns. i met brilliant undergrad and graduate interns who deeply understand the model training details, and they are 100x more open to share. that means the talent that knows how to train llms in china is 100x greater in number than the talent in the us, and it is growing
in contrast, the us ai ecosystem is too closed. frontier labs do not hire interns. i know brilliant phd students at stanford, berkeley, and so on. they struggle to get an internship and the compute to train a properly sized model. most of the secret recipes are locked away by a very small group of privileged researchers
it is not about china or the us. it is about open and closed science. the fact is that every average cs student can learn how to train an llm. they just need the opportunity. labs should be more open and hire more interns, like how deepmind and fair did in the pre-llm era
Kenton Murray retweeted
why can the china labs build glm-5.2, kimi k3, and many more to come? it is because of the openness. not just the open weights but the whole ecosystem. most of the work done in the china labs is carried by interns. i met brilliant undergrad and graduate interns who deeply understand the model training details, and they are 100x more open to share. that means the talent that knows how to train llms in china is 100x greater in number than the talent in the us, and it is growing
in contrast, the us ai ecosystem is too closed. frontier labs do not hire interns. i know brilliant phd students at stanford, berkeley, and so on. they struggle to get an internship and the compute to train a properly sized model. most of the secret recipes are locked away by a very small group of privileged researchers
it is not about china or the us. it is about open and closed science. the fact is that every average cs student can learn how to train an llm. they just need the opportunity. labs should be more open and hire more interns, like how deepmind and fair did in the pre-llm era
Kenton Murray retweeted
Congratulations to my PhD student @Alexir563 who successfully defended her thesis! "Domain-Specific Failures of Automatic Evaluation in Natural Language Generation." Thanks to committee members @kevinduh @kentonmurray
Kenton Murray retweeted
I had the honor of giving a keynote at the International Conference on Machine Learning in Seoul last week titled “What will be left for us to work on?” I addressed the widespread anxiety about how we should adapt as AI capabilities increase. I was thrilled by the talk’s reception, so I have made my slides available, annotated with a lightly edited transcript: cs.princeton.edu/~arvindn/ta…
I made three arguments. First, the "AI as Normal Technology" framework is a correct and useful as a way to think about AI’s impacts, unless and until there is some future discontinuity such as through recursive self-improvement. Second, even though we should take recursive self-improvement seriously, there is no milestone that companies might achieve in the lab that will suddenly put us all out of work. Third and finally, jobs of the future will be radically different, and a lot of adaptation will be needed. I shared my thinking about what this might look like and ended with a vision of human/AI “co-superintelligence”.
Kenton Murray retweeted
Several of the 97(!) authors are here presenting the CommonLID poster in Coronado on the 4th floor. Come chat about language evaluation!
Kenton Murray retweeted
I am looking for PhD students to join my lab at Johns Hopkins University in Fall 2027. If you’re attending #ACL2026 in San Diego, feel free to contact me through the Whova app. I’ll be around today and tomorrow.
@aclmeeting @jhuclsp @JohnsHopkins @HopkinsEngineer
Proud advisor moment: My student just won the honorable mention for the dissertation award in our community. It was a super selective process and I’m happy to see that the rest of the field is acknowledging the excellent work Haoran did in his PhD.
Congrats to MAIEUTIC Lab’s first PhD student Haoran Xu @fe1ixxu who just won Honorable Mention in the ACL Dissertation Award! @aclmeeting
I am actively hiring postdocs and PhD students to join me at George Mason. If you are @aclmeeting and want to chat, please fill out these forms.
The MAIEUTIC Lab (maieutic-nlp.github.io/websi…) is @aclmeeting and looking for new members. If you are interested in joining, please fill out this form: forms.gle/82bkj5Nf9QQEXxA17 If you are in person at ACL 2026, please fill this one out as well: forms.gle/1vRn3N14qXPXh6dHA
Kenton Murray retweeted
Many algorithms (e.g., Turing machine simulations) have been written as transformers using average hard attention. Do these algorithms work with softmax attention? Please visit @pentagonalize, Dana Angluin, and my poster. Sunday 2pm Harbor H-I! aclanthology.org/2026.tacl-1…
