Mohamed Aghzal

GMU-Veritaserum🥳🥳🥳 We had a fun summer (led by @Skywalkerx47 with @aghzalm)!

Thanks for featuring our work! The earliest work I can think of about agentic interpretability is MAIA by @TamarRottShaham et al. arxiv.org/pdf/2404.14394. Other than those listed in the post, more recent works include: - @GoodfireAI's goodfire.ai/blog/you-and-you… - Liu et al. and @DuMNCH arxiv.org/pdf/2601.03671 - @Elenal3ai et al. arxiv.org/pdf/2602.18458 - @Arnauya and @javifer_96 arxiv.org/pdf/2605.01555 - @tal_haklay et al. arxiv.org/pdf/2603.20101 These works vary in scope (e.g., behavioral vs. mechanistic interp, feature level vs. circuit level, discovering vs. verifying, etc.). Just like other agent applications, benchmarks and systematic evaluations are needed for this line of research to be trackable. Meanwhile, data leakage/memorization (e.g., LLMs giving good explanations about well-known explanation facts because of their memory from training, not their reasoning) is a key concern, which was discussed in Haklay et al.'s paper and also motivated us to consider a semi-synthetic benchmark for now.

Honored to receive the NSF CAREER Award on the topic of "Towards Scalable and Actionable Interpretability of Language Models for Code Generation"! Thank you to @NSF for this recognition and all the support ever since my PhD study, and to my students, collaborators, colleagues, advisors/mentors, and everyone else who has helped me and inspired me in my career! This idea is a convergence of my research topics started during PhD study at @osunlp (code generation) and after I joined @GMUCompSci (interpretability). The terms "scalability" and "actionability" are widely discussed in the interpretability community today. I'm excited to continue exploring these topics in the coming years! Finally, special thanks to my students; the award belongs to you as well: - @DakingRai on interpretability - @yue_murong21307, @aghzalm, and @salokr_deep on LLMs and LLM agents - Hao Yan on code generation (Pictures taken when we celebrated Saurabh's graduation in a lovely Mexican restaurant)

Missing #ACL2026 (having a 6-month-old👶 to take care at home), but my student @aghzalm will be presenting our work about Web Agents tomorrow at 11 am in poster session A. Go chat with him about: - Web agents - LLM planning and reasoning - Mechanistic interpretability - or soccer/world cup⚽️ Paper: aclanthology.org/2026.acl-lo…

🌟Our new paper seeks to address the scalability issue of the current circuit study in #MechanisticInterpretability of #LLMs, a foundation for LM interpretability to be actionable. A circuit study consists of generally two phases: localizing the circuit components, and explaining what they do (e.g., knowing that a head is a name-mover head). While localization has been automated by techniques such as ACDC, EAP, and EAP-IG, circuit explanation remains largely a manual process. Given how LM agents are advancing AI4science or AI4research these days, we wonder: Can LM agents automate circuit explanation and assist interpretability research? Our project funded by @foresightinst gives a positive answer! Check out 🧵below for: - A synthetic benchmark AgenticInterpBench (extending from Lindner et al.'s Tracr and Gupta et al.'s InterpBench) - An LM agent framework HYVE (Hypothesize, Validate, and Explain) for circuit explanation - Results with 4 LM backends (GPT, Claude, Gemini, Qwen) - Application to explaining the AF1 circuit in Llama3-8B (arxiv.org/pdf/2509.09650) The work complements existing efforts in Automated Mechanistic Interpretability, generalizing it from feature explanation to circuit explanation. We release dataset and code for more exploration along the line! Paper: arxiv.org/pdf/2606.24026 Dataset and code: github.com/Ziyu-Yao-NLP-Lab/… Work proudly led by my first-year PhD student @Skywalkerx47 and collaborated with @hhsun1 @hkohli14 @yuekun_yao at @osunlp

🙌Sharing our latest work to appear at #ACL2026 diagnosing LLM web agents from a hierarchical planning perspective. We answer the following questions: - Do the current web agents plan at the high level in the same way as humans do? - If not, how to improve this alignment? - If a web agent makes a good high-level plan, can the task be solved at the low level? - Finally, can replanning help the agent resolve its mistake and get back on the right track? 🌟Our contributions: We propose a new high-level plan representation based on PDDL. We found that: - Planning with PDDL can produce higher-quality high-level plans, - but even with perfect plans, execution at the low level remains the critical bottleneck. - Replanning boosts success but hurts plan alignment, though PDDL is more robust here too. We hope this breakdown analysis brings clarity to the challenges remained in this rapidly evolving field and we advocate for more such fine-grained analysis in agent development. Paper: arxiv.org/abs/2603.14248 Code: github.com/Ziyu-Yao-NLP-Lab/… Work led by @aghzalm and with @GregoryJStein @GMUCompSci #ACL2026 #WebAgent #LLMAgent