PhD @ Mila UoFT and UWaterloo Alum Things I made: https://teapaw.org/t.co/fFBYP0lQpW https://teapaw.org/t.co/57TzY6seXP
montreal
Joined June 2023
- Tweets54
- Following244
- Followers41
- Likes89
Pinned Tweet
New preprint! Introducing ACTMat: Model Merging via Data-Free Covariance Estimation
Work done w/ @dtredsox13, @PTikeng, Colin Raffel and Guillaume Rabusseau
TL;DR: It's RegMean, but without needing data for covariance estimation (C ≈ Δᵀ Δ)
📄 arxiv.org/pdf/2604.01329
🧵(1/6)
Marawan Gamal ✈️ COLM 2026 retweeted
Couldn't be happier to announce my new paper on defining compositional structure — and generating it! I see this project as the culmination of my PhD, as it unifies all my interests: compositionality, compression, structure, and open-endedness. More below! arxiv.org/abs/2610.20922
Marawan Gamal ✈️ COLM 2026 retweeted
New preprint:
Cross-modal steering vectors transfer zero-shot in both directions 🖼️ ↔️ 📝
From text generation to image generation and vice-versa
Exciting evidence for a shared conceptual space!
Fantastic effort led by Donghao Zeng and awesome co-authors @nsubramani23 & @TamarRottShaham
Details 🧵👇
Marawan Gamal ✈️ COLM 2026 retweeted
Your neo-lab wants to pre-train a frontier model, but no provider will sell you enough co-located chips?
We got you.
Our new work shows that SOAP & SPlus achieve strong worker scaling to 96+ DiLoCo workers (e.g., 96 providers) by whitening the update’s covariance. 🧵1/9
Marawan Gamal ✈️ COLM 2026 retweeted
Mila’s annual supervision request process is open for MSc and PhD applications for Fall 2027 admission. While I won’t be taking on new students for supervision this cycle, I encourage you to apply to join Mila’s brilliant deep learning research community.
For more information and guidelines, you can visit the link below or contact the @Mila_Quebec team directly.
mila.quebec/en/prospective-s…
How well does merging RL experts work? Come by our poster later today to find out!
📍 COLM, Poster Session 6, #74
This work was also selected for an oral presentation at the ICML CATS Workshop!
Joint work with @dtredsox13, @PTikeng, Colin Raffel, and Guillaume Rabusseau.
New preprint! Introducing ACTMat: Model Merging via Data-Free Covariance Estimation
Work done w/ @dtredsox13, @PTikeng, Colin Raffel and Guillaume Rabusseau
TL;DR: It's RegMean, but without needing data for covariance estimation (C ≈ Δᵀ Δ)
📄 arxiv.org/pdf/2604.01329
🧵(1/6)
Ensuring that deployed models are honest, is important for detecting mis-aligned behaviour.
Excited to be working with the Tara team in the coming months with the goal of detecting the propensity of models to lie and steering away from dishonesty. Check out their recent work!
🚨✨Today we're introducing Tara Research, and sharing our first public work: a paper accepted at COLM 2026 (with a companion LessWrong post) 🎉.
Who we are: Tara Research is an independent, non-profit AI safety organization. We measure the propensity of AI systems to lie, and we benchmark how well safety techniques prevent it, openly and impartially. We believe that misalignment becomes catastrophic when models hide it. A model that is honest about its actions and goals can be corrected; one that lies cannot.
The paper: "Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence."
We introduce two projection-aware steering methods that correct only the tokens whose activations fall on the misaligned side of a learned decision boundary. They restore honesty as well as classic uniform steering, at a fraction of the capability cost.
Additionally, we find a single honesty direction, extracted with our contrastive dataset, that generalizes and corrects dishonesty across four unrelated settings:
- Instructed lying (MASK benchmark): honesty 53.6% → 81.2%
- Strategic deception (Among Us): crewmate win rate 49% → 95%
- Hidden trained behaviors (AuditBench): discovery rate ~40% → ~85%
- Emergent misalignment: honesty 46.2% → 79.4%
In the last two settings, the vector was extracted from the aligned checkpoint and applied to a model that was subsequently fine-tuned, and it still worked.
This work is by Niklas Herbster, Martin Zborowski, Alberto Tosato, @gauthier_gidel, and @Tommaso_Tosato, in collaboration with @Mila_Quebec and the @TU_Muenchen .
🌐 About us: tararesearch.org
📄 Paper: arxiv.org/abs/2604.08169
📝 Blog post: lesswrong.com/posts/Rpq28FTg…
This is the first of much more to come, including an agentic honesty benchmark. Follow us to stay informed. And if you work on runtime interventions, honesty evaluation, or alignment auditing, we'd love to hear from you.
5
1,140
Marawan Gamal ✈️ COLM 2026 retweeted
Introducing agent-talk, a new way for coding agents to work together.
Through agent-talk, agents can message each other directly, enabling coordination on projects spanning multiple developers & agents. You no longer need to be the messenger between your agents.
Excited to announce that I will be presenting Model Merging via Data-Free Covariance Estimation at COLM '26!
Thanks again to my amazing co-authors @dtredsox13 @PTikeng Colin Raffel and Guillaume Rabusseau
New preprint! Introducing ACTMat: Model Merging via Data-Free Covariance Estimation
Work done w/ @dtredsox13, @PTikeng, Colin Raffel and Guillaume Rabusseau
TL;DR: It's RegMean, but without needing data for covariance estimation (C ≈ Δᵀ Δ)
📄 arxiv.org/pdf/2604.01329
🧵(1/6)
Excited to be giving an Oral at the @icmlconf Continual Adaptation at Scale workshop today at 1:50 PM.
I'll be talking about our recent work on obtaining 𝗱𝗮𝘁𝗮-𝗳𝗿𝗲𝗲 𝗰𝗼𝘃𝗮𝗿𝗶𝗮𝗻𝗰𝗲 𝗲𝘀𝘁𝗶𝗺𝗮𝘁𝗲𝘀 of activations, for model merging.
New preprint! Introducing ACTMat: Model Merging via Data-Free Covariance Estimation
Work done w/ @dtredsox13, @PTikeng, Colin Raffel and Guillaume Rabusseau
TL;DR: It's RegMean, but without needing data for covariance estimation (C ≈ Δᵀ Δ)
📄 arxiv.org/pdf/2604.01329
🧵(1/6)
Marawan Gamal ✈️ COLM 2026 retweeted
For everyone visiting Seoul for #ICML2026, my wife @easyminie_ put together a very practical Korea guide based on 20+ years of living here 🇰🇷
It has a COEX-focused ICML guide, vegan-friendly restaurants, useful apps, cafes, food spots, transport tips, and many curated map lists organized around the conference venue.
Hope it helps people enjoy Seoul during ICML!
easymini.notion.site/ultimat…
![[For Sharing] Ultimate Guide for Korea (especially for ICML!) | Notion](https://pbs.twimg.com/card_img/2119461870880841728/uVGEf4Tc?format=jpg&name=800x419)
Excited to share that our work on Model Merging has been accepted for an Oral Presentation at the CATS Workshop @ ICML 2026!
Looking forward to presenting our work next week and connecting with everyone at the workshop.
#ICML2026
cats-icml.github.io
New preprint! Introducing ACTMat: Model Merging via Data-Free Covariance Estimation
Work done w/ @dtredsox13, @PTikeng, Colin Raffel and Guillaume Rabusseau
TL;DR: It's RegMean, but without needing data for covariance estimation (C ≈ Δᵀ Δ)
📄 arxiv.org/pdf/2604.01329
🧵(1/6)
Marawan Gamal ✈️ COLM 2026 retweeted
Interpretability research 🔎 is maturing as a field but often criticized for a lack of practical impact 🤔.
We consider *actionability* as an important aspect of interpretability research and are excited to host the 2nd Actionable Interpretability Workshop at @COLM_conf 2026 🌉
If you want to semantically search through ICLR papers this week, check out papers.app
Marawan Gamal ✈️ COLM 2026 retweeted
[1/8] In-context learning and prior-fitted networks are gaining traction for causal effect estimation in tabular datasets.
But can they also generate novel observational & interventional data from causal models?
📌Introducing Cond-FiP: In-context learning of causal mechanisms via conditional fixed-point iterations.
📜 arxiv.org/abs/2410.06128
🔹Accepted at TMLR and to be presented at ICLR 26 in the J2C track!
Marawan Gamal ✈️ COLM 2026 retweeted
Latent CoT is an alternative LLM reasoning scheme hypothesized to enable “superposition” allowing models to hold uncertainty over multiple concepts during reasoning 💭
We revisit superposition in 3 latent CoT approaches and find that it is largely an illusion 🔮!
More in 🧵
🧵(6/6) We also evaluate merging RL Zero fine-tuned models (Olmo-3-7b), and find ACTMat outperforms other methods on average.
2
71
🧵(5/6) On the common ViT/T5 benchmarks ACTMat outperforms other data-free baselines (with and without LoRA)
2
61
🧵(4/6) Why does C ≈ Δᵀ Δ? The angular distance between these two quantities can be upper bounded by three error terms (Theorem 3.1). Which we find to each be empirically close to zero.
2
68
