Michaël Eli Sander (@m_e_sander) header
Michaël Eli Sander (@m_e_sander) profile picture

Research Scientist at Google DeepMind

Paris
Joined February 2021
Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)Photo by Michaël Eli Sander (@m_e_sander)
Michaël Eli Sander retweeted
Teacher forcing is the standard way to train AR models from data but it is known to introduce exposure bias: the model is not trained to use its own predictions for inference. To address this, we studied the possibility to design sequence-level losses. arxiv.org/abs/2611.16736
5
26
1
138
13,063
Michaël Eli Sander retweeted
I'm delighted to announce that I'll join @MITEECS as an Assistant Professor next Fall! I will be recruiting graduate students this cycle, please apply if you're interested in the science of deep learning! Despite the ambient doomerism, now is a great time to do research.
27
34
6
581
61,951
Michaël Eli Sander retweeted
We will be presenting this at NeurIPS in Paris in December! 🇫🇷 📉
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
2
11
1
104
8,769
Welcome Alex 🎉🎉🎉🎉
personal update: a few weeks ago I joined DeepMind as a student researcher in the paris office, hosted by the great @m_e_sander in the Frontier Gemma team :)
2
25
2,059
Michaël Eli Sander retweeted
The alpha version of my new book "Optimal Transport for Machine Learners" is out, with in particular an online version with interactive figures gpeyre.com/ot4ml/
9
102
10
499
39,936
Michaël Eli Sander retweeted
Background noises are there just waiting for their prompter. #JustPromptIt #Omni
I'm just as surprised nobody in AI voice tech has realized a voice needs background and environmental noise to sound realistic Even @ElevenLabs the leader in voice AI can not produce voice with background noise, or environment reverb sound AI voices are always going to sound non-passable as human if they don't have that And it's only me and this other guy even talking about it
3
3
32
6,842
Michaël Eli Sander retweeted
1/9 Excited to share TextSeal, our new state-of-the-art watermark for large language models at FAIR / Meta Superintelligence Labs (@AIatMeta) 🔐 Paper: arxiv.org/abs/2605.12456 Code: github.com/facebookresearch/…
1
11
2
35
4,416
Michaël Eli Sander retweeted
A Study on Hidden Layer Distillation for Large Language Model Pre-Training Maxime Guigon, Lucas Dixon, Michaël E. Sander arxiv.org/abs/2605.11513 [𝚌𝚜.𝙲𝙻 𝚌𝚜.𝙰𝙸]
1
1
121
Michaël Eli Sander retweeted
Just added to torch-fidelity - give it a go! github.com/toshas/torch-fide…
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
5
3
17
2,276
Michaël Eli Sander retweeted
Very cool work!
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
2
3
21
5,009
Michaël Eli Sander retweeted
MIND: Monge Inception Distance for Generative Models Evaluation Quentin Berthet, Yu-Han Wu, Clement Crepy, Romuald Elie, Klaus Greff, Michael Eli Sander arxiv.org/abs/2605.06797 [𝚌𝚜.𝙻𝙶]
1
2
170
Michaël Eli Sander retweeted
Beautiful idea to used sliced Wasserstein to bypass complexity of computing high dim OT distances instead of resorting to approximating the embeddings with Gaussians (which would give FID)!
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
1
10
61
5,943
Check-out our new preprint from GDM with Quentin Berthet, Yu-Han Wu, Clément Crepy, Romuald Elie, Klaus Greff and myself!
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
2
9
1,552
Michaël Eli Sander retweeted
🚨 New paper: Introducing MIND (Monge Inception Distance) Everyone agrees that FID is broken, requires too many samples, slowing down evals. MIND requires 10x fewer samples, is more robust, faster to compute. Our new drop-in replacement for evaluating generative models. 🧵👇
12
70
11
493
64,055
Clustering in Deep Stochastic Transformers Lev Fedorov, Michaël E. Sander, Romuald Elie, Pierre Marion, Mathieu Laurière arxiv.org/abs/2601.21942 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙶]
3
16
647
Michaël Eli Sander retweeted
Distillation is becoming a major paradigm for training LLMs but its success and failure modes remain quite mysterious. Our paper introduces the phenomenon of "teacher hacking" and studies how to mitigate it. arxiv.org/abs/2502.02671 More details in the thread below.
1/ If you’re familiar with RLHF, you likely heard of reward hacking —where over-optimizing the imperfect reward model leads to unintended behaviors. But what about teacher hacking in knowledge distillation: can the teacher be hacked, like rewards in RLHF?
12
54
6,408
Michaël Eli Sander retweeted
Really proud of these two companion papers by our team at GDM: 1) Joint Learning of Energy-based Models and their Partition Function arxiv.org/abs/2501.18528 2) Loss Functions and Operators Generated by f-Divergences arxiv.org/abs/2501.18537 A thread.
3
25
1
165
24,138