Machine Learning Group @Cambridge_Uni
Cambridge, UK
Joined February 2011
- Tweets622
- Following147
- Followers6.6K
- Likes829
Cambridge MLG retweeted
Reading the Skaling paper I felt like I was sure I’ve seen that functional scaling law form before..
But yes, I can’t emphasise enough how much better your N,D->Loss fits will be if you use the coupling exponent.
Cambridge MLG retweeted
Glad to see folks coming (back!) around to the coupling variable :)
We find it useful for:
1) Continuous end-to-end speech diffusion at scale
2) Tri-modal discrete diffusion (MDM)
3) Distillation scaling laws.
arxiv.org/abs/2608.07222
Simple scaling law formulation that considers the interaction between N and D, and efficient experiment sampling strategy for parameter estimation.
Congratulations to @katie_m_collins on this amazing nature paper!
Games have long served as a testing ground for understanding problem-solving and strategic reasoning in humans and machines, across many fields – from AI, to cognitive science, economics, and more. Yet, in life people are regularly faced not just with repeated instances of the same problem or “system of rule and reward”, but entirely new problems, new systems, new games.
And when encountering new games, people not only need to decide what actions to take but whether a game is even worth playing at all!
How can people think reasonably about new games for the first time, so flexibly and quickly?
We take steps towards this question in our new work, out now in @Nature!
Cambridge MLG retweeted
Can we improve energy-based diffusion models by solving a classification task? 🤔
🚀 Excited to share DiffCLF, accepted at ICML 2026 🇰🇷! ! We predict a sample’s noise level, and recover densities as a by-product.
Joint work with @theh2o64 (co-lead) and @jmhernandez233. [🧵1/n]
Cambridge MLG retweeted
Something I see getting ignore over and over: if you take Marek et al. (2025) seriously, a ‘vanilla’ continuous-time story is not enough to explain Muon's advantage over Adam in LLM training.
You have to account for discretisation error to explain the improved convergence.
Cambridge MLG retweeted
Once both sides define what they mean by Shampoo and Muon before debating their relationship, there will be nothing left to debate.
One motivation for me to add these optimizers was seeing @_arohan_, a senior researcher whose contributions I respect, repeatedly claim that Muon is Shampoo.
IIUC, his argument is that if we disable accumulation in both Muon and Shampoo, then they become...
4/6
We’re willing to believe it could produce what passes for a paper over at Oxford.
Claude Code can now run an entire PhD-level research pipeline by itself.
it runs a 10-stage workflow from blank page to publication-ready PDF, replacing the work of a PhD advisor, three peer reviewers, and a copy editor in one repo.
→ Deep research with 13 agents (PRISMA + systematic review)
→ 12 agents write the paper section by section
→ 5-person peer review (Editor + 3 Reviewers + Devil's Advocate)
→ Integrity agent catches fabricated citations + stat errors
→ Final output: LaTeX → PDF, ready to submit
After the paper is finalized, it runs a Collaboration Quality Evaluation that scores YOU, across 6 dimensions, 1–100. Direction setting, intellectual contribution, quality gatekeeping.
It tells you exactly where you were the bottleneck.
Drop it into .claude/skills/ and the whole pipeline auto-loads. Works in Claude Code, Cowork, and as a Claude Project.
100% open source. CC-BY-NC 4.0.
Cambridge MLG retweeted
Hyperparameter tuning is critical for proper A:B comparisons. Too many times have I seen undertuned hypers yielding incorrect conclusions.
Tuning these hypers at scale is intractable (this is further compounded with scaling law research -- see thread about Marin below!). You need to rely on hyperparameter transfer.
If you missed out on Complete(d)P give it a read! xitter.cf/jramapuram/status/2008… -- width, depth, batch, duration and per module transfer!
Cambridge MLG retweeted
Awesome to see this work from Marin.
One detail I’m especially pleased about: their final recipe adapts ideas from our Complete(d)P work, especially horizon and batch-size hyperparameter transfer, and they verify it scales to very large scales.
Not only do we want to train a good model, we want to know it'll be good before we even start training.
About a month ago, the Marin team launched a 129B (16B active) 1e23 FLOPs MoE run and preregistered a loss of 2.252. The run finished this past week and landed at 2.234.
xitter.cf/percyliang/status/2044…
Cambridge MLG retweeted
Super excited to release our paper from a collaboration between @Angstrom_ai and @AstraZeneca evaluating our new model CSP-MACE-Å, a machine learning interatomic potential intended to replace DFT in crystal structure prediction (CSP)! arxiv.org/abs/2605.28905 🧵1/5
Cambridge MLG retweeted
I'm absolutely thrilled to be joining @BernieSanders @tegmark @yi_zeng and Xue Lan to discuss the existential threat posed by AI and the need for international cooperation!
Next Wednesday, April 29, 7pm (doors 6:15pm) at the U.S. Capitol.
RSVP here: forms.office.com/Pages/Respo…
Senator @BernieSanders has invited me and three other AI researchers to a public panel on AI existential risk & international cooperation at the U.S. Capitol 7pm Wednesday April 29th. RSVP here to join us for this important conversation: forms.office.com/Pages/Respo…
Cambridge MLG retweeted
It’s such a beautifully diverse, yet cohesive book. There is even a section with hot-takes on causality that still feels pertinent decades later
A really excellent book.
A few people independently told me this was one of their favorite books over a decade ago. I bought it, and it became one of the textbooks on my shelf I revisit from time to time to spark the joy of holding ideas to a different light.
It brings to life the elegance of information theory.
A good day to recognize 10 years from the passing of David McKay.
Cambridge MLG retweeted
✨New AI Safety work on Steganography and LLM monitoring✨
We propose ‘steganographic gap’: the first principled metric for detecting and quantifying encoded reasoning in LLMs, which can reveal hard-to-detect forms of steganography, e.g., paraphrasing-resistant steganography.
Cambridge MLG retweeted
Discrete diffusion models are such a natural fit for multimodal data.
This project is an extensive ablation of the key aspects of training and inference with multimodal diffusion models.
In a way, it’s a compendium on how to make them work at scale.
Autoregressive models dominate, but what if we treat multimodal generation as discrete order agnostic iterative refinement? Excited to share our systematic study on the design space of Tri-Modal Masked Diffusion Models (MDMs). We pre-trained the first Tri-Modal MDM from scratch on (text,), (image, text), and (audio, text). The same model can do ASR, TTS, T2I, captioning and native text generation.
What I'm the most proud of in this work is the scientific rigor. Over 3,500 training runs. Principled hyperparameter transfer. Honest results. Carefully controlled ablations across multiple different axis of entanglement.
A thread on our empirical findings (arXiV: arxiv.org/abs/2602.21472)
Cambridge MLG retweeted
1/14 Is Muon “better” than Shampoo?
We argue that their relationship parallels Adam's relationship with Signum. Analogous to @lukas_balles and Hennig’s (2018) decomposition of Adam into element-wise scaled Signum, we can decompose Shampoo as left- and right-adapted Muon.
Cambridge MLG retweeted
Hyperparameter transfer from small -> large scale is crucial to train modern large models. In our latest work from Apple ML research, we show that **per-module** scaling terms coupled with our extension of CompleteP -> Complete(d)P can reduce training time by 1.32x at 7B scale!
In our new work—Complete(d)P—we tackle 3 questions about hyperparameter (HP) scaling:
• How to transfer across model size, tokens&batch-size?→Complete(d)P
• Do per-module HPs matter?✔️2x speed-ups possible
• Do they transfer to larger scale?✔️ With the right parameterisation
ALT Abstract of a paper titled “Completed Hyperparameter Transfer across Modules, Width, Depth, Batch & Duration”. The paper describes a principled method for scaling up training by using a hyperparameter-transfer enabling parameterisation. The paper is relevant to people working on µP, scaling, reparameterisation rules, optimisation and pre-training.
Cambridge MLG retweeted
In our new work—Complete(d)P—we tackle 3 questions about hyperparameter (HP) scaling:
• How to transfer across model size, tokens&batch-size?→Complete(d)P
• Do per-module HPs matter?✔️2x speed-ups possible
• Do they transfer to larger scale?✔️ With the right parameterisation
ALT Abstract of a paper titled “Completed Hyperparameter Transfer across Modules, Width, Depth, Batch & Duration”. The paper describes a principled method for scaling up training by using a hyperparameter-transfer enabling parameterisation. The paper is relevant to people working on µP, scaling, reparameterisation rules, optimisation and pre-training.
Cambridge MLG retweeted
Probabilistic Modelling is Sufficient for Causal Inference
Bruno Mlodozeniec, David Krueger, Richard E. Turner
(arXiv:2512.23408) arxiv.org/abs/2512.23408