Research scientist at Google DeepMind. Opinions are my own.
Joined February 2014
- Tweets68
- Following545
- Followers1.1K
- Likes366
Pinned Tweet
Are you interested in training large models in JAX but are set back by the complicated partition specs and sharding configurations required to scale up? I've recently created scalax, a small library to help developers easily scale up JAX models. github.com/young-geng/scalax
Xinyang (Young) Geng retweeted
Today we're putting out an update to the JAX TPU book, this time on GPUs. How do GPUs work, especially compared to TPUs? How are they networked? And how does this affect LLM training? 1/n
Xinyang (Young) Geng retweeted
2.5 Flash is out!
You can now specify thinking budgets, or disable thinking entirely for lower latency.
Strong code & reasoning capabilities, cost effective, fast.
It's a great workhorse model for enterprise and developers, excited to hear your feedback.
Gemini 2.5 Flash is described as being optimized for speed and scalability. Despite its lighter design, the community voted for it's impressive performance on Hard Prompts, Coding, and Long Queries. Matching the strength of its older sibling, Gemini 2.5 Pro at #1 in these categories
Xinyang (Young) Geng retweeted
The team has built such a smart model. The gap becomes bigger on harder problems!
Xinyang (Young) Geng retweeted
Today we are launching 2.5 Pro!
I think it's the best model in the world. State-of-the-art reasoning and great vibes (+39 ELO gap on lmsys!)
2.5 Pro improves in coding, stem, multimodal, instruction following, and lots more.
Available in AI Studio & the Gemini App!
Xinyang (Young) Geng retweeted
Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n
Xinyang (Young) Geng retweeted
Despite many complaints about Jax being hard to use, it has a crucial advantage over PyTorch: for distributed jobs, XLA is sufficiently good at auto-scheduling parallelism strategies, e.g., sharding, overlapping compute and comms. If PyTorch becomes good at that, it's checkmate.
Xinyang (Young) Geng retweeted
For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task, which favors a large community of collaborators.
Xinyang (Young) Geng retweeted
Whether you like it or not, the future of AI will not be canned genies controlled by a "safety panel". The future of AI is democratization. Every internet rando will run not just o1, but o8, o9 on their toaster laptop. It's the tide of history that we should surf on, not swim against. Might as well start preparing now.
DeepSeek just topped Chatbot Arena, my go-to vibe checker in the wild, and two other independent benchmarks that couldn't be hacked in advance (Artificial-Analysis, HLE).
Last year, there were serious discussions about limiting OSS models by some compute threshold. Turns out it was nothing but our Silicon Valley hubris. It's a humbling wake-up call to us all that open science has no boundary. We need to embrace it, one way or another.
Many tech folks are panicking about how much DeepSeek is able to show with so little compute budget. I see it differently - with a huge smile on my face. Why are we not happy to see *improvements* in the scaling law? DeepSeek is unequivocal proof that one can produce unit intelligence gain at 10x less cost, which means we shall get 10x more powerful AI with the compute we have today and are building tomorrow. Simple math! The AI timeline just got compressed.
Here's my 2025 New Year resolution for the community:
No more AGI/ASI urban myth spreading.
No more fearmongering.
Put our heads down and grind on code.
Open source, as much as you can.
Acceleration is the only way forward.
Xinyang (Young) Geng retweeted
It’s done because it’s much easier to 1) collect, 2) evaluate, and 3) beat and make progress on. We’re going to see every task that is served neatly packaged on a platter like this improved (including those that need PhD-grade expertise). But jobs (even intern-level) that need long, multimodal, coherent, error-correcting sequences of tasks glued together for problem solving will take longer. They are unintuitively hard, in a Moravec’s Paradox sense.
Fwiw I’m ok and happy to see harder “task” evals. Calling it humanity’s last exam is a bit much, and misleading.
Unpopular opinion: benchmarks like these are moving the field in the wrong direction
No I don't want an AI to be able to memorize (useless?) questions like "How many paired tendons are supported by a sesamoid bone?" in its weights
I want the "intern", as @karpathy is suggesting
Xinyang (Young) Geng retweeted
Appreciate @aidan_mclau looking into the thinking model results. Originally scores looked weak as the response was plucked from the thought content versus output.
We are looking into ways of making thinking output less confusing for people running evals.
This is why we 🚢, to collect feedback and iterate!
Xinyang (Young) Geng retweeted
People completely misunderstand the data wall.
It's the data slop wall. Most of the data is so bad it's a waste of a good gpu to backprop it.
Xinyang (Young) Geng retweeted
Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task?
We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵
Xinyang (Young) Geng retweeted
People learning JAX, feel free to reach out if the learning feels too steep, hopefully we can flatten it out.
Also, checkout the JAX LLM for help from the community: discord.gg/m9NDrmENe2
This has been and will continue to be my recommendation for anyone in this position. Learn jax and sign up for sites.research.google/trc/ab…
Its one of the best things Google has ever done. You can do meaningful research for free, but the learning curve is steep. strap in
Xinyang (Young) Geng retweeted
We finally have an official `nvidia-smi` for TPU 🎉
Simply install it with `pip install tpu-info`
Xinyang (Young) Geng retweeted
Good post-training data is precious and scarce; compute is less so.
We should focus on methods which squeeze more out of existing data by spending additional compute per datapoint, rather than optimizing for cheaper post-training methods
Xinyang (Young) Geng retweeted
1/5. Excited to share a spicy paper, "Rethinking conventional wisdom in machine learning: from generalization to scaling", arxiv.org/pdf/2409.15156.
You might love it or dislike it!
NotebookLM: notebooklm.google.com/notebo…
While double-descent (generalization-centric, data-efficiency) is cool, "skydiving” 🪂 (scaling-centric, compute efficiency) is adventurous!
The thinking is different.
Generalization-centric: reducing generalization gap using regularization.
Scaling-centric: reducing approximation error via scale -- as generalization gap has yet to emerge.
Two central principles of the generalization paradigm may not be applicable:
1) regularization as the primary guiding principle;
2) model comparison via a validation set.
