Satwik Bhattamishra (@satwik1729) header
Satwik Bhattamishra (@satwik1729) profile picture

Researcher at Cartesia AI | CS PhD, Oxford | Previously at Google, Cohere, and Microsoft Research

Oxford, England
Joined December 2019
Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)Photo by Satwik Bhattamishra (@satwik1729)
Given black-box access to a Transformer's output, can we efficiently recover its parameters? We analyse the learnability of attention-based models with query access in our new work. Accepted at #ICML2026 🎉 Work done with @shahkulin98, @mhahn29 and Varun Kanade. 🧵
9
23
5
169
29,030
Satwik Bhattamishra retweeted
A student takes notes for an open book exam and shares them with a friend. They both use the notes during the exam, yet only the one who made it aces Similarly, for language models, neural and text memory stack
In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.
3
4
2
62
11,774
Satwik Bhattamishra retweeted
Introducing Avartha. We're building the infrastructure to bring frontier-grade intelligence to real-time applications. Our first release, Ava Zero, is a voice agent entirely running open models using our inference infrastructure and ranks #1 on Sierra's tau-voice benchmark with highest task completion rate and generation quality and just costs $2.5 per hour. Open models are now at the frontier of voice AI.
19
25
15
168
293,616
Excited to share our work on hierarchical models with dynamic tokenisation! ✨ All digital data can be represented as bytes. H-Nets are a step towards building end-to-end models that can operate across modalities and domains without handcrafted tokenisers. Check out the blog for more details!
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
6
7
73
2,894
Satwik Bhattamishra retweeted
Super happy to share our paper on predicting downstream capabilities of LLMs has been accepted to #NeurIPS2026! 🥳
Excited to share our new paper! “Forecasting Downstream Performance of LLMs With Proxy Metrics” w/ my amazing advisors @sivareddyg, @mariusmosbach, @DBahdanau Cross-entropy loss is a poor predictor of how models perform on downstream tasks (esp. reasoning). We propose something better: proxy metrics computed over expert reasoning traces. 🧵 Thread below 👇
4
7
45
3,360
Satwik Bhattamishra retweeted
I sometimes see people trying to use complexity theory to argue that building "true" AI is impossible. I find this unreasonably annoying. It requires ignoring what is in front of your face and it ignores that worst-case complexity has been an awful guide in machine learning.
8
14
3
135
12,798
Satwik Bhattamishra retweeted
🚀🚨We're hiring at Microsoft Research India! We're looking for Researchers, Senior Researchers, and Senior Applied Researchers to join our team in Bengaluru. 📧 Send your CV to [email protected] #Hiring #MicrosoftResearch #InformationRetrieval
10
25
3
338
52,779
Satwik Bhattamishra retweeted
Today, I had the pleasure to pledge my loyalty to my new boss @HandsomeDanYale. I joined Yale CS & @WuTsaiYale last month, starting up a new lab advancing fundamental research on intelligence. Open positions will be announced in the next few days!
16
8
52
2,381
Satwik Bhattamishra retweeted
Introducing Sonic-3.6: our most lifelike TTS yet, and another step change in naturalness across 44 languages. Just three months after our Sonic-3.5 launch, we've made fundamental model improvements based on feedback from the teams building on Sonic. We're #1 on @ArtificialAnlys across both the provider and controlled voice streaming leaderboards — best-in-class voices + the best underlying model. Available in beta today. Hear it for yourself 👇
39
72
26
661
90,962
Satwik Bhattamishra retweeted
Excited about this solution to a long-standing puzzle in Transformer theory. Via algebra+category theory, we achieve a highly predictive and detailed account of transformers’ state-tracking abilities.
Which state-tracking problems can transformers length generalize on? Transformers' performance on different finite-state tasks varies drastically, even when the tasks are structurally similar. Now, for the first time, we can precisely predict when length generalization occurs.
2
40
490
36,802
Satwik Bhattamishra retweeted
Which state-tracking problems can transformers length generalize on? Transformers' performance on different finite-state tasks varies drastically, even when the tasks are structurally similar. Now, for the first time, we can precisely predict when length generalization occurs.
7
68
8
473
64,208
Satwik Bhattamishra retweeted
The Myth of “Love Learning” People often ask me how to get better at chess. My answer is almost the opposite of what people expect. You don’t have to love learning. In fact, if you wait until you love the process, you’ll probably never become very good. We romanticize improvement. We imagine great players waking up excited to study endgames, analyze losses, or memorize opening lines. Sometimes that’s true. Most of the time it isn’t. Improvement is often boring. The difference between an amateur and a professional isn’t that the professional enjoys every minute. It’s that they keep going when they don’t. People say children are fearless learners. I’m not so sure. Children quit things constantly. Piano. Swimming. Languages. Football. Chess. They usually continue only because someone else insists they do. Parents. Teachers. Coaches. Discipline often comes before passion, not after. The same is true for adults. We tell people to “follow your curiosity.” That’s wonderful advice if curiosity happens to last. Usually it doesn’t. Every meaningful skill has a point where curiosity runs out and routine takes over. That’s where improvement actually begins. Chess certainly did not always feel like play to me. There were tournaments where the last thing I wanted to do after six hours of defending a miserable endgame was analyze another five hours. There were openings I studied not because they fascinated me, but because my opponents forced me to. There were positions I analyzed simply because they were objectively important. Not because they were fun. Because they needed to be done. People often criticize schools for asking the wrong questions. But there’s another side to that story. If everyone only studied the questions they found interesting, most people would develop huge blind spots. Sometimes someone else knows what you need to learn before you do. Nobody is naturally curious about tax law before becoming an accountant. Or anatomy before becoming a surgeon. Or rook endings before losing enough of them. External structure isn’t always the enemy of learning. Often it’s the bridge that gets you to the point where genuine curiosity develops. The biggest obstacle isn’t fear of looking stupid. It’s our addiction to doing only what feels rewarding today. Modern life gives us endless opportunities to switch the moment something becomes difficult. A new opening. A new productivity system. A new app. A new hobby. Very few people simply keep doing the same useful thing for years. That’s the superpower. So when people ask how to improve at chess, I don’t tell them to fall in love with learning. Love helps. Curiosity helps. Being willing to fail helps. But none of those are reliable. Build habits that survive the days when none of those feelings are there. Because mastery isn’t built on motivation. It’s built on showing up after motivation has left the room.
104
367
77
2,586
250,704
Satwik Bhattamishra retweeted
🇮🇳GOLDEN SWEEP FOR INDIA 🏆 All 5 Indian students win GOLD at the 56 International Physics Olympiad (IPhO) 2026 in Bucaramanga, Colombia -placing India at Rank #1 in the world (jointly with China, Kazakhstan, Russia, South Korea & Taiwan) among 381 students from 87 countries 1/5
54
584
106
1,922
384,021
Satwik Bhattamishra retweeted
How long do training strings need to be in order for transformers to generalize at test-time to inputs of arbitrary length? We answer this question in two different numerical precision regimes, using connections with Diophantine equations and linear temporal logic!
1
6
34
2,379
Satwik Bhattamishra retweeted
Many algorithms (e.g., Turing machine simulations) have been written as transformers using average hard attention. Do these algorithms work with softmax attention? Please visit @pentagonalize, Dana Angluin, and my poster. Sunday 2pm Harbor H-I! aclanthology.org/2026.tacl-1…
2
7
24
2,277
Satwik Bhattamishra retweeted
HorizonMath has won the AI4Math@ICML Best Paper Award and is now part of the ByteDance Seed 2.1 model card! Stay tuned for an updated version with more model evals (Fable 5/CC) and question categories. If you want to quickly measure capabilities on unsolved questions, HorizonMath is the benchmark to run.
Excited to give a talk on HorizonMath next week at ICML in Seoul! Swing by Hall D1 at 10:15am on Sat 7/11 or otherwise reach out to chat if interested in mathematical discovery with unsolved problems! And s/o to the ByteDance team for reporting their results with Seed 2.1 😎
8
6
1
48
6,083
Hey 👋 I wrote a tutorial on computational hardness in query learning, about a question I found interesting: when can exact learning with queries be hard for computational reasons rather than information-theoretic ones? Please check it out if interested: satwikb.com/writings/comp-ha…
1
4
17
1,004
Among many other things, Servedio and Gortler (2004) [1] showed a class where the bottleneck for query learning is computation. In the tutorial, we discuss a simpler and more instructive example of a class that can be exactly learned with a small number of queries by an exponential-time algorithm, but not by any algorithm that runs in polynomial time. [1] cs.columbia.edu/~rocco/Publi…
1
72
A membership-query learning algorithm uses computation to decide which inputs to query and how to process the labels. The tutorial shows how some classes can be hard to learn exactly because a learner might need a lot of computation, even if a small number of queries suffice. Check out the write-up to learn more!
68