@robinomial

Assistant Professor @CSatUSC | Previously Visiting Researcher @facebookai | Stanford CS PhD @StanfordNLP

 Los Angeles, CA
robinjia.github.io/
Joined June 2018
  • Tweets388
  • Following961
  • Followers4.9K
2Media
The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem. A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:
Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.

ALT Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.

7
20
4
183
18,898
Robin Jia retweeted
Yoav Artzi@yoavartziOct 21
Some thoughts on a hard cap on the number of submissions a conference can handle, thereby capping the load on both reviewing and size. This currently is a partially specified mechanism, with the aim of soliciting suggestions. yoavartzi.com/log/peter-pan-…

Peter Pan Conferences: What If a Conference Refuses to Grow?

The explosion in ICLR submissions is a sign of things to come. This writeup proposes that conferences should consider a hard cap on the number of submissions they can handle, thereby capping the load...

yoavartzi.com
3
1
25
3,543
Super excited to share our new paper: Reward misspecification degrades chain-of-thought (CoT) monitorability! During RL training, when the model is prompted with one task intent (e.g., response ≤ 900 words) but rewarded for a different or even contradictory behavior (e.g., response > 900 words), the CoT keeps reasoning about the task intent while the output violates it. We also propose a simple mitigation method that requires no prior knowledge of the true reward or the reward hack behavior. 🧵
10
38
2
178
9,912
A very overdue life update: came across my graduation photos and realized I never shared them! I finished my PhD at @CSatUSC ! 🎓 So grateful to my advisors @robinomial and @_jessethomason_ for all their guidance and support, and to my partner for being there through all the ups and downs. ❤️ I’ve since joined @UCBerkeley as a postdoc, working with @serinachang5! 🐻
11
4
1
57
5,435
I’ll be at #COLM2026 next week, presenting our work on effective KV-cache eviction at the Efficient Reasoning Workshop on Oct 9, 12:30–13:30. I work on LLM capabilities, efficient reasoning, and evaluation, and would be happy to chat with anyone working on related problems! I’m also looking for Spring/Summer 2027 internships. Feel free to DM me if you think I'd be a good fit.
Deqing Fu
@DeqingFuJun 4
Introducing VaSE: Value-Aware Stochastic KV Cache Eviction. Reasoning models think in CoT, bloating the KV cache. Eviction caps memory but suffers capability drop. VaSE is a training-free recipe that cuts that cost: keep large-magnitude value states, evict stochastically.
2
4
2
24
3,470
Coming to COLM next week? The schedule is live at colm.eventhosts.cc/virtual/2… 5 invited talks, a panel discussion, 24 orals, and 856 posters await! This llama will need plenty of coffee to see all that...
Screenshot of schedule

ALT Screenshot of schedule

Cartoon of llama drinking coffee looking at the COLM schedule

ALT Cartoon of llama drinking coffee looking at the COLM schedule

1
8
58
5,980
Robin Jia retweeted
Deqing Fu
@DeqingFuSep 30
Introducing TabFM-Auto, my summer project at @GoogleResearch . LLMs possess rich world knowledge yet bad at handling massive tables with millions of number cells. Tabular Foundation Models are so good at tabular in-context learning but don't have semantic knowledges.
3
15
3
120
19,050
We have a few spots remaining for social events at COLM! If you're interested in hosting a social, please apply by 10am PT on Friday Oct 2 at forms.gle/qmozegbFu4DwHMzT9W…
4
16
4,580
Robin Jia retweeted
Deqing Fu
@DeqingFuSep 30
We now release the technical report for TabFM: arxiv.org/abs/2609.37959. Together, we introduce TabFM-Auto: an LLM agent working on feature engineering on top of a frozen TabFM. You can find more details at arxiv.org/abs/2609.37989 and explore features LLMs found at deqingfu.github.io/tabfm-aut….
Google Research
@GoogleResearchJun 30
Introducing TabFM, a foundation model designed specifically for tabular data classification & regression. This approach allows generation of high-quality predictions on previously unseen tables in a single forward pass. Learn more and try out the model →goo.gle/4eR7uku
5
36
2
253
31,935
Robin Jia retweeted
Deqing Fu
@DeqingFuSep 28
I’ll be at the Bay Area for @COLM_conf next week! I’m on the job market this year and I’ll be presenting 3 posters on: 1️⃣ How different models learn similar number representations. convergent-evolution.github.… 2️⃣ Token-level off-policy learning. arxiv.org/abs/2607.17524 3️⃣ SAEs for Reasoning. arxiv.org/abs/2506.09967

Convergent Evolution in Language Models

Fourier spikes in number embeddings are universal. Linear mod-T separability is not.

convergent-evolution.github.io
4
11
1
57
11,160
Robin Jia retweeted
Transluce
@TransluceAISep 24
Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets. In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵 Our blog: transluce.org/agent-activity NYT: nytimes.com/2026/09/23/techn…

Early rogue AI agent activity and attempts to hack found on urlquery.net

We found evidence on urlquery that AI agents were active earlier than previously reported and attempted hacks against public data providers.

transluce.org
119
557
206
2,496
824,986
Robin Jia retweeted
Transluce
@TransluceAISep 16
Frontier lab CEOs are calling for embedded 3rd party evaluators to help oversee AI risks. But what should third parties actually do within labs? We share some initial thoughts on how embedded evaluators could help avoid incidents like the Hugging Face hack and monitor for future risks 🧵 transluce.org/embedded-evalu…

Some Focus Areas for Embedded Evaluations and How to Approach Them

Initial thoughts on key risks third parties should monitor and a proposal for how to evaluate them.

transluce.org
9
28
4
136
10,677
New paper: We trained models on synthetic stories about humans only (no AIs).
 We found the Assistant adopts quirky behaviors from the stories in ordinary chat. Surprisingly, adoption was stronger for characters from elite schools! Why does this happen? 🧵
36
88
50
820
101,285
Robin Jia retweeted
Transluce
@TransluceAIAug 31
Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises. We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
23
83
26
445
79,921
Robin Jia retweeted
Niloofar
@niloofar_mireAug 17
Lately I have been receiving some emails of former undergrad and MS students I mentored 3-4 years ago thanking me for giving them advice to 'not do a phd', 'pre-agi' lol. To be honest, if you are debating a phd and you are not absolutely psyched about doing a phd, you probably shouldn't do it. And this isn't cause of AI takeover yada yada. Phd is a huge commitment that suits ppl with obsession to fully dedicate themselves to something. Same with post phd, if you are on the fence about academia, you probably shouldn't pick it as a career. This being said, I do think we definitely need ppl to do phds, cause we need to have a strong pipeline. if we stop training good researchers, we are doing a huge disservice to the generations that come in 10+ yrs.
Nathan Lambert
@natolambertAug 17
I tell a ton of people that getting a PhD is not a good idea for your career, but it’ll likely be far higher in personal growth and fulfillment which is maybe worth way more anyways. I need to think about how to price that in.
7
16
2
399
76,512
Hi all, surreal moment: the recommendations in our social media regulation paper have been adopted by EU law! The Digital Services Act previously required companies to report the "accuracy" of content moderation systems, and we suggested precision and recall instead.🧵[1/4]
7
4
1
52
7,922
Robin Jia retweeted
Deqing Fu
@DeqingFuJul 31
Your (token-level) reward model is also a (better) language model. Teach it to spot bad tokens. It stops writing them. Off-policy LoRA training is very interpretable — as conditional steering. Check out this amazing #COLM2026 work by @ZitongHuang12 and @CtrlVGustavo!
Zitong Huang
@ZitongHuang12Jul 31
Can training a language model for binary classification improve generation? Surprisingly, yes. We introduce Token-Level Off-Policy Labeling (TOPL) for faithful generation under distribution shift. Instead of optimizing next-token prediction, we train an LLM to classify whether each generated token is faithful. 🔗 arXiv: arxiv.org/abs/2607.17524 🧵👇
1
6
28
2,920
Check out our work on the interesting connections between LoRA, conditional steering, and reward modeling! See yall in sf #COLM2026 🔥🔥🔥
Zitong Huang
@ZitongHuang12Jul 31
Can training a language model for binary classification improve generation? Surprisingly, yes. We introduce Token-Level Off-Policy Labeling (TOPL) for faithful generation under distribution shift. Instead of optimizing next-token prediction, we train an LLM to classify whether each generated token is faithful. 🔗 arXiv: arxiv.org/abs/2607.17524 🧵👇
2
10
1,272
Can training a language model for binary classification improve generation? Surprisingly, yes. We introduce Token-Level Off-Policy Labeling (TOPL) for faithful generation under distribution shift. Instead of optimizing next-token prediction, we train an LLM to classify whether each generated token is faithful. 🔗 arXiv: arxiv.org/abs/2607.17524 🧵👇
5
29
3
207
16,730
Several people with no technical background told me they submitted to NeurIPS using agents for all coding—stating they didn't understand what the agent did, nor underlying code. Asked how they could trust the code/results, they said they assume reviewers will catch any errors.
41
31
30
582
183,919
Load more