CS PhD student at Stanford
Joined December 2022
- Tweets18
- Following179
- Followers90
- Likes25
Elise L. Han retweeted
Instead of trying to win the cat-and-mouse game of trying to build adversary-proof LLM watermarks, why can't we build watermarks that reduce a user's incentives to remove the watermark by guaranteeing their privacy?
Anthropic and OpenAI will watermark their responses.
Yet this violates user privacy and incentivizes users to paraphrase away the watermark. A lose for both sides!
Instead, we rethink the incentive structure, and propose to shift the AI provenance goal from individual tracking to estimating aggregate AI usage.
We develop locally differentially private watermarks which provably limit the evidence that any individual text can contribute to detection, while still enabling accurate estimation of population prevalence of watermarked texts.
Elise L. Han retweeted
So refreshing to see such a thoughtful and principled paper! Privacy is one of the most pressing problems in the era of AI. @EliseHan4 makes progress by fundamentally rethinking incentive structures and proposing sound statistical methodology, congrats!
Anthropic and OpenAI will watermark their responses.
Yet this violates user privacy and incentivizes users to paraphrase away the watermark. A lose for both sides!
Instead, we rethink the incentive structure, and propose to shift the AI provenance goal from individual tracking to estimating aggregate AI usage.
We develop locally differentially private watermarks which provably limit the evidence that any individual text can contribute to detection, while still enabling accurate estimation of population prevalence of watermarked texts.
Elise L. Han retweeted
If we want to track AI usage in the wild, we need to make sure users aren't incentivized to avoid detection. This is a very elegant solution to the problem!
Anthropic and OpenAI will watermark their responses.
Yet this violates user privacy and incentivizes users to paraphrase away the watermark. A lose for both sides!
Instead, we rethink the incentive structure, and propose to shift the AI provenance goal from individual tracking to estimating aggregate AI usage.
We develop locally differentially private watermarks which provably limit the evidence that any individual text can contribute to detection, while still enabling accurate estimation of population prevalence of watermarked texts.
3/3
Read our paper for more nuanced arguments and results!
paper: arxiv.org/pdf/2611.06100
code: github.com/eliselyhan/dp-wat…
Thank you to @tatsu_hashimoto for great mentorship and collaboration!!
2/3
On C4 language modeling tasks, our watermark accurately estimates the fraction of watermarked texts (lambda), at strong privacy levels (left to right: epsilon=1,2,3).
The y=x line is all you need!
Anthropic and OpenAI will watermark their responses.
Yet this violates user privacy and incentivizes users to paraphrase away the watermark. A lose for both sides!
Instead, we rethink the incentive structure, and propose to shift the AI provenance goal from individual tracking to estimating aggregate AI usage.
We develop locally differentially private watermarks which provably limit the evidence that any individual text can contribute to detection, while still enabling accurate estimation of population prevalence of watermarked texts.
A great method that extracts more signal about model capabilities at a low cost!
Can we forecast benchmark accuracy from models that score zero?
We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
Elise L. Han retweeted
Can we forecast benchmark accuracy from models that score zero?
We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
So cool!!
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Elise L. Han retweeted
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Elise L. Han retweeted
I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)!
The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵
Elise L. Han retweeted
Introducing the world's fastest tokenizer implementation, Gigatoken!
Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.
These baselines are already multithreaded Rust implementations! 🧵
Elise L. Han retweeted
Often in research level math, you can't verify a proof with Lean because the formal libraries needed don't exist yet.
Our fix: Pseudo-Formalization (PF), a proof verification method with the modularity and precision of formal proofs, but the flexibility of natural language.
Elise L. Han retweeted
Today, agents execute isolated tasks. Tomorrow, agents will steer complex decisions across long horizons.
Introducing CEO-Bench, a first step to measure "Steering Intelligence." In CEO-Bench, agents are asked to run a simulated startup for 500 days.
ceobench.com
Elise L. Han retweeted
How far can we compress the discrete tokens in an LLM's context into compact latent vectors?
With the right training recipe at large scale, our Latent Context Language Models (LCLMs) compress context up to 16× and land on a new Pareto frontier for long-context inference. 🧵(1/n)
Elise L. Han retweeted
Some new results I found surprising that I’m tweeting for Chris (who isnt on here). With enough compute, the best data filter for LMs (on DCLM) might be no filter. Why? Large models can tolerate a surprising amount of nominally 'low quality' data, and can sometimes even benefit.
Elise L. Han retweeted
Self-play led to superhuman Go performance, why hasn’t it for LLMs?
In practice, long run self-play plateaus like RL. We study why this happens, and build a self-play algorithm that scales better. It solves as many problems with a 7B model as the pass@4 of a model 100x bigger.
Elise L. Han retweeted
New paper!
Want to precisely optimize synthetic training data to do practical or even wacky things?
Dataset Policy Gradients get you there, letting you target any differentiable training or post-training metric. We embedded a QR code in GPT-2’s weights using only training data!
