@smsampark

Researcher at GDM. Previously: Stanford Postdoc, MIT PhD, Cornell BS

 San Francisco, CA
sungminpark.com
Joined July 2013
  • Tweets65
  • Following782
  • Followers523
Working on data attribution, data provenance, citations, or their downstream applications? Please consider submitring to ATTRIB@NeurIPS! We have an exciting lineup of speakers and our prev iterations were very successful (in fact, two of the organizers were past participants!)
Elisa Nguyen@_elinguyenAug 30
📢 Deadline extension for ATTRIB @NeurIPSConf Sydney! We've pushed the submission deadline to **Sep 5 AoE**. Working on data attribution, data provenance, citations, or their downstream applications? We would love to see your work. 🔗 Submission link below.
3
11
1,882
The ATTRIB workshop is back at @NeurIPSConf 2026 (Sydney) in its 3rd edition! This year's focus: data attribution and provenance. -> Submissions open Aug 1, Deadline Sep 1 (AOE). More info: attrib-workshop.cc/
ATTRIB 2026 @ NeurIPS 3rd Workshop on Attributing Model Behavior at Scale: Data Attribution and Provenance Sydney, Australia Contact info: attrib-neurips26 [at] googlegroups [dot] com

ALT ATTRIB 2026 @ NeurIPS 3rd Workshop on Attributing Model Behavior at Scale: Data Attribution and Provenance Sydney, Australia Contact info: attrib-neurips26 [at] googlegroups [dot] com

2
3
1
12
1,894
Self-play led to superhuman Go performance, why hasn’t it for LLMs? In practice, long run self-play plateaus like RL. We study why this happens, and build a self-play algorithm that scales better. It solves as many problems with a 7B model as the pass@4 of a model 100x bigger.
30
152
19
1,014
156,539
Excited about this new work led by @TristanThrush! Obvious implications for targeted synthetic data generation and data poisoning (check out the original thread for very cool results), but wanted to highlight a few points (1/7)
Tristan Thrush@TristanThrushApr 10
New paper! Want to precisely optimize synthetic training data to do practical or even wacky things? Dataset Policy Gradients get you there, letting you target any differentiable training or post-training metric. We embedded a QR code in GPT-2’s weights using only training data!
1
10
53
8,681
more replies
More broadly, would love to better understand the interplay b/w inner optimization dynamics and outer meta-/policy optimization to make these methods more robust. Very curious about surprising links here, e.g. b/w recent results on central flows and metasmoothness. (6/7)
1
1
194
Also, big thanks to @andrew_ilyas, @logan_engstrom, Ben Chen et al. for sharing metagrads implementation and a bunch of insights that helped some of our early investigations as well as inspiring this line of work! (7/7)
4
177
One of the coolest scaling analyses I've seen in a while!
Suhas Kotha@kothasuhas19 Sep 2025
Since compute grows faster than the web, we think the future of pre-training lies in the algorithms that will best leverage ♾ compute We find simple recipes that improve the asymptote of compute scaling laws to be 5x data efficient, offering better perf w/ sufficient compute
1
11
995
(1/n) Check out our new paper: "Fantastic Pretraining Optimizers and Where to Find Them"! >4000 models to find the fastest optimizer! 2× speedups over AdamW? Unlikely. Beware under-tuned baseline or limited scale! E.g. Muon: ~40% speedups <0.5B & only 10% at 1.2B (8× Chinchilla)!
13
96
19
443
185,284
Cool application of training data attribution / influence estimation (using TRAK) to data selection for robot imitation learning!
Christopher Agia@agiachris25 Jun 2025
What makes data “good” for robot learning? We argue: it’s the data that drives closed-loop policy success! Introducing CUPID 💘, a method that curates demonstrations not by "quality" or appearance, but by how they influence policy behavior, using influence functions. (1/6)
1
15
682
At #ICLR, check out Perplexity Correlations: a statistical framework to select the best pretraining data with no LLM training! I can’t make the trip, but @tatsu_hashimoto will present the poster for us! Continue reading for the latest empirical validations of PPL Correlations:
1
15
1
53
12,173
To stress-test our technique, we apply Synthetic Continued Pretraining on 1K ICLR papers (joint effort with @ChengleiSi), obtaining an LM that “knows” your ICLR submission in its weights. If you’d like to try out our LM, come to the oral or poster session below! Oral Session 4E: Peridot 202-203, 3:30PM Poster Session: Hall 3 + Hall 2B #194
3
12
835
Sam Park retweeted
Ken Liu
@kenziyuliu3 Apr 2025
An LLM generates an article verbatim—did it “train on” the article? It’s complicated: under n-gram definitions of train-set inclusion, LLMs can complete “unseen” texts—both after data deletion and adding “gibberish” data. Our results impact unlearning, MIAs & data transparency🧵
An illustration of one of our findings: LLMs may verbatim generate an unseen text T, after training on texts D with zero n-gram overlap with T.

ALT An illustration of one of our findings: LLMs may verbatim generate an unseen text T, after training on texts D with zero n-gram overlap with T.

13
92
9
321
92,648
Machine unlearning ("removing" training data from a trained ML model) is a hard, important problem. Datamodel Matching (DMM): a new unlearning paradigm with strong empirical performance! w/ @kris_georgiev1 @RoyRinberg @smsampark @shivamg_13 @aleks_madry @SethInternet (1/4)
2
22
2
136
28,063
Have a paper on understanding how ML design choices relate to model behavior? A reminder that the (first) ATTRIB deadline is this Wednesday AoE! The second deadline is October 4th AoE, but one author will have to volunteer as an emergency reviewer.
Logan Engstrom@logan_engstrom13 Sep 2024
Announcing a deadline extension for the ATTRIB workshop! Submissions are now due September 25th, with an option to submit October 4th if at least one paper author volunteers to be an emergency reviewer. More info here: attrib-workshop.cc
1
4
12
4,120
The ATTRIB workshop is back @ NeurIPS 2024! We welcome papers connecting model behavior to data, algorithms, parameters, scale, or anything else. Submit by Sep 18! More info: attrib-workshop.cc Co-organizers: @tolgab0 @logan_engstrom @SadhikaMalladi @_elinguyen @smsampark
15
2
49
8,275
Thanks to all who attended our tutorial "Data Attribution at Scale" at ICML (w/ @smsampark @logan_engstrom @kris_georgiev1 @aleks_madry)! We're really excited to see the response to this emerging topic. Slides, notes, ICML video: ml-data-tutorial.org/ Public recording soon!
2
28
152
13,423
Load more