ML Professor @ CMU 🦋 gingsmith
Joined December 2011
- Tweets63
- Following30
- Followers748
- Likes16
Virginia Smith retweeted
New paper: Codetta
Can AI agents secretly coordinate through messages that look completely normal?
We show they can, at high capacity and without assumptions on the agents’ models, context, or memories, while the transcript remains rigorously (computationally) indistinguishable from ordinary agent outputs. 🧵
Virginia Smith retweeted
Over 400 academics in 30+ countries signed our open letter calling for immediate changes in AI development and deployment practices to ensure that cybersecurity incidents do not continue to occur, as future incidents will be far more damaging to our critical infrastructure.
Virginia Smith retweeted
CollusionBench detects agentic collusion by probing agents’ internal representations, not just their outputs. Its feature dictionary can flag collusive reasoning even when visible behavior looks benign. From CMU’s @gingsmith and @AashiqMuhamed. One of 25+ posters at Frontier Data Summit.
frontierdatasummit.ai/
Virginia Smith retweeted
Today, we’re expanding @SnorkelAI’s Open Benchmarks Grants by 10x to a $30M commitment:
- Funding a more diverse, robust, and continuously-updated ecosystem of open benchmarks
- Launching the Open Benchmarks Red Team to continually test and strengthen them
- Introducing the Snorkel Research Fellowship to support independent contributors developing new evaluation methods
Since our launch earlier this year, we’ve been humbled to partner with the teams behind Terminal-Bench, TB-Science, ARC-AGI-3, OSWorld 2.0, Agents’ Last Exam, Continual Learning Bench, Senior SWE-Bench, SlopCodeBench, and more. Alongside these teams, we’ve helped design evaluation methodologies, develop task construction pipelines, and scale quality control. OBG-funded benchmarks have appeared on the latest model cards from every major frontier lab; have helped to index and measure frontier progress and alignment; and have contributed to guiding the frontier of AI development.
Benchmarks have always been guideposts for—and drivers of— AI progress: setting a metric and then “hillclimbing” against it is the core of how AI works! But when benchmarks fall behind frontier capabilities, our ability to evaluate and align these systems does as well. They become too simple, static, or correlated, and the risk of “benchmaxxing” increases.
We need more diverse, robust, and continuously-updated benchmarks, developed by a broader ecosystem of researchers and domain leaders.
We’re incredibly excited to accelerate frontier evaluation with Open Benchmarks Grants— raising the ambition and investment in robust, open, and independent measurement. Read our full announcement and apply here: benchmarks.snorkel.ai/fronti…
Virginia Smith retweeted
Model abliteration is a simple technique that can remove refusal behavior from open-weight models.
Are there any simple techniques that can defend against it?
Introducing our paper, Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration.
Several existing defenses require safety finetuning for each new checkpoint. DDO can be applied post-hoc, with the base model frozen during decoy optimization.
On Llama-3-8B, mean attack success under standard abliteration drops from 85.1% to 1.8%, competitive with finetuned defenses.
Virginia Smith retweeted
Poison selection matters even when the poison count stays fixed.
Introducing our @AnthropicAI Fellows project: Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks ☠️
Backdoor poisoning adds malicious examples to a model's finetuning data. The model learns to behave differently when a trigger appears.
Existing evaluations focus on how many examples an attacker can poison. We look at which examples they should choose.
Attack success ranges from 3% to 80% depending on the selected poison set.
Virginia Smith retweeted
Excited to share our new paper:
DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment accepted to EMNLP 2026 Main Conference.
Can we align LLMs to human preferences without updating a single base-model weight?
We show that sparse autoencoders can make activation steering prompt-conditional. 🧵
📄 Paper: arxiv.org/abs/2603.21461
💻 Code: github.com/jtbwedgwood/dspa
Virginia Smith retweeted
Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises.
We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
🌟Reminder that submissions to the #NeurIPS2026 Workshop on Child Safety in AI are due on Aug 29!
Excited to announce the NeurIPS 2026 Workshop on Child Safety in AI!
As AI becomes an increasingly common part of children's lives, it also presents new risks related to child development, mental health, privacy, and the misuse of synthetic content. Addressing these issues requires advances in the development, deployment, maintenance, and governance of safe AI systems.
Virginia Smith retweeted
I'm giving a talk today on my latest work on 'Decomposition Attacks for Agents' at the AI Safety Night organized by @AIM_Intel !
(Details in the next tweet)
Virginia Smith retweeted
In the age of social media, the most vulnerable are the children and often because of none of their fault. If your work can be used for Child Safety in AI, consider submitting your work to this workshop! CSAM is a bigger problem now than 7 years ago!
Excited to announce the NeurIPS 2026 Workshop on Child Safety in AI!
As AI becomes an increasingly common part of children's lives, it also presents new risks related to child development, mental health, privacy, and the misuse of synthetic content. Addressing these issues requires advances in the development, deployment, maintenance, and governance of safe AI systems.
Excited to announce the NeurIPS 2026 Workshop on Child Safety in AI!
As AI becomes an increasingly common part of children's lives, it also presents new risks related to child development, mental health, privacy, and the misuse of synthetic content. Addressing these issues requires advances in the development, deployment, maintenance, and governance of safe AI systems.
This workshop will bring together researchers, practitioners, and policymakers to consider the technical and sociotechnical challenges of protecting children across the AI ecosystem. We welcome submissions from researchers working on child safety across AI, security, privacy, HCI, and related fields. Submissions due August 29.
Learn more: childsafety-ai.github.io/
#NeurIPS2026 #AISafety #ChildSafety
Virginia Smith retweeted
Huge thanks to Pratiksha for writing a thread about our works here!! Highly encourage you to check it out.
For details on tomorrow’s presentations:
10:30-12:15 - Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safety (Spotlight) - Hall A #3013
Co-led this work with Rebecca Portnoff from @thorn, and will be presenting with her colleague and our coauthor Michael Simpson! We outline 15 open problems in AI safety that remain critically unaddressed by current work, and are research barriers to preventing AIG-CSAM. These range from concept fusion and data cleaning to unlearning and robust watermarking. Hope to see you there and spark some great discussions, about what can be done to attack this problem as ML researchers. Michael is highly knowledgeable about the policy and governance landscape surrounding this problem, so encourage anyone interested to come ask those questions too! (Link to all 15 papers in thread).
2:30 - 4:15 - Membership Inference Attacks for Unseen Classes - Hall A #4004
Co-led this work with @prthaker_ who was the driving force behind the initial idea. We’ve been working on this paper almost since I first joined CMU in 2024, and I’m really happy to finally present it here on behalf of our small team! In addition to advancing our understanding of MIAs (and showing we could audit models for CSAM in training data without accessing CSAM), this paper was one of my first ML research experiences and taught me so much about producing high -quality research, putting theory first, and how to persevere on a seemingly impossible research problem - detecting whether a model was trained on data without any in-distribution samples! Come check out the poster, super stoked to share more.
Huge thanks to my research advisor @gingsmith whose guidance was so insightful on both papers, and all coauthors involved - @prthaker_, @zstevenwu, @chhaviyadav_, Kevin Kuo - and Rob Wang, Michael Simpson, and Rebecca Portnoff from Thorn! These works were truly cross-domain team efforts.
See you all tomorrow!
Virginia Smith retweeted
🌟 I'm not at ICML this year, but thrilled about some of our work that will be presented there! 🧵
Virginia Smith retweeted
1/ 🌳 We are releasing Pando, a benchmark to climb for interpretability methods. We created 720+ model organisms with known ground-truth decision rules planted inside, then asked: do white-box tools actually beat just asking the model?
ALT Performance of various methods in the no verbalization regime
Virginia Smith retweeted
Next week at the Simons Institute, a workshop on Agency in Collaborative Learning. Join us!
simons.berkeley.edu/workshop…
Virginia Smith retweeted
Excited to give a talk at @SimonsInstitute Trust in Decentralized Systems Workshop on Tuesday at 11am!
Title: "2026 Is the New 2016" — on federated memory, contextual privacy, and personalized agents.
The privacy conversation has moved way past training data memorization. With persistent memory, tool use, and agents acting on your behalf (#clawdbot👀), the real risk is what models do with the data you feed them at inference time. Context window is the new attack surface!
I'll talk about our new benchmark CIMemories, where we test whether models can actually make context-dependent decisions about what to share with whom from memory. Turns out they really can't, up to 69% violation rates, and it only gets worse the more you use them.
Link to slides🔻🔻🔻
Virginia Smith retweeted
[1/n] Open Problems in AI Child Safety
AI is misused to generate CSAM at alarming scale. 400% increase in AI-generated CSAM since 2024 (IWF). 1 in 17 teens are victimized by deepfake nudes.
We outline 15 open problems where AI safety research can help.
🔗aichildsafety.github.io
🎅 A good reminder about LLM biases and a fun holiday project led by @neilkale, @prthaker_ & James Wedgwood
We asked LLMs: Is Santa real? 🎅
GPT-4o says Yes at any age. Claude tells 5-year-olds the truth.
What does this reveal about invisible assumptions in AI? Do LLMs believe in the tooth fairy or the Illuminati?
New holiday post from @mlcmublog 🔗⬇️
