phd-ing @MIT_CSAIL w/ interest in multimodal representations, fine-grained perception, AI for science; prev @VectorInst, @UofTRobotics
Cambridge, Massachusetts
Joined August 2022
- Tweets111
- Following422
- Followers402
- Likes577
Pinned Tweet
My first first-authored (w/ @shobsund) paper of my phd is finally out! 🚀
Check out our thread to see how general-purpose representations + personalized synthetic data enable personalized vision representations.
🌐: personalized-rep.github.io
Personal vision tasks–like detecting *your mug*-are hard; they’re data scarce and fine-grained.
In our new paper, we show you can adapt general-purpose vision models to these tasks from just three photos!
📝: arxiv.org/abs/2412.16156
💻: github.com/ssundaram21/perso…
(1/n)
👀 open to new discussions!
Some ideas:
- make this work also when the generation process is a diffusion-autoregression hybrid
- cross-modal non-linear manifold steering? (@thomas_fel_ @GoodfireAI 👀)
- been thinking with @juliachae_ how to formalize better what modality-unique concepts are, in some automated way... in this paper we had to creatively come up with such concepts, like high-to-low resolution images or formal-vs-colloquial language
(talk to us if this is of interest to you!)
this is such a neat analysis and a great sneak peak into shared semantic geometry between modalities!
New preprint:
Cross-modal steering vectors transfer zero-shot in both directions 🖼️ ↔️ 📝
From text generation to image generation and vice-versa
Exciting evidence for a shared conceptual space!
Fantastic effort led by Donghao Zeng and awesome co-authors @nsubramani23 & @TamarRottShaham
Details 🧵👇
Julia Chae retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
Julia Chae retweeted
Thrilled to share that I’ve recently joined the incredible team at @Arda_Industries! We’re working to make manufacturing move at the speed of innovation with AI.
And we’re hiring! Come build more things with us 🚀
We started Arda in 2025 to speed up American manufacturing. We have spent the last year building with Anduril as our design partner. Today we are launching the company publicly on CNBC and announcing our Series A: $160M at a $1.5B valuation.
Our platform helps manufacturers design and operate manual assembly lines, putting designs into production faster now. Next we'll build automated lines that can be reconfigured as quickly as human workers, letting America make products that now require armies of workers in Asia.
Julia Chae retweeted
What internal structure do experts in multimodal MoEs develop during pre-training? Can we put this structure to practical use to make model adaptation significantly faster?
In our new blog post and paper with @RaphiKang and @georgiagkioxari we discuss this and more 👇
check out this cool work by jenny highlighting importance of human verification before blindly accepting LLM outputs when using them within our workflow 👀
Julia Chae retweeted
Make UMAP and embeddings cool again
Taking a short break from LLM research to go back to my roots.
Introducing... Concept Lenses! A training-free way to do similarity search along whatever concept you care about.
Typically, embedding-based similarity search is constrained by whatever notion of similarity the model learned in training. This means that if the model is primed for semantic similarity, but you actually care more about visual appearance, you'd need to switch models.
Under a Concept Lens projection, though, distance between embeddings means distance only along the concept you care about.
Blog post: jasperlu.com/blog/prompted-s…
Demo: jasperlu.com/projects/concep…
🤩 love to see scaled up supervision from clever alternative domain sources of data! synthetic data for CT go brr
X-rays are everywhere in medicine, but extracting general, quantitative anatomical information from them remains remarkably difficult.
Announcing FleXray: an open-source, open-weight model for zero-shot anatomical segmentation across the human body—from different regions, viewing angles, and acquisition settings.
FleXray learns from large-scale synthetic supervision generated from 3D anatomy, then transfers directly to real clinical X-rays.
Try it now on your own X-rays in our in-browser demo!
🧵 1/N — project page, demo, weights, code & paper below ↓
Julia Chae retweeted
a rollout from persimmon where a neolab employee gets asked what they work on and they have to deflect
no longer me after today
hiring btw @humansand
Julia Chae retweeted
So proud to announce what I’ve been working on over the last few months with the incredible @314Bansal & many other talented humans!
Persimmon is a huge first step towards a future where AI is built *for people* and we can’t wait to see what work it inspires 🌟
Do reach out (or request access to our research preview!) if you’re interested in learning more.
Julia Chae retweeted
How similar are two images? Prior metrics (e.g., LPIPS, DreamSim) give just a single score. But actually, there are multiple *senses* of similarity (color, pose, etc.)
We introduce TPIPS -- Text-Prompted Image Perceptual Similarity
"pip install tpips"
peterwang512.github.io/TPIPS 🧵
Now published in @ECOLINFORM 🎉 We show open-ended VLM retrieval can work inside an actual ecological workflow, turning ~300M incidental iNaturalist photos into a queryable dataset — surfacing previously buried phenomena and widening the range of answerable scientific questions.
Don’t miss this work at ICML!!
I'll be at #ICML2026 to present our *spotlight* work! TLDR: LLMs can learn to self-generate curricula for problems they can't yet solve, using self-play with meta-RL.
Please reach out to chat about self-improving agents, synthetic data & environments, curriculum learning, or anything else!
We've updated the paper with some fun additions ⬇️
Julia Chae retweeted
🤖 We introduce Ambient Diffusion Policy, a simple and principled method for training policies with suboptimal data in robotics.
Suboptimal data is everywhere in robotics…
❌ Data filtering is wasteful
❌ Co-training learns both good and bad features
✅ Ambient Diffusion Policy selectively learns useful features via noise-dependent data usage
👇🧵(1/5)
Julia Chae retweeted
As ecological data collection scales, expert-in-the-loop annotation is becoming essential.
These workflows are inherently transductive: the goal is to efficiently label a fixed pool of collected data.
Julia Chae retweeted
We never really knew how to train nonlinear RNNs well… BPTT struggled with vanishing grads (no long-range memory) and sequential rollout (hard to parallelizable).
What if instead an oracle told us the optimal memory state m_t at each step? Then the RNN could do one-step supervised learning on (m_t, x_{t+1}) → m_{t+1} labels.
We call this Supervised Memory Training (SMT): a replacement for BPTT that trains RNNs without unrolling them. SMT is time-parallelizable and solves vanishing gradients.
Website: akarshkumar.com/smt/
arXiv: arxiv.org/abs/2606.06479
Julia Chae retweeted
It was awesome!! Great to meet so many of you and get outside together ❤️🐦 – at Denver, CO
Presenting tomorrow (Friday June 4) at @CVPR Poster Session #2 from 4pm-6pm
📍ExHall A, #378
Looking forward to chatting!
Excited to share ID-Sim, our identity-focused similarity metric, presenting at #CVPR2026 this week in Denver! 🎉
Humans are remarkably good at distinguishing highly similar objects across different contexts.
We asked: can we train a metric that does the same?
Excited to share ID-Sim, our identity-focused similarity metric, presenting at #CVPR2026 this week in Denver! 🎉
Humans are remarkably good at distinguishing highly similar objects across different contexts.
We asked: can we train a metric that does the same?
Huge thank you to the wonderful collaborators! Would not be possible without guidance from mentors at @AdobeResearch during my internship Cusuh Ham (@cusuh_ ), Jui-Hsien Wang (@JuiHsienWang1), Nicholas Kolkin, Richard Zhang (@rzhang88) and my advisor Sara Beery @sarameghanbeery
Poster will be presented at:
🗓️ Poster Session 2 on Friday, June 4
📍Poster #378, Exhibit Hall A