Alex Oh retweeted
Yann LeCun and his team dropped yet another paper!
"V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning"
In this V-JEPA upgrade, they showed that if you make a video model predict every patch, not just the masked ones AND at multiple layers, they are able to turn vague scene understanding into dense + temporal stable features that actually understands "what is where".
This key insight drove improvements in segmentation, depth, anticipation, and even robot planning.
Alex Oh retweeted
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
Ever see a Waymo and wonder why it needs those bulky LiDAR sensors? While modern autonomous systems rely on expensive, specialized hardware, humans navigate the world through vision alone. This motivated our development of LeDEEP, a vision-only framework that uses LeJEPA’s self-supervised learning to extract 3D structure from simple camera frames. By leveraging stable SSL objectives to produce rich latent spaces, we’ve built a resource-efficient alternative that show that robotics doesn't need a thousand-dollar "hat" to perceive depth.
Read More Here:
lnkd.in/gSBrfkmW
43
Alex Oh retweeted
It’s extremely good that Anthropic has not backed down, and it’s siginficant that OpenAI has taken a similar stance.
In the future, there will be much more challenging situations of this nature, and it will be critical for the relevant leaders to rise up to the occasion, for fierce competitors to put their differences aside. Good to see that happen today.
Who taught us how to stand?
No one really.
We fell down. We got up. We did it again. It's that simple
As it turns out, AI models can learn the same way. They fall. They adjust. They try again. It's just gravity, repetition, and a stubborn refusal to stay on the floor.
As a Master’s student pursuing into AI research, I’ve been exploring how to make reinforcement learning more efficient through self-supervised frameworks. I took a strong recent goal-conditioned RL setup and asked a question:
Can we replace the critic's heavy contrastive learning (InfoNCE) with a lighter, theoretically grounded regularization method (SIGReg) without breaking performance?
The answer was surprisingly nuanced:
On the Ant → the simpler method worked beautifully.
On the Humanoid → it reached the goal but couldn’t stay standing. Why?
As it turns out, standing still isn’t a passive state, it’s an active micro-policy of constant balance corrections. Contrastive loss naturally learned those fine distinctions; plain regularization didn’t.
So I tried a hybrid: InfoNCE + SIGReg with a unified encoder.
The Result: Identical performance… but 24 % faster training wall-time.
I also ran capacity ablations, froze critics, tried SIGReg on the actor: several “what if” experiments you do when something doesn’t work as expected. Every failure taught me something useful.
In both humans and machines, learning what to do is fragile. Learning what to do and what not to do is robust.
The full write-up (with plots and honest lessons) is now on my blog → api.wandb.ai/links/aho13-duk…
If you’re working on RL, robotics, or efficient self-supervised learning, I’d love to hear your thoughts.
@randall_balestr #ReinforcementLearning #DeepLearning #WorldModels
29
Embedding models are the hidden engines behind modern search. These models translate data into lists of numbers (embeddings) which allow systems to understand the meaning of a document or image. As image-text embeddings become central to multimodal LLMs and information retrieval processes, their reliability is critical.
We have been exploring Projected Gradient Descent (PGD) attacks, attacks that make small pixel changes to images that cause significant embedding drift, leading the model to completely misinterpret the input. This highlights a risk for information retrieval or performance when ingesting unverified public data.
The upside is that effective defenses exist. Our initial tests show evidence of the following:
1. Newer, larger models seem to be more robust to these attacks.
2. These are "white box" attacks: attackers usually require prior knowledge about the model type to be successful.
3. Strategies like adversarial training on PGD attacks can successfully mitigate these vulnerabilities.
To visualize this vulnerability, we used a PGD attack to force the model to identify this image of Tom Brady as "A Brick."
- Center (CLIP Base): The attack succeeded. The model's similarity score for "A Brick" (0.31) actually overtook the score for "Patriots Football Quarterback" (0.28). The model is more confident that this is a picture of "A Brick" than Tom Brady.
- Right (CLIP Large): The attack failed. As noted above, the larger model was more robust to this attack; the model correctly maintained "Patriots Football Quarterback" as the top match despite the adversarial noise.
I would be happy to share more information about our results and code. I would love to hear your thoughts!
References:
Github Repo : lnkd.in/ecEiuvX9
MMEB Dataset (Experiment Benchmark data and Tom Brady Photo): lnkd.in/e22h7uq7
13






