Megi Dervishi retweeted
Replying to @hugues_va
Yeah LeJEPA is not a JEPA, that's some bullshit and I'm annoyed about it
Megi Dervishi retweeted
see you at ICML in Seoul and the AMI × SBVA mixer on thursday evening! looking forward to meeting everyone :)
luma.com/rv0q6b5q
Megi Dervishi retweeted
V-JEPA: a step towards getting machines to understand how the world works by watching.
The Joint embedding Predictive Architecture (JEPA) is a non-generative architecture that predicts the representation of a signal from a corrupted or transformed version of that signal.
In V-JEPA the signal is a short video that is corrupted by masking a large portion of each frame.
After training, the learned representation is used as input to a simple classifier tested on action recognition from videos.
The system gives excellent results on action recognition on SSv2 and K400 with frozen backbone and fine-tuning.
blog post: ai.meta.com/blog/v-jepa-yann…
paper: ai.meta.com/research/publica…
code (CC-BY-NC): github.com/facebookresearch/…
Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards @ylecun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks.
Details ➡️ bit.ly/49fCeaM
We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information.
It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency.
The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training.
Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models.
We believe that this work is an important milestone on the path to advancing machine intelligence.
Megi Dervishi retweeted
Have a look at the paper for more details!
arxiv.org/abs/2311.15930
Work done at @AIatMeta
Core WorldSense team: @megdrv @marksibrahim @pascal20100 @benchek_youssef @_dieuwke_ & Emmanuel Dupoux







