Research Engineer @GoogleDeepMind working on video understanding.
Joined June 2008
- Tweets596
- Following244
- Followers410
- Likes1.6K
Tobias Weyand retweeted
We’re hiring! This is a rare opportunity to work with the folks that built Veo, Genie, and Gemini Omni — and advance the frontier of world models.
More details below 👇
Gemini's Agentic Video Understanding delivers a massive leap in long video reasoning on Minerva (github.com/google-deepmind/n…) . +5.3pp accuracy at only 42% of the token cost!
I really enjoyed "The AI Doc: Or How I Became an Apocaloptimist" (imdb.com/de/title/tt39150120…). Interviews with lots of big names, balanced view of ai-doomers vs ai-optimists and an attempt to answer what it all means for our kids.
Attending #ICCV2025? Come chat with us about our Minerva dataset that tests if models can truly reason about videos! 🕵️♀️
@ahmetius and @SachitMenon will be presenting the dataset at Poster Session 5 tomorrow (Thurs, Oct 23) morning. Find them at poster #391.
Our team is hiring! If you have experience in video understanding and/or generation, join us @GoogleDeepMind and help push the frontiers with Veo and Gemini!
We're hiring at @GoogleDeepMind! Looking for a talented Research Engineer to help build the future of Video generation and undrestanding (Veo and Gemini).
Apply here: job-boards.greenhouse.io/dee…
Excited that our Minerva and Neptune datasets are both featured in the Gemini 2.5 tech report! Minerva is among the most challenging video benchmarks with a large gap between SotA (Gemini 2.5 Pro, 67.6%) and humans (92.5%).
github.com/google-deepmind/n…
Tobias Weyand retweeted
Excited! VideoPrism-Base/Large are publicly available now: github.com/google-deepmind/v…
Check it out if you need a versatile video encoder for video-language or video-native tasks. Feedback appreciated!
Introducing VideoPrism, a single model for general-purpose video understanding that can handle a wide range of tasks, including classification, localization, retrieval, captioning and question answering. Learn how it works at goo.gle/49ltEXW
Gemini 2.5 Pro sets the state of the art on our newly released Minerva video reasoning benchmark by scoring 63.5%.
📜 Paper: arxiv.org/abs/2505.00681v1
📊 Dataset: github.com/google-deepmind/n…
We're excited to release Minerva 🕵️♀️, a benchmark to evaluate if AI can truly reason about videos, from spotting game-changing moments in sports 🏀 to understanding character motivations in short films 🍿. We provide the "why" behind the answers! Pointers below 👇
The newly released Gemini 2.5 Pro (Preview 05/06) sets the state-of-the art on Minerva with 63.5% accuracy. Human accuracy is 92.5%.
developers.googleblog.com/en…
Listen to the @agi_breakdown episode on Minerva here:
aibreakdown.org/arxiv-paper-…
Excited to share Long-Video Masked Autoencoder (LVMAE) our team just published at @NeurIPSConf! We boost the context length of video models using an adaptive decoder and a dual-masking strategy and achieve SotA on several video benchmarks.
Paper: arxiv.org/abs/2411.13683
Training video understanding models on longer contexts is computationally intensive. To address this, we present a novel approach that reduces the computational load while also improving the quality of the learned representations. More at: goo.gle/4fW5aIc
ALT Illustration of the adaptive tokenizer.
Thank you @JeffDean , very much appreciate the boost! This is really a team effort with my amazing colleagues @NagraniArsha, Mingda Zhang, @raminia, Rachel Hornung, @nitesh_ai, @under_fitting, Austin Meyers, @zhouxy2017, @BoqingGo, @CordeliaSchmid, @sirotenko_m, @ZhuZhu66595.
A nice new benchmark for long video understanding by Tobias Weyand @0xtob and others. This is likely to be one of the new frontiers of capabilities for large-scale multimodal models, and it's great to have a new benchmark to assess others in this area.
Excited that our work on Long video understanding is being featured by @GoogleAI !
Can #AI truly understand long videos? Tobias Weyand & the Google Research team are testing the limits w/ Neptune, an open-source benchmark for long video understanding. Dive into the details & see how AI tackles temporal reasoning, cause & effect, & more →goo.gle/4esTTNM
The other day I let my kids talk to Gemini live. Today my 3 year old asked my 6 year old: "Can you tell me a joke?" - 6 year old: "Sorry, I'm just a language model."
Excited to share what our team has been working on! With expanding context lengths, frontier models are able to process longer and longer videos. But how well do they really understand them? Today we release Neptune, a challenging benchmark for long video understanding.
Datasets for evaluation of long video understanding are rare. So with this in mind, today we describe Neptune, an open-source evaluation dataset that includes multiple-choice and open-ended questions for videos of variable lengths up to 15 minutes. More →goo.gle/3B41nZV
ALT The Neptune data pipeline.
New long video understanding benchmark from my colleagues @GoogleDeepMind pushing LLMs to their limits!
Can current LLMs solve video reasoning Qs like:
Over 1-hour, when does the camera holder go down stairs... ??
Watch the teaser... Can you distinguish up/down stairs - p.s. stairs are not visible when you go down any
youtu.be/Ddgvr4OReL4
Hour-Long PerceptionTest VQA @eccvconf