Senior Research Associate @ University of Bristol | Research Focus: Multimodal Learning.
Bristol, England
Joined November 2011
- Tweets27
- Following85
- Followers108
- Likes186
Jacob Chalk retweeted
Come find us at ExHall poster #70 at 4:30 PM CEST during Session 2 at #ECCV2026!
We’ll be presenting EPIC-Contact and HOPformer, our work on recovering 3D hands and objects from cluttered, occluded egocentric video. Looking forward to chatting!
sid2697.github.io/epic-conta…
Today at ECCV:
Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation, ExHall Poster: #70, PM CEST, Poster Session 2.
* 3D hand–object pose estimation in unconstrained, cluttered, occluded egocentric video
* EPIC-Contact, an in-the-wild dataset with dense 3D hand–object contact
* HOPformer, a transformer that conditions object pose on strong hand priors to predict both hands and the object in a single forward pass.
co-authors: @Sid__Bansal, @zhifan_zhu , @sha2nk_t, @terry_jhzhao00, @dimadamen
sid2697.github.io/epic-conta…
Humans maintain a mental map of “what is where” for many objects, even if they are out of sight, but how does a model do this? That’s what we tackle in our #ECCV2026 paper, Whareformer, which we present tomorrow.
📍 Poster 363 · Session 4
🗓️ Friday 11 Sept 4:00–6:00 PM CEST
🧵1/3
Highlights:
The first learning-based approach to the "Out of Sight, Not Out of Mind" task
Jointly reasons about the appearance and location of an object to track it
Learns when to create new tracks in one forward pass
SOTA on 260 long test videos across 3 datasets
🧵2/3
Come find us at the poster to find out more!
Project page: jacobchalk.github.io/Wharefo…
Arxiv: arxiv.org/abs/2607.08537
Code & Data: github.com/JacobChalk/Wha…
In collaboration with: @sinha_saptarshi, @dimadamen, Yannis Kalantidis, and @dlarlus .
🧵3/3
Jacob Chalk retweeted
NEW
Whareformer: Learning to Track What is Where in Long Egocentric Videos
@eccvconf #ECCV2026 paper
The first online learning approach to track dynamic objects in ego video reasoning on appearance &3D location w track memory & explicit new track token.
jacobchalk.github.io/Wharefo…
🧵
Jacob Chalk retweeted
*NEW* Our #ECCV2026 @eccvconf paper
Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation
Now on ArXiv w Dataset, Code&model
sid2697.github.io/epic-conta…
arxiv.org/abs/2606.30598
Two contributions:
1. EPIC-Contact Dataset
2. HOPformer Method &Checkpoint
🧵 1/6
Jacob Chalk retweeted
Our #ECCV2026 @eccvconf paper
Gen2Balance: Generative Balancing for Long-Tailed Video Action Recognition
now on ArXiv w/ code&data&models
prajwalgatti.github.io/gen2b…
Can we fill imbalanced action recognition datasets with generative images to outperform long-tail SOTA?
Yes we can
🧵
Jacob Chalk retweeted
*NEW PAPER&MODEL*
HOI-DETR for detecting hands, object in-hand, &object-interacted-with (through a tool).
This is the foundation model you've been waiting for! One that works off-the-shelf, single-image, using strong detectors, generalises and is stable
ahmaddarkhalil.github.io/HOI…
🧵
🚀 Call for papers!
Take part in Uniworld workshop @ ECCV’26. We welcome submissions on visual foundation models, multimodal learning, world models, embodied intelligence, 3D/4D scene understanding, and generalisation.
More details at: uniworld-ai.github.io/
🚀 Call for Papers
Very excited to announce the UniWorld workshop @ ECCV’26 on Universal Representations for Perception, Reasoning, and World Modeling
We welcome both archival/non-archival submissions.
📌 Deadlines: July 1 / July 20, 2026
More details: uniworld-ai.github.io
Jacob Chalk retweeted
📢 Preprint
Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach
arxiv.org/abs/2512.16456
masashi-hatano.github.io/pri…
First work to synthesise the natural behavior of full-body priming (i.e. gazing towards) an object b4 reaching - curated from 5 public datasets.
1/3
Ecstatic to announce that I passed my PhD yesterday. The past 4 years have been an incredible experience and I am so grateful to all the people I’ve worked with and who have supported me throughout, especially @dimadamen who has been an amazing supervisor!
Congratulations Jacob Chalk @JacobChalkie who passed his PhD viva @bristolcs on
"Leveraging Multimodal Data for Egocentric Video Understanding" w no corrections
📜ICASSP23, CVPR24, CVPR'25, 3DV'25, TPAMI'25
jacobchalk.github.io
🙏examiners @HildeKuehne @andrewhowens @WayHomeLi
Jacob Chalk retweeted
An extended EPIC-SOUND paper was accepted at TPAMI available on ArXiv
arxiv.org/abs/2302.00646v3
This follows @ieeeICASSP 2023 oral short paper, extended for detection and further analysis
epic-kitchens.github.io/epic…
work by @huh_jaesung @JacobChalkie E Kazakos @Oxford_VGG @BristolUni
Jacob Chalk retweeted
Attending #AI_UK #AI #UK @turinginst today&tomorrow?
Make sure to visit our demonstration booth - Towards Wearable Assistive AI @BristolUni @JGIBristol - our journey into world leading egocentric datasets
@mwray0 @Sid__Bansal &Rhodri Guerrier are representing the team in London.
New Dataset released! This was a huge project from the whole lab and a monumental effort. We’re delighted that we can announce it now and would like to encourage anyone who is interested to check out the paper, webpage, wide range of annotations and the VQA Benchmark!
🛑📢
HD-EPIC: A Highly-Detailed Egocentric Video Dataset
hd-epic.github.io
arxiv.org/abs/2502.04144
New collected videos
263 annotations/min: recipe, nutrition, actions, sounds, 3D object movement &fixture associations, masks.
26K VQA benchmark to challenge current VLMs
1/N
Jacob Chalk retweeted
We published an extended version of our #ICASSP2023 paper:
EPIC-SOUNDS: A Large-scale Dataset of Actions That Sound
+ sound event detection baseline
+ detailed annotations pipeline
+ analysis of visual vs audio events
+ audio-visual models
arxiv.org/abs/2302.00646v2
Jacob Chalk retweeted
Our seminal paper (yes, I do believe this is transformative to the field)
"Spatial Cognition from Egocentric Video: Out of Sight not Out of Mind"
is accepted @3DVconf #3DV2025
Camera ready soon
Congrats 2 great coauthors @plizzari38126 @goelshbhm Toby @JacobChalkie @akanazawa
🆕on ArXiv
Out of Sight, Not Out of Mind
Spatial Cognition from Egocentric Video.
dimadamen.github.io/OSNOM/
arxiv.org/abs/2404.05072
3D tracking active objects using observations captured through egocentric camera. Objects are tracked while in hand, from cupboards and into drawers
We will be presenting our CVPR paper “TIM: A Time Interval Machine for Audio-Visual Action Recognition” this Friday (23rd). If you’re interested in finding out more, make sure to register and check out these links:
Paper: arxiv.org/abs/2404.05559
Webpage: jacobchalk.github.io/TIM-Pro…
In the 56th session of #MultimodalWeekly, we have three exciting presentations across different video understanding tasks: action recognition, video description, and video summarization.
Was great to present TIM with @huh_jaesung and @dimadamen yesterday at #CVPR2024! Thank you to everyone who came and asked questions! If you’re interested in the paper you can find more at:
Paper: arxiv.org/abs/2404.05559
Webpage: jacobchalk.github.io/TIM-Pro…
Proud that can finally announce TIM! A lot of hard work has gone into this and we’re glad to be accepted into CVPR! Check out the paper and project webpage in the following:
Paper: arxiv.org/abs/2404.05559
Webpage: jacobchalk.github.io/TIM-Pro…
🆕📰📢 on @arxiv w code&Models
Our @CVPR #CVPR2024 paper
TIM: A Time Interval Machine for Audio-Visual Action Recognition
arxiv.org/abs/2404.05559
jacobchalk.github.io/TIM-Pro…
TIM uses modality-specific time intervals as queries, to attend to the interval in long audio-visual input.
Jacob Chalk retweeted
📢 "An Outlook into the Future of Egocentric Vision"
44 pages + 385 references survey now available on @openreviewnet
We invite comments/suggestions/corrections from researchers for 30days.
Major contributions will be acknowledged
[instructions in 🧵]
openreview.net/forum?id=V397…
1/4
