
I'm a research scientist @Meta in Pittsburgh where I work on audio-visual modeling for photorealistic avatars.
Joined June 2020
- Tweets46
- Following171
- Followers731
- Likes130
‼️500h of 3D motion data released‼️
Our team at the Codec Avatars Lab just released a large scale dataset of 3D tracked human motion, including audio and text annotations. Check it out here: meta.com/emerging-tech/codec…
Working on your next big speech paper but still looking for a suitable dataset?
Check out EARS: 100h of full-band expressive, anechoic recordings of speech from 107 speakers with 22 different emotions, 7 different reading styles, and more.
sp-uhh.github.io/ears_datase…
Real Acoustic Fields are here!
Check out or dataset of densely captured room impulse responses paired with multi-view images! See you all on CVPR :)
project page: facebookresearch.github.io/r…
arxiv: arxiv.org/abs/2403.18821
Motion generation for photorealistic avatars? Say no more!
Check out how we animate full body avatars exclusively from audio input!
Paper: arxiv.org/abs/2401.01885
Project page: people.eecs.berkeley.edu/~ev…
Dataset + Code: github.com/facebookresearch/…
3D body models now have sound!
We demonstrate 3D spatial audio synthesis for 3D full body models. (NeurIPS 2023 Spotlight)
Paper: openreview.net/pdf?id=zQTi3p…
Dataset + Code (to be released): github.com/facebookresearch/…
**Dataset Release!**
We released high-quality 3D face data of 13 persons captured with up to 150 cameras while performing 100+ facial expressions!
github.com/facebookresearch/…
Photorealistic 3D avatars for everyone?! Check out this exciting work from our lab in Pittsburgh!
tinyurl.com/4xk25spe
Check out our @CVPR 2022 oral, presenting a new paradigm for audio-visual speech enhancement! We demonstrate high-quality results with a guarantee that noise can not bleed through. On top, the system allows for simultaneous voice conversion.
arxiv.org/abs/2203.17263
Some news here! We share the Facestar Dataset, a high-resolution, high-quality dataset with conversational freeform speech!
Also check out how we used that dataset for audio-visual speech enhancement in our @CVPR paper.
Get it here: github.com/facebookresearch/…
Diffusion Probabilistic Models for Speech Enhancement!
While many generative approaches still lag behind regression-based speech enhancement, we show that diffusion models prove to be particularly robust to domain shifts. @shinjiw_at_cmu
arxiv.org/abs/2202.05256
MLPs are great for 3D scene representations (NeRF etc). Not surprisingly, they also do great for acoustic scenes!
Deep Impulse Responses allow to estimate accurate audio filters from highly noisy data and interpolate better than traditional methods
arxiv.org/abs/2202.03416




