Machine Learning for Audio | Research Engineer @AdobeResearch | Previously @NI_News @iZotope @Microsoft @Sony @AudioshakeAI
San Francisco
Joined January 2022
- Tweets36
- Following254
- Followers261
- Likes270
Pinned Tweet
🥳 Excited to share that our latest work on generating sample-based instruments from text or audio prompts has been accepted at #ISMIR2024! 🎹 — Our take on making AI-based music generation models useful for musicians!
🔊 gen-inst.netlify.app
📜 arxiv.org/abs/2407.15641
(1/6)
Johannes Imort retweeted
✨ Train language models directly on continuous data - without tokenization ✨
We propose an easy way to train GPT-style autoregressive models on continuous data, without error accumulation.
We test it on audio 🔊, but this method can easily work with other modalities 🎆
👇🧵
#ISMIR2024 in San Francisco was an absolute blast! 🌉 Big thanks to the organizers and attendees for creating such an inspiring event. Looking forward to see you all in South Korea next year, titans!
In case you missed it, here is our paper demo video!
🔊 Demo page: gen-inst.netlify.app/
📄 Full paper: arxiv.org/abs/2407.15641
Johannes Imort retweeted
``FINALLY: fast and universal speech enhancement with studio-like quality,'' Nicholas Babaev, Kirill Tamogashev, Azat Saginbaev, Ivan Shchekotov, Hanbin Bae, Hosang Sung, WonJun Lee, Hoon-Young Cho, Pavel Andreev, ift.tt/RWPXAVt
Johannes Imort retweeted
AFX-Research: an Extensive and Flexible Repository of Research about Audio Effects
All the research on audio effects of the last few decades. A table with lots of metadata. Search. Filter. Order. Contribute.
repo: github.com/mcomunita/AFX-Res…
web: mcomunita.github.io/AFX-Rese…
Johannes Imort retweeted
GRAFX is an open-source library for audio graphs in PyTorch. Audio processing can be efficiently done on GPU with batched processing and various differentiable audio effects, including a multitap delay and zero-phase EQ
Great work by @SunghoL10754073 !
github.com/sh-lee97/grafx
GRAFX: An Open-Source Library for Audio Processing Graphs in PyTorch
Been waiting for this one. An extremely useful library for data generation related to audio effects with many applications in audio engineering. Code now available
arxiv.org/abs/2408.03204
🎶 We deliberately chose this task as an alternative to more conventional prompt-to-audio systems and we hope to expand sound synthesis possibilities for music creators in order to realize their artistic visions. (5/6)
🙌 A big thank you to my collaborators Shahan Nercessian, Ninon Devis, and @f_blang at Native Instruments/@NI_News for their incredible work. Can’t wait to see you all at #ISMIR2024/@ISMIRConf in San Francisco! 🌉 (6/6)
Johannes Imort retweeted
Excited to share our latest work: Distortion Recovery: A Two-Stage Method for Guitar Effect Removal, published at #DAFx 2024! 🎸 This work is in collaboration with @PositiveGrid.
🔗 Demo page: y10ab1.github.io/guitar_effe…
📄 Paper: arxiv.org/abs/2407.16639
🎹Ever wondered what a certain word or phrase might sound like if it were transformed into a musical instrument?
🎉We are excited that our work on #InstrumentGen has been accepted at the #NeurIPS2023 ML for Audio Workshop!
📄arxiv.org/abs/2311.04339
🔊instrumentgen.netlify.app/ (1/4)
🎶 To achieve this, we propose InstrumentGen, which leverages components of MusicGen, Descript Audio Codec, and CLAP!
📊 Plus, we're introducing a new differentiable loss function for assessing timbral consistency within sample-based instruments. (3/4)
We hope to establish a foundational baseline, paving the way for further research in automatic sample-based instrument generation as a complementary approach to DDSP-based methods.
Had a blast working on this with Shahan Nercessian at Native Instruments/@NI_News! (4/4)
Johannes Imort retweeted
InstrumentGen: Generating Sample-Based Musical Instruments From Text
Introduces the task of text-to-instrument. By leveraging components of MusicGen, Descript Audio Codec, and CLAP they propose a model for this task.
abs: arxiv.org/abs/2311.04339
web: instrumentgen.netlify.app/
Johannes Imort retweeted
What if we had a way to remove audio effects from a signal?
It may seem a bit strange at first, but Johannes I. of AudioShake shows some compelling use cases for removing effects, as well as how he pulled it off for his internship at Sony.
youtu.be/O0dMfcHkiJI
Today, I'll be discussing the practical aspects of audio effect removal at the Neural Audio Plugin Competition! 🎛 With 4 additional exciting presentations lined up, you won't want to miss out. Join us at 1 PM EST / 5 PM GMT!
This Friday at 1pm EST / 5pm GMT we have a BIG event - 5 guests will be joining us from around the world to present their work in neural audio, AI, and music!
Teresa Pelinski (QMUL) - "Toward Prototyping with Neural Networks on Embedded Hardware"
Haici Yang (Indiana University) - "Latent Space
Processing for Music Remixing and Upmixing"
CJ Carr (Dadabots / Harmonai) - "Al Death Metal-Eliminating Humans From Music"
Johannes Imort (AudioShake) - "Exploring the Potential of Audio Effect Removal for Music Production"
António Ramires (Beatoven.ai) - "How Deep Learning Can Improve Sample Retrieval for Electronic Music Creation"
Join us here: youtube.com/live/eEqZJ41PbpQ…
