Gianluigi Silvestri (@gisilvs) profile picture
Joined January 2014
Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)Photo by Gianluigi Silvestri (@gisilvs)
1/6 I built Ava, a synthesizer that turns text descriptions into editable presets. Not just audio samples, but actual sounds you can play, tweak, and save. I wrote about the whole project, with audio examples and an interactive demo: gisilvs.github.io/blog/ava/
1
4
175
5/6 The results are a bit hit or miss. Some sounds come surprisingly close, while others end up somewhere completely different. But since they're editable presets, those happy accidents can become something new.
1
20
6/6 I also turned it into a C++ plugin for Mac, so you can actually make music with the generated presets. Everything is open source: PyTorch + training: github.com/gisilvs/ava-torch C++ plugin: github.com/gisilvs/ava-cpp Would love to hear what you think!
19
Gianluigi Silvestri retweeted
Tired to go back to the original papers again and again? Our monograph: a systematic and fundamental recipe you can rely on! 📘 We’re excited to release 《The Principles of Diffusion Models》— with @DrYangSong, @gimdong58085414, @mittu1204, and @StefanoErmon. It traces the core ideas that shaped diffusion modeling and explains how today’s models work, why they work, and where they’re heading. 🧵You’ll find the link and a few highlights in the thread. We’d love to hear your thoughts and join some discussions! ⚡ Stay tuned for our markdown version, where you can drop your comments!
51
498
58
2,403
926,442
Gianluigi Silvestri retweeted
I spent the past month reimplementing DeepMind’s Genie 3 world model from scratch Ended up making TinyWorlds, a 3M parameter world model capable of generating playable game environments demo below + everything I learned in thread (full repo at the end)👇🏼
94
267
42
2,377
223,015
Gianluigi Silvestri retweeted
1/2) Have you have noticed that the forward process in a diffusion model looks a lot like the reparameterization trick in VAEs? It turns out that there is a deep connection! Curious? Watch our new vedio in the Generative Memory Lab channels (link below)
10
76
678
38,243
New preprint out! Reach out if you are interested, or let's grab a coffee at ICML this week!
1/3) I am biased, but I think this is going to be big! CoVAE: Consistency Training of Variational Autoencoders We unify consistency models with VAEs to obtain a powerful and elegant generative autoencoder! The brainchild of the brilliant @gisilvs (who is looking for jobs!)
1
11
896
Gianluigi Silvestri retweeted
I am very happy to share our recent work on consistency model led my my brilliant PhD student @gisilvs during his internship at @SonyAI_global in collaboration with @JCJesseLai, @takiko_san and @mittu1204. "Training Consistency Models with Variational Noise Coupling"
2
16
3
75
16,031
Gianluigi Silvestri retweeted
I am very happy to share: " Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion" We study the evolving latent geometry of diffusion by analyzing gaps in the Jacobian spectrum with statistical physics methods Paper: arxiv.org/pdf/2410.05898
6
65
1
442
35,111
Gianluigi Silvestri retweeted
Amazing presentation by Marvin Li on critical windows in diffusion models. A work very closely related to our research on spontaneous symmetry breaking. youtu.be/NbnBmi5k1UQ?si=86gH…
6
18
2,440
Gianluigi Silvestri retweeted
Job alert! I am looking for a talented PhD student to do research on generative diffusion and its connections to statistical physics and memory. Feel free to send me a PM if you need more info! You can apply here: ru.nl/en/working-at/job-oppo… Otherwise, please retweet!
8
62
5
187
92,746
Gianluigi Silvestri retweeted
An amazing talk by @aaron_lou on diffusion language transformers. Bracing for the era of large language diffusion! youtu.be/_1qv_LNjH9U?si=xgNW…
2
33
1
182
24,256
Gianluigi Silvestri retweeted
You should follow our Generative Memory Lab channel! Plenty of content on generative models, diffusion models, associative memory! youtube.com/@gemlab3695?si=J…
2
20
1
92
11,479
Gianluigi Silvestri retweeted
A very insightful presentation about probabilistic Hopfield nets and their link with diffusion models! youtu.be/hK_L9Typ8PM?si=SrH5…
1
30
1
86
6,950
Gianluigi Silvestri retweeted
Modern Hopfield networks are related to transformers, but did you know that they are mathematically equivalent to generative diffusion models? Happy to share: "In search of dispersed memories: Generative diffusion models are associative memory networks" arxiv.org/abs/2309.17290
8
154
9
733
131,809
Fantastic presentation from @dblueeye on diffusion models with discrete diffusion process 👏👏 very clear explanation and promising results for their 'Blackout Diffusion'. Take a look at the presentation here youtu.be/LnCI7I7FmAA?si=9F8n…
3
1
9
1,009