Building AI agents, previously AI/ML at big tech. In my spare time explore the intersection of society, computers, complexity, and intelligence.
New York, NY
Joined August 2019
- Tweets359
- Following1.7K
- Followers108
- Likes3.3K
IdeaRadar retweeted
📢 It looks like relative representations are here to stay!
I'm beyond thrilled to announce that our work has been selected as one of the notable top 5% (oral) papers at #iclr23 ! 🥳
teapaw.org/moschella_luca/status/…
[1/5]
Welcome Relative Representations, enabling zero-shot communication between latent spaces without any training!
arxiv.org/abs/2209.15430
It turns out that distinct neural networks learn intrinsically equivalent latent spaces [1/6]
IdeaRadar retweeted
We have little mechanistic understanding of how deep learning models overfit to their training data, despite it being a central problem. Here we extend our previous work on toy models to shed light on how models generalize beyond their training data.
transformer-circuits.pub/202…
IdeaRadar retweeted
Neural networks often pack many unrelated concepts into a single neuron – a puzzling phenomenon known as 'polysemanticity' which makes interpretability much more challenging. In our latest work, we build toy models where the origins of polysemanticity can be fully understood.
IdeaRadar retweeted
Working on a walkthrough of transformer code with side-by-side comparison to the computation graph. Felt the default image everyone uses is hard to interpret. Also, I like to show the matrices in my computation graphs. § refers to sections in a colab notebook (coming soon)
ALT Transformer architecture computation graph
IdeaRadar retweeted
It should be illegal to launch a startup without memorizing these 21 principles from Y-Combinator's Startup Playbook:
IdeaRadar retweeted
MDM: Human Motion Diffusion Model
abs: arxiv.org/abs/2209.14916
project page: guytevet.github.io/mdm-page/
IdeaRadar retweeted
A small child once told me "Every map should lead to treasure" and I have developed a completely incoherent life philosophy around this wonderful little truism
IdeaRadar retweeted
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
project page: gweb-research-imagen.appspot…
sota FID(7.27 on COCO), without ever training on COCO, human raters find Imagen samples to be on par with the COCO data itself in image-text alignment
IdeaRadar retweeted
Has machine learning solved computer graphics?
Let's find out by trying to re-create a bunch of classic graphics images using #dalle2! A thread. 🧵 [1/n]
Left: original image
Right: DALL-E 2 image
In each case I tried many times & show the best result. Full query string given.
IdeaRadar retweeted
Read a bit about Grokking recently. Here's some learnings:
"Grokking" is a curious neural net behavior observed ~1 year ago (arxiv.org/abs/2201.02177).
Continue optimizing a model long after perfect training accuracy and it suddenly generalizes.
Figure:
IdeaRadar retweeted
Some gradients between two colours look good, others suck. It's easy to tell the difference, but what *is* that difference?
IdeaRadar retweeted
"Triadic interactions induce blinking and chaos in the connectivity of higher-order networks".
Here percolation turns into a fully-fledged dynamical process! with @sunhanlin151 Filippo Radicchi and Juergen Kurths. Check it out!!
arxiv.org/abs/2204.13067
IdeaRadar retweeted
Two very particular patterns of random movement — Brownian motion and Lévy walks — describe a great many physical phenomena. Recent data hint that Lévy walks may be a default movement pattern for many animal species. quantamagazine.org/random-se…
IdeaRadar retweeted
How to sell info products in 2022:
1. Host your content in @NotionHQ.
2. Design product cover in @canva.
3. Upload product to @gumroad.
4. Build website in @typedreamHQ.
5. Launch on @AppSumo & @ProductHunt.
Don’t overcomplicate it.
IdeaRadar retweeted
Sara (SFK) and @alexyu00 just presented Plenoxels: Radiance Fields without Neural Networks at the morning oral session!
We will be around for the in-person🎉poster session!
alexyu.net/plenoxels/ #CVPR2022
IdeaRadar retweeted
Elucidating the Design Space of Diffusion-Based Generative Models
abs: arxiv.org/abs/2206.00364
improve efficiency and quality obtainable with pre-trained score networks from previous work, including improving the FID of an existing ImageNet-64 model from 2.07 to near-SOTA 1.55
IdeaRadar retweeted
And a powerful, informative prior for the latent variables! Our recent #ICML2022 paper (arxiv.org/abs/2206.05895 ) investigates “diffusion prior” in DLVM and it works out pretty well on capturing complex latent structure and interpretable text modeling (1/x)
