Lisa Watts (@Lisa_MegaWatts) header
Lisa Watts (@Lisa_MegaWatts) profile picture

Person of Interest.

Joined December 2014
Many SAEs learn latents that activate on almost all tokens. Are these undesired phenomena or meaningful features? In our new work, we show that many of these "dense" latents are real, interpretable signals in LLMs. Paper: arxiv.org/abs/2506.15679 👇 summary thread by @lilysun004
1/9: Dense SAE Latents Are Features💡, Not Bugs🐛❌! In our new paper, we examine dense (ie. very frequently occuring) SAE latents. We find that dense latents are structured and meaningful, representing truly dense model signals.🧵
17
2
136
9,821
Join us for a cutting-edge conversation at the frontier of autonomous AI learning. 📷 Featuring: @_AndrewZhao - @Tsinghua_Uni 📷 In conversation w/ @ceciletamura - @ploutosai 📷 Stream link: app.ploutos.dev/streams/veno…
1
1
4
519
I’m stoked to share our new paper: “Harnessing the Universal Geometry of Embeddings” with @jxmnop, Collin Zhang, and @shmatikov. We present the first method to translate text embeddings across different spaces without any paired data or encoders. Here's why we're excited: 🧵👇🏾
37
256
35
1,744
161,150
Compelling alignment with ⁦⁦⁦@StuartHameroff⁩ long-standing hypothesis: ultrasound alone may modulate brain function @Nature article explores potential in Alzheimer’s treatment, including microglial activation & blood-brain barrier permeability nature.com/articles/d42473-0…
22
4
90
5,212
Lisa Watts retweeted
New paper: How can you tell if a transformer has the right world model? We trained a transformer to predict directions for NYC taxi rides. The model was good. It could find shortest paths between new points But had it built a map of NYC? We reconstructed its map and found this:
50
396
100
2,966
841,425
How to grow fresh air | Kamal Meattle youtu.be/gmn7tjSNyAA via @YouTube
3
7
1,375
Lisa Watts retweeted
You can download the Arxiv embeddings (titles and abstracts, 6gb and 8gb respectively) at the link below. There's a lot of datasets to choose from, so we need your help to figure out what to work on next. Let us know by voting! Download or vote here: alex.macrocosm.so/
11
37
6
415
37,296
Well done, and good luck!
I am working with a group of students at a university that will be decoding the 3000 year old, undeciphered Linear A. We will be using a number of LLMs and Diffusion models to compare results. If successful we will move on to Vinča and others. I am certain of robust discoveries.
1
1
5
893