
ai scaling + simplifying. founder @LatentSpaceAI spoilers from my favorite story: “Overhead, without any fuss, the stars were going out”
San Francisco, CA
Joined February 2014
- Tweets82
- Following1.5K
- Followers417
- Likes122
Ian Thompson retweeted
THRILLED to share that ChatGPT, our new model optimized for dialog, is now public, free, and accessible to everyone. chat.openai.com/
Try talking with ChatGPT, our new AI system which is optimized for dialogue. Your feedback will help us improve it. openai.com/blog/chatgpt/
Ian Thompson retweeted
Avalon is the world's fastest 3D simulator for RL agents. All baselines train on 1 GPU in ~1 day.
We want academic researchers to be able to study aspects of intelligence missing from today’s models, even w/o access to large-scale compute.
Get started: generallyintelligent.com/ava…
Ian Thompson retweeted
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascie…
Ian Thompson retweeted
Bluesky is building a social protocol. We released “ADX” (the X stood for Experiment) in May. Now that the design is starting to solidify, we’re renaming it to the “Authenticated Transport Protocol” — the “AT Protocol.”
blueskyweb.xyz/blog/10-18-20…
Ian Thompson retweeted
Could a DAO build the next great city? Honored to give the first TED Talk on DAOs!
ted.com/talks/scott_fitsimon…
Ian Thompson retweeted
It takes a lot of human ratings to align language models with human preferences. We found a way to learn from language feedback (instead of ratings), since language conveys more info about human preferences. Our algo learns w just 100 samples of feedback. Check out our new paper!
Can we train LMs with *language* feedback?
We found an algo for just that. We finetune GPT3 to ~human-level summarization w/ only 100 samples of feedback
w/ @jaa_campos @junshernchan @_angie_chen @kchonyc @EthanJPerez
Paper: https://teapaw.org/t.co/BBeQbFMtVi
Talk: https://teapaw.org/t.co/48uwmwakOH
Ian Thompson retweeted
Excited to share a peek at what I've been working on recently: we're building networks of little sensors and using ML to help wind farms produce more energy!
Ian Thompson retweeted
It’s technically possible for our online identities and relationships to belong to us the way an email address, phone contact book, or password keychain does. So I think it will happen, one way or another. It’s just a matter of time.
Always love a Library of Babel reference :)
Catch @DarrylBarnhart speak @NVIDIAGTC today about our exciting work @LatentSpaceAI on scaling LLM with retrieval! 📚
There’ll be an open QA in 30 mins: reg.rainfocus.com/flow/nvidi…
2
Here’s another memetic isomorphism:
Crypto is more like physics, and AI is more like biology – not in “biologically inspired” sense, but in the nature of possible solutions.
So using the wordcel / rotator basis, Darwin is a wordcel and Newton is the ultimate rotator. (1/4)
Replying to @VitalikButerin
16. It's fascinating how the two axes seem to imply each other:
Pure tinkering is fast and effective, but it is unprincipled, so lacks large-scale legitimacy, so can only cooperate through centralization
Theorycel thinking is slower, but well-suited for decentralization
Would Darwin/Newton gravitate more towards AI or crypto today?
Newton seems like he’d be more interested in snark hunting, while Darwin would work on scaling laws. (3/4)
1
2
...though Newton did spend ~20% of his time working on alchemy, and we've had some open problems in AI alchemy for a while 😂 youtube.com/watch?v=x7psGHga… (4/4)
3
Ian Thompson retweeted
Our first AI alignment paper, focused on simple baselines and investigations: A General Language Assistant as a Laboratory for Alignment arxiv.org/abs/2112.00861
Ian Thompson retweeted
Excited to share a bit about what @vmcheung and I have been working on this year!
At Gantry, we build infrastructure to help you go from making ML projects to shipping ML-powered products.
I wrote about some of the main challenges we see here: gantry.io/blog/toward-contin…
TL/DR 👇
Ian Thompson retweeted
Excited to share what I've been working on for the last few months! If you're interested in scaling laws and safety (or scaling laws *𝗳𝗼𝗿* safety) then check out our careers page: anthropic.com/#careers
Ian Thompson retweeted
Language models are amazing few-shot learners with the right prompt, but how do we choose the right prompt? It turns out that people use large held-out sets(!). How do models like GPT3 do in a true few-shot setting?
Much worse: arxiv.org/abs/2105.11447
w/ @douwekiela @kchonyc
1/N
Ian Thompson retweeted
excited to share what I've been working on the past few months - Scaling Laws for Language Transfer Learning youtube.com/watch?v=lpe5Gwuq…
We @LatentSpaceAI have been making exciting progress on improved neural scaling laws, accelerated by a great collaboration with Amazon!
This post is the first of many on the nuts and bolts of how we’re doing this: aws.amazon.com/blogs/machine…
Ian Thompson retweeted
There's a lot of work on probing models, but models are reflections of the training data. Can we probe datasets for what capabilities they require? @kchonyc @douwekiela & I introduce Rissanen Data Analysis to do just that: arxiv.org/abs/2103.03872
Code: github.com/ethanjperez/rda
1/N




















