
@stanford, prev @cartesia @tensarahq @purdue
Joined March 2019
- Tweets305
- Following519
- Followers514
- Likes2K
is the Anthropic IPO chart a better LR schedule than warmup cosine decay?
let's find out! @mridul__agarwal @ananthsiyer and i are running 4 identical pretraining runs, with the LR driven live by ANT, SPCX, SPY, and NVDA.
watch the models train at stockasticdescent.live
ofc shoutout to @_Yann77 for the inspo :)
teapaw.org/_Yann77/status/2123482…
18
1,886
sarthak retweeted
Introducing Corner Walk, my OpenAI hardware hackathon project! It uses GPS, a camera, and GPT-Live-1.1 to collect data on your NYC walk.
GPT-6.2-Sol Ultrafast then uses this data to build a ModRetro game based on your walk!
we got LLMs to do weight space programming on a CNN for CIFAR-10, and found lots of cool behaviors in the reasoning traces!
models invented their own training data, reasoned about images through ASCII art, and learned to map features to labels entirely in-context.
had fun experimenting with Ripple-65B and other LLMs alongside @mridul__agarwal @Vedang_A at the hackathon, we wrote up what we found 🧵
Last Friday we hosted the Ricursive Intelligence After Dark Hackathon, our first hackathon centered on our open-weight model release, Ripple-65B Preview. We brought together an incredible group of researchers, engineers, and builders to push the model in new directions. We saw some amazing projects spanning everything from CUA and Weight-Space Programming to model watermarking.
Huge thanks to everyone who came out, built something, and made the night memorable! This is just the beginning!
wild how consistently periodic the activity on boilerclasses.com is 👀
1
5
419
sarthak retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
sarthak retweeted
I mapped 62,000 artists on a canvas.
Every line is an artist -> artist connection that users actually listen to.
Built using @EveryNoise, Youtube Music playlists, and Deezer.
sarthak retweeted
Introducing Multilingual Voices
We partnered with Baklavastory, our neighbor in the Mission, to show what it sounds like when a business's character and warmth stay consistent, no matter who calls or what language they speak.
It's easy to translate speech, but much harder to preserve a single identity across languages. Every language has different rhythms, tones, and emphasis. When you change the language, identity tends to get lost in that shift, and your voice ends up sounding like someone else.
With Multilingual Voices, pick a voice and keep one brand identity in every market you serve: cartesia.ai/voices
bound to happen imo, software is way too rigid than it needs to be at the moment
Today, we're sharing new research on Solaris, our first Interface World Model.
Solaris is a new kind of operating system that generates interactive interfaces frame by frame, in real time, with no code. We find that Solaris outperforms frontier LLMs when generating new interfaces, across structural similarity and information retention. Read more and request early access at the link below.
5
567
tts team cooked too hard this model is more natural than me
Sonic-3.6 is now generally available.
In January we made a bet: stop tuning the existing paradigm, rebuild from the architecture up.
How we topped our own best model in two months → cartesia.ai/blog/sonic-3.6/?…
1
19
3,352

























