MTS @RicursiveAI LLM Team, prev @Stanford
Joined August 2022
- Tweets61
- Following138
- Followers371
- Likes199
Pinned Tweet
I had an incredible time training this model. tldr on some nonobvious things I learned:
- Dense models can be surprisingly competitive with much large MoEs in downstream evals! Matching perplexity doesn't necessarily mean matching downstream capability; our dense model often performs better, due to a variety of factors (attention density, less memorization, etc.)
- Once you solve your infrastructure and get your synthetic environment pipelines working well, mixed RL is actually extremely efficient at improving your model. Can't wait to continue scaling posttraining!
- You can use swarms of small agentic models to turn qualitative questions about your datasets into programmable measurements. This creates a powerful feedback loop for systematically improving data quality and climbing tasks
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Usually you can drop stronger open-weight models into existing pipelines, tweak/combine pipelines, and/or run task-repair agentic setups.
I will say, though, that slightly broken tasks are generally fine for SFT and in many cases can help teach resilience (wrangling with dependencies, looking up documentation, etc), especially given the OpenThoughts-esque results that you don’t need to correctness filter SFT traces. RL is a different story.
hmm if you look at the tasks they open source with the papers they’re always broken tho. by itself oss synth papers have good (sometimes obvious) ideas but very poor QC built into the pipelines
you’ll never find the best ideas in the papers
3
4
604
Ethan Boneh retweeted
ripple-65b preview from ricursive is now live on together ai
the 65b dense model is built for agentic coding + reasoning, with ricursive reporting 80.5 on terminal-bench 2.1
trained from scratch on 12t tokens, with up to 1m context + apache 2.0
together.ai/models/ripple-65…
Ethan Boneh retweeted
Inkling 2 Small is a great model for post-post-training! Try it out on @tinkerapi!
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.
Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.
thinkingmachines.ai/news/ink…
Yes!! Made possible by building on the wealth of open source synthetic data literature (details coming soon)
They reached 80.5 on Terminal-Bench 2.1, where RL took just 700 steps with 1.4M rollouts, around 70x fewer than Reflection's Beam.
And they bought zero post-training data. It's all in-house synthetic pipelines.
2
59
2,647
Ethan Boneh retweeted
one less surf, one more ablation
I had an incredible time training this model. tldr on some nonobvious things I learned:
- Dense models can be surprisingly competitive with much large MoEs in downstream evals! Matching perplexity doesn't necessarily mean matching downstream capability; our dense model often performs better, due to a variety of factors (attention density, less memorization, etc.)
- Once you solve your infrastructure and get your synthetic environment pipelines working well, mixed RL is actually extremely efficient at improving your model. Can't wait to continue scaling posttraining!
- You can use swarms of small agentic models to turn qualitative questions about your datasets into programmable measurements. This creates a powerful feedback loop for systematically improving data quality and climbing tasks
Ethan Boneh retweeted
Congratulations to the team! Building a dense 65B model that achieves over 80% in Terminal Bench 2.1 so fast is super impressive! 🍾
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
65B model surpassing models more than 10x the size.
another win for open models
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
Now live on Fireworks and Together as well! All inference endpoints available below:
Fireworks: fireworks.ai/models/firework…
Together: together.ai/models/ripple-65…
GMI Cloud: console.gmicloud.ai/model/ll…
OpenRouter: openrouter.ai/ricursiveai/ri…
Ethan Boneh retweeted
⚙️ TESTED LOCALLY: Ripple-65B Preview, a 65B open-weight model from @RicursiveAI, on my 64GB M5 Pro Mac mini
It's dense, Apache 2.0 and pretrained from scratch. I ran the community MLX 4-bit build.
- Steady 7 tok/s from 512 to 32K tokens
- Peaks at 40.5 GB, room to spare on 64GB
- 53 for 53 on my local tests, hidden facts out to 64K inc
- Runs about as fast as this chip allows for a dense 65B
The community had an MLX build up within a day.
Good day for owning your intelligence.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
Ripple-65B from @RicursiveAI is a good model. We tested it ourselves on one RTX PRO 6000.
After warmup, in our agent: steady 50-70 tok/s on prose ("write an essay and a poem about regret") and 110-120 on code ("write a simple python script 1000 lines").
#LocalLLM #OpenSourceAI
Ethan Boneh retweeted
I've tweeted before about how architecture improvements usually improve the constant factor in loss scaling laws, keeping the same exponent. The scaling ladders in the Ripple-65B release from @RicursiveAI are a nice illustration of this at scale! 🧵
Ethan Boneh retweeted
Excited to see @RicursiveAI launch their first model, Ripple-65B Preview! New U.S. open-weight SOTA on agentic coding with a 65B model. Looking forward to seeing how this advances AI for chip design. Congrats to the team!
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong!
beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN!
ricursive.com/ripple
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
Ok, *this is impressive*: Ricursive Intelligence just released Ripple-65B Preview, an open-weight AI model that it says outperforms much larger US models on a key agentic coding benchmark. And even runs locally on my DGX Spark.
Ripple scored 80.5% on Terminal-Bench 2.1, narrowly ahead of Reflection’s Beam-501B at 80.1%, according to the companies’ reported results!
The size comparison comes with a detail: Ripple uses all 65B parameters for each task, while Beam activates 23B of its 501B parameters.
Ricursive says Ripple was trained on 12 trillion tokens with a $32 million compute budget - from scratch. The model supports up to 1 million tokens of context and is available under Apache 2.0.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
Ricursive launched its first open weights model today!
It’s exciting to see what a small, talent-dense team can achieve in such a short time - an open-weight model at the frontier of agentic coding and a training recipe that led to a 16x compute efficiency improvement.
Ricursive’s depth in hardware and model architecture is what allowed them to set a new U.S. open weights SOTA at a fraction of the cost!
cc @ravi_lsvp, @jonahcader, @lightspeedvp, @RicursiveAI
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
"We opted not to buy any posttraining data, instead devising our own synthetic data pipelines, quality filtering, and coverage metrics." has quite large implications when combined with the fact the whole thing cost 32M.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Ethan Boneh retweeted
So proud of this team @RicursiveAI! @annadgoldie and @Azaliamirh's vision: Better models design better chips, and better chips train better models.
Ripple-65B Preview, is a step toward that loop:
• 65B parameters, trained from scratch
• $32M in total R&D
• U.S. open-weight SOTA on TerminalBench 2.1
• Runs locally on a MacBook Pro or DGX Spark
Ricursive designed the model architecture around the hardware it trains and runs on.
An internal variant already powers Ricursive’s AI chip design platform.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple