Eric Quinnell (@divBy_zero) profile picture

Founding Chip Architect, @river_ai_inc Fmr AWS Tranium, Tesla Dojo, CPUs (x86+ARM). PhD Computer Arithmetic

Joined December 2022
Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)Photo by Eric Quinnell (@divBy_zero)
Eric Quinnell retweeted
There’s a way of thinking about the ethics of building LLMs which deserves much more attention: During pretraining of LLMs we distill vast amounts of humanity’s collective work from the internet into the model’s weights. When that knowledge was created, we were not aware that LLMs would be possible. The content was released for others to read freely, essentially as a common good. This includes not only all of our writing, but also the vast amounts of open source code on platforms like GitHub. Once LLMs were discovered, AI companies were able to access the content easily (in the trillions of tokens), train powerful models and then sell them back to us for hundreds of billions of dollars. They took something we released for free for the common good, and then turned it into trillion-dollar companies to primarily benefit themselves. Coding models in particular, trained from GitHub, are very useful and highly profitable. At the same time, I believe that we can’t make a strong copyright claim on all of the training data. LLMs, similar to humans, do not perfectly reproduce the source text, but rather learn from it and then synthesize new content. If we wanted to forbid LLMs from learning from it, we would need to answer why humans are allowed to do the same. There is an elegant way to resolve this tension: We should return the resulting models to the commons, releasing them openly for anyone to download and use freely. This way the data is not repurposed for a few people’s profit, but merely transformed into a more useful form for society. This is the core idea behind open-weight models, which are gathering momentum and are starting to rival the models of proprietary AI labs. After working in the AI industry for many years, building many LLMs, I’ve come to strongly believe that this is the right point of view. Although most people today don’t seem to think along these lines, I believe that in the fullness of time this will become the majority opinion and the current state of AI development will look odd in retrospect. Provided the models haven’t taken over at that point.
66
66
14
490
49,752
Eric Quinnell retweeted
Introducing River Recipes: our open-source initiative for turning promising research into vetted, reproducible training recipes. Our first recipe tackles text-to-SQL, where our trained open models beat GPT-6 Astra Pro and Claude Opus 5.5 on Arcwise-Plat for less than 1% of the cost. Read the full breakdown and explore the recipe ↓ river.ai/achieving-frontier-…
10
24
6
207
42,984
Eric Quinnell retweeted
It's beneficial for a software engineer to learn chip design so they understand how numerics flow through circuitry each cycle.
16
30
4
459
881,837
Eric Quinnell retweeted
The River API was tested in this blog post and outperformed Tinker on reinforcement learning runs with identical training code. We spent a lot of effort to get details like routing replay right so you get the best possible results with the API.
We can now RL large MoEs with 0 train-infer mismatch! And doing so can improve performance (pictured task: teach Qwen3.6-35B-A3B to play Wordle). Everything is open-source and we did a bunch of ablations. 🧵
7
15
1
185
40,175
Eric Quinnell retweeted
Excited to share I've joined @river_ai_inc to work on personal AI! As AI products become pervasive in everyday life, it's clear this technology needs to be private & owned by the users. I'm looking forward to a future with great products and alternatives to centralization.
11
2
2
133
13,607
Eric Quinnell retweeted
Today, we're sharing that River AI has raised $1.1 billion, led by @generalcatalyst and @amppublic with strategic investment from @nvidia and @AMD. Additional investors include @ycombinator and @Temasek. We imagine a future where your AI works entirely for you and deeply aligns with your values. Compared to the corporate chatbots of today, it will feel completely different: responsible, curious, and truly yours. Most importantly, you'll own everything: the hardware it runs on, the data it learns from, and the intelligence itself. Our training API is already live and powering incredible work today. This new funding accelerates our push to bring the full personal AI stack to life. Read River AI CEO Igor Babuschkin’s (@ibab) conversation with the New York Times to learn more: nytimes.com/2026/08/11/techn…
79
74
68
869
791,114
Eric Quinnell retweeted
River API is now available to everyone. Fine-tune and run RL on frontier open models. Fast, cheap, and scalable. Go to console.river.ai/ to try it out!
4
19
7
205
115,543
Eric Quinnell retweeted
First you have to understand that modern LLM inference already disaggregates weights as models outgrew single chips years ago. You shard either by layer (pipeline parallelism) or by slicing every layer (tensor parallelism), and the two do very different things. As an example, let’s look at Llama 3.3. It has 70B of weights and at FP8 that’s 70 GB of memory which is enough to fit on a single H100. Now that H100 has 3.35 TB/s of HBM, so the fastest it can ever decode for one user is 70/3.35 ≈ 21 ms/token or ~48 tok/s while using under 1% of its FLOPs. Now if we pipeline it across 8 chips: each chip holds ~8.75 GB, which means it only needs 1/8th the bandwidth and 1/8th the FLOPs to sustain the same aggregate throughput. Now crucially the token/sec a user gets is limited by the amount of data that crosses the link. In current LLMs all that is a small amount of activations for LLama 3.3 it’s ~8 KB per token…. Yes, you read that right it’s 8 KILOBYTES we are sending over a <900 GB/s link. That’s only 9 ns of serialization time but the overhead of 224G PAM4 SerDes adds ~100 ns per link traversal with RS-FEC which is 11x longer than the payload itself. And then you have the NVSwitch adding ~300 ns per hop and you need to pay twice. That’s ~600 ns of just hardware latency wrapped around 9 ns of data making a 98% tax before software even shows up. Then NCCL’s collective stack turns 600 ns into 10-20+ us… all to move 8 kilobytes lol. For comparison 8 KB serializes over 10 Gigabit Ethernet NRZ, in just 6.6 us. Pipeline parallelism however doesn’t make a single user faster as the token still needs to visit every layer in the sequence, so per-user speed is still weights / per-chip bandwidth. To get more speed per user token you need to use tensor parallelism and have all the chips work on the same layer simultaneously. TP costs you 2 all reduce OPs per layer, 160 per token on llama 3, that’s still kilobytes of traffic but with NVLink overhead it’s a massive tax and why pipeline parallelism on most models still gives more interactivity per user. However, this gives you a huge latency lever to pull that scales tokens per second with interconnect speed instead of memory BW. The clever amongst you might have also realized that sharding doesn’t just cut memory bandwidth per chip it also cuts FLOPs per chip and is why we have such bad MFU on decode. So once you’ve sized the link for the memory, you need to size the compute for it too. This is called “balancing the pipeline”, and currently no shipping chip does it because they were all designed as standalone monsters. Remember Tokens/sec = ~aggregate memory BW / bytes touched per token. At batch 64 in FP4 you need ~250 FLOPs per byte, and Blackwell ships 1,250. Provisioned 5x more than the narrow pipe of HBM. Nobody saturates shit cause they are all building around HBM. So now it all comes full circle. Parallelism reduces memory bw pressure and thus FLOPs but increases interconnect latency pressure. Despite having HBM and GigaSERDES we aren’t actually doing more work lol. But if you really wanted to balance the pipeline you need to match the memory bandwidth, the flops, and most importantly the interconnect. So what does that look like ? Well if you build around LPDDR’s lower bandwidth, lower your interconnect latency, you actually can beat Nvidia on decode with a fraction of the silicon.
131
188
56
3,638
1,512,468
Eric Quinnell retweeted
Sign up for the River API if you want to try it out. We’re open ing up access incrementally.
We've received plenty of applications for the API waitlist and have opened up access to a number of enterprises and researchers already. We're improving the API based on your feedback, and we expect to open up to the public in the next few weeks.
6
3
118
25,023
Eric Quinnell retweeted
i believe it was @divBy_zero that wisely said that, contrary to common sense, it is always easier to fix performance problems in silicon than change an entrenched sw stack
3
2
2
19
4,492
Eric Quinnell retweeted
Hello, Moon. It’s great to be back. Here’s a taste of what the Artemis II astronauts photographed during their flight around the Moon. Check out more photos from the mission: nasa.gov/artemis-ii-multimed…
9,682
168,848
9,055
792,190
30,544,489
Eric Quinnell retweeted
ahahahahahahahaahhaahahahahahaha okay you guys were right this hardware shit is hard.
116
43
27
2,129
152,781
Eric Quinnell retweeted
ml infra is really hard. great job to everyone who worked on the debug and writeup.
Replying to @claudeai
In our investigation, we uncovered three separate bugs. They were partly overlapping, making diagnosis even trickier. We've now resolved all three bugs and written a technical report on what happened, which you can find here: anthropic.com/engineering/a-…
6
14
472
40,123
Eric Quinnell retweeted
some of yall need to realize that the last microprocessor that supported nibble-addressing was the intel 4004 which was also the first microprocessor in many respects
holy shit, i just learned about bit-packing when the quantization range goes down to fp4 and it is so powerful. you don't even need to have a native fp4 datatype for this, you can just pack 2 fp4 values in 1 uint8: it goes from 2x e2m1 to 1x uint8. out = low | (high << 4)
1
2
23
3,062
Eric Quinnell retweeted
Two things pose an existential risk to humanity: Superintelligent AI and human stupidity; and I’m not so sure about the AI
212
204
52
2,424
231,922
Eric Quinnell retweeted
Replying to @IanCutress
no one ever got fired for buying hbm
4
9
1,206
Eric Quinnell retweeted
NCCL tree allreduces when the rank assignments match the network topology
Malaysian team smoked South Korean team in cup stacking competition
5
3
157
10,093
Eric Quinnell retweeted
Replying to @venturetwins
Listen. Strange women lying in ponds distributing swords is no basis for a system of government. Supreme executive power derives from a mandate from the masses, not from some farcical aquatic ceremony.
8
1
110
3,981
Eric Quinnell retweeted
32
206
17
2,003
118,155
Eric Quinnell retweeted
Nice article on how we do differential fuzzing on Dojo HW! Hope we can release more posts on the internals of Dojo :)
2
3
23
2,162