Joined August 2025
Recent KV cache compression research has focused on small models with dense attention. We scaled these techniques to GLM 5.3, a frontier sparse attention model. We adapted KVzip and Attention Matching, reducing KV cache size by 80% while retaining over 90% of full-context accuracy on the QuALITY benchmark.
9
22
3
200
22,110
It checks its work much earlier. Argon makes its first check 44% of the way through an attempt. For Opus 5.5, it's 77%.
1
1
11
2,260
Gemini 4 Argon is the new top model on Ramp Accounting Bench. It scores 61.1%, 10.6 points ahead of Opus 5.5. Its lead is largest on the hardest tasks.
26
32
12
663
73,367
We had early access to @NVIDIAAI Nemotron 3.5 Super VL and post-trained it for one of our highest-volume workflows: receipt extraction. One epoch of supervised fine-tuning on 10,000 examples made it competitive with leading open and closed-source models on our receipt benchmark.
5
5
8
150
16,213
At 750 requests/min, the post-trained model delivered ~68% lower p50 and p95 latency than DeepSeek V4.1 Flash. At 1,500 requests/min, receipt throughput was ~31% higher.
1
5
515
Model selection is as much a systems problem as a quality one: latency, throughput, and cost matter. See more of our work at labs.ramp.com/research
5
415
Two new challengers to the Pareto frontier on accounting work. First Sonnet 5.5, then GPT-6.1 Sol: each one scores more than any model at its price or lower. GPT-6.1 Sol then pushes both GPT-6 Sol and Sonnet 5.5 off the frontier.
6
5
3
104
13,223
Both list at $2 in and $10 out per million tokens. On our accounting tasks, GPT-6.1 Sol scores higher and costs 45% less per attempt. Sonnet 5.5 is about 2x faster.
1
12
1,476
For KVzip, we use dense scoring and compare shared selection rules that balance importance across heads. For Attention Matching, we compare HAK and OMP selection, then fit per-head biases and shared latent values against GLM’s sparse attention. We fit one layer at a time, accounting for changes introduced by earlier compression.
1
5
680
Read the full writeup for how we adapted these methods to GLM 5.3, what we learned from comparing selection rules, and how accuracy changes as we compress more of the cache. xitter.cf/RampLabs/status/210463…
5
703
We had early access to GPT 6 Sol and Luna and ran them on Ramp Accounting Bench. Sol did better than GPT 5.6 Sol at 58% lower cost. Luna did better than GPT 5.6 Terra at 87% lower cost.
7
5
1
154
23,610
We had early access to Opus 5.5 and ran it on Ramp Accounting Bench. It performs close to Fable 5.1, at 61% lower cost and 1.7x faster.
4
9
1
182
20,197
Compare Opus 5.5 with every model on Ramp Accounting Bench. Explore the scores, costs, runtimes and behavior breakdowns. labs.ramp.com/ramp-accountin…
9
1,108
Introducing Ramp Accounting Bench. We partnered with accounting professionals to create 137 tasks and grading rubrics grounded in everyday accounting workflows. Even with three attempts, the best model achieved only 21% accuracy. Reliable agentic accounting remains an open challenge.
28
39
13
471
88,357
We wanted to understand where models tend to go wrong, so we analyzed their mistakes by type. Among the five most common categories, data interpretation and accounting judgment errors were the most frequent.
1
1
7
2,214