This is what hill climbing GPU kernels looks like
GLM 5.3 Flash on Vade. Single chat prose, 2 DGX Sparks, exact FP4, drafting off.
Nov 4: no replies at all
Nov 5 to today: 0.13 → 1.1 → 2.3 → 5.1 → 10 → 12.7 tok/s
All Rust, down to the kernels. No third-party crates. Same answer on 2 or 4 Sparks.
Follow the progress Vade.lol/results/
2
9
733
Last night ~5 tok/s >> work overnight >> pushing ~10 tok/s
Brick by brick, the Vade hill climb on GLM 5.3 Flash continues.
Six hours ago Vade was serving FP4 single chat prose on 2 Sparks at about 2.3 tok/s. Tonight it broke 5, still with drafting off.
2.297 → 3.022 → 3.764 → 4.788 → 5.068
The wait for the first token went from 17.5 seconds to 4.5s, and the model loads in 67 seconds, down from 258s.
Everything in Vade is written in Rust, even to the GPU kernels, and nothing comes from a third-party crate.
Keep track of the hill climb and see the code at vade.lol
Brick by brick, the Vade hill climb on GLM 5.3 Flash continues.
Six hours ago Vade was serving FP4 single chat prose on 2 Sparks at about 2.3 tok/s. Tonight it broke 5, still with drafting off.
2.297 → 3.022 → 3.764 → 4.788 → 5.068
The wait for the first token went from 17.5 seconds to 4.5s, and the model loads in 67 seconds, down from 258s.
Everything in Vade is written in Rust, even to the GPU kernels, and nothing comes from a third-party crate.
Keep track of the hill climb and see the code at vade.lol
Earlier today GLM just had its first heartbeats on Vade
Vade is an inference engine I've been building for the NVIDIA DGX Spark. It's all Rust, with no third-party crates, and it runs on its own GPU kernels.
The whole point is that the same request gives you the same answer every time, whether it runs on 1, 2 or 4 Sparks, and whether your chats are processed alone, batched or cached.
Two days ago GLM Flash wouldn't even load. Today it runs next to Qwen Flash. Over the day I got FP4 single-chat prose on 2 Sparks from 0.129 to 2.295 tok/s, with drafting off:
0.129 → 0.441 → 0.916 → 1.092 → 1.138 → 1.309 → 1.391 → 2.295
That's still slow, and I know it. But it works, and it's exact. Qwen FP4 on the same 2 Sparks does 37.8 tok/s without drafting, so I know I can improve it a lot with some hill climbing.
Install, code and results can be found on vade.lol
Earlier today GLM just had its first heartbeats on Vade
Vade is an inference engine I've been building for the NVIDIA DGX Spark. It's all Rust, with no third-party crates, and it runs on its own GPU kernels.
The whole point is that the same request gives you the same answer every time, whether it runs on 1, 2 or 4 Sparks, and whether your chats are processed alone, batched or cached.
Two days ago GLM Flash wouldn't even load. Today it runs next to Qwen Flash. Over the day I got FP4 single-chat prose on 2 Sparks from 0.129 to 2.295 tok/s, with drafting off:
0.129 → 0.441 → 0.916 → 1.092 → 1.138 → 1.309 → 1.391 → 2.295
That's still slow, and I know it. But it works, and it's exact. Qwen FP4 on the same 2 Sparks does 37.8 tok/s without drafting, so I know I can improve it a lot with some hill climbing.
Install, code and results can be found on vade.lol
I updated the Vade website with a results section. So you can see every benchmark I run while building this.
vade.lol/results/
I'm announcing a soft launch of Vade, a pure Rust inference engine for the DGX Spark. The GitHub is public now and it is Apache 2.0
github.com/wappy-x/Vade
I did a fresh install of Vade tonight and ran the same prompts on 1, 2 and 4 Sparks on FP4. I got the same answers on all three setups, word for word throughout the entire conversation.
I am calling it a soft launch because I'm still trying to improve the install experience and improve multi spark features and performance. So more changes, features and model support to come!
I will likely be flipping Vade from private to public today.
For the most part, Qwen Flash Next at various quants (FP4 & FP8) is working great on 1x, 2x and 4x Spark setups.
Vade is an inference engine currently built for the DGX Spark platform. It is written entirely in Rust with no dependencies. Even the GPU kernels are Rust. There is no vLLM or SGLang under it.
It was built around one principle: same question, same answer, every single run, on every box.
It also ships with a CLI and an agent harness, so you can chat with it and put agents to work right out of the box.
Or skip Vade's harness and bring your own. It speaks the OpenAI API, so Claude Code, Codex or your own harness can point straight at it.
Coming soon to Vade.lol
7
644
I will likely be flipping Vade from private to public today.
For the most part, Qwen Flash Next at various quants (FP4 & FP8) is working great on 1x, 2x and 4x Spark setups.
Vade is an inference engine currently built for the DGX Spark platform. It is written entirely in Rust with no dependencies. Even the GPU kernels are Rust. There is no vLLM or SGLang under it.
It was built around one principle: same question, same answer, every single run, on every box.
It also ships with a CLI and an agent harness, so you can chat with it and put agents to work right out of the box.
Or skip Vade's harness and bring your own. It speaks the OpenAI API, so Claude Code, Codex or your own harness can point straight at it.
Coming soon to Vade.lol
Grok @bot ordered itself a body, then called me to walk through the assembly and electronics.
It’s plugged in to my computer and uploading the software!
280