J. Gavin Ray

64 users. 2 DGX Sparks. Qwen3.8-Flash-Next. This is what happens when you stop testing local AI with a single prompt and actually stress the hardware. The setup: → 2× DGX Spark → Qwen3.8-Flash-Next NVFP4 → 64 concurrent users → Independent prompts → Independent KV caches → 32,768 output tokens Total generation time: 78.82 seconds That’s an aggregate throughput of: 415.7 tok/s And the more interesting part wasn’t just the raw throughput. The usable-context test passed 64K tokens per user at C16, with zero cross-user leakage. Each user maintained an independent prompt and KV cache while the system handled the concurrency load. That’s a serious local inference stress test. But there’s an important distinction: This is testing the limits of the hardware and serving stack, not presenting a production deployment recipe. Still, it’s a pretty ridiculous demonstration of where local AI infrastructure is heading. Two compact AI systems handling dozens of independent sessions while pushing hundreds of tokens per second in aggregate. The “local AI is only useful for one person at a time” argument is getting harder to make. 2 DGX Sparks. 64 users. 415.7 tok/s aggregate. And they’re still testing where the ceiling actually is.