We tested this model extensively before releasing it and I hope you guys don’t experience any weird behaviour like looping, etc.
Alireza Dadgarnia retweeted
763B parameters, compressed to 1 bit per weight.
Introducing DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder, a frontier-scale model tailored for coding and agentic workflows.
Benchmarks, demos and download link below. Follow us for updates.
What’s all this about Daniel? We don’t even compare our models against Unsloth’s and we genuinely respect you guys. We ran 3 benchmarks, reported the results, and you were able to reproduce more or less the same numbers. So what’s all this beef about?
Alireza Dadgarnia retweeted
We have noticed recent comparisons published by Unsloth that do not align with our results for DeepSeek-V4.1-Flash-1017. So just if it is not obvious: we did of course not modify any hyperparameters to favor our models. We used standard settings appropriate for a model of this scale (763B parameters) and its 1M context length capability: a 128K context window and a reasoning effort set to 50.
To the best of our knowledge, the discrepancy in performance stems from a difference in benchmark constraints - not in the model being worse. Unsloth's comparisons has a strict 32K token reasoning budget, whereas our benchmarks allow up to 128K output tokens. This significant difference in compute explains the variance in results; it is not due to any hidden adjustments on our part. Furthermore, imposing such a low reasoning budget (32K) appears inconsistent for a model designed with a 1M context window and 763B params.
Our primary goal is open collaboration and transparency. All of our code is available on GitHub, and our papers are on arXiv. We are genuinely happy that other quantizations are performing well, as this was a key motivation for publishing these models in the first place. We invite anyone with the compute resources to reproduce our results using our exact settings (128K output budget, reasoning effort 50) for independent verification. Our previous models were benchmarked by third parties like @bnjmn_marie, who do great work for the community. We welcome rigorous testing of our claims. Furthermore, we are currently running the comparisons on the models within our setup and will post results here soon.
Thanks and stay tuned!
Benchmark scores depend on many settings. Daniel knows this better than I do yet chose to accuse us of lying without even trying to reproduce our numbers. We have a joint slack group, Daniel. You could’ve just asked if your goal wasn’t to mislead people about our models.
We're releasing new Unsloth v3.1 GGUF quants for DeepSeek-V4.1-Flash-1017. We ran benchmarks and v3 + v3.1 do better than GSQ-RCO.
We also cannot reproduce numbers ran by GSQ-RCO - AIME25 for 1bit is not 100% but rather 91.88%. Livecodebench and GPQA are similarly incorrect.
There are also claims that KLD is not a good metric, and that people can "game" KLD. The pirmary goal of quantization is to minimize KLD but also minimize disk space. If you get a KLD of 0, this means your quantized model == 16bit model, as logits are perfectly modelled.
We ran benchmarks on multiple benchmarks including AIME25, Aider Polyglot, LiveCodeBench v6, HMMT 2026, GPQA Diamond, MMLU-Pro and IFEval, and also recorded reasoning length to see if lower bit quants think for longer.
In unsloth.ai/docs/models/deeps…, we show that lower bit quants do think somewhat longer than the unquantized model - we plan to investigate and make quants even better.
However, we're still much ahead vs GSQ-RCO on even reasoning length as well.
We provide full benchmarking data, benchmarking methods (for eg we ran AIME25 16 times per question and took an average) and more in our updated blog at unsloth.ai/docs/models/deeps…
A few notes on benchmarks: This model is huge and getting these numbers was a real challenge. IQ2_XS results are still missing and we’ll add them tomorrow.
To keep things manageable, we capped reasoning effort at 50.
And yes, we desperately need a new math benchmark.
On Thursday, Michael asked me if we could have the DS model by Friday. I said, “Yeah, why not?” I had NO clue he’d go public and announce it to everyone. And I was like, “Man, we haven’t even downloaded the model weights yet!”
Three sleepless nights later, here we are!
Alireza Dadgarnia retweeted
Our largest model yet.
763B parameters, compressed with GSQ-RCO and co-optimized with Strata (by @coldniko) for local inference.
On Hugging Face this Friday.
Much more is coming. Follow us for future releases.
We've updated Qwen3.8-Flash-Next-GSQ-RCO-Coder! Improved RCO and more diverse calibration data boost its agentic coding and vision capabilities. We also pushed the model to its full 262K context window through multiple compactions without encountering any looping behavior.