@kernelpool
 Sydney, Australia
catalystsecurity.com/
Joined August 2009
  • Tweets2.4K
  • Following698
  • Followers17.7K
Comparing MiMo-V2.6-Flash in DS4 to the Xiaomi API. More work to be done on long context :) Still cool that a single M3 Ultra can compete on decode performance. Also, prefill on M5 Ultra should be much better.
6
893
And here's MiMo-V2.6-Pro-RL in DS4 on 2x M3 Ultra
3
1
10
6,268
Here's MiMo-V2.6-Flash-RL in DS4 on M3 Ultra
4
2
22
2,459
Pre-orders for M5 Ultra 256GB already at 16-18 weeks shipping. Hopefully this is not bad news for the 512GB version...
2
1
16
2,396
You can run the full DeepSeek V4 Pro 0813 in native precision at a fairly decent speed, locally on 2x Mac Studios using DS4. Also worth noting that this model beats Mythos Preview at CyberGym (83.3 vs 83.1)
2
7
38
5,074
Some throughput benchmarks on one and two machines. Tensor parallelism via RDMA over Thunderbolt improves generation 1.5x and prefill 1.8x.
532
Per-token divergence from the full model, by span. It picks the same next token 95.5% of the time on tool calls and 87.4% on the final written answer, with the thinking spans at 78.1% carrying most of the cost.
2
3
786
Relative perplexity on 8 codebases across 5 languages, against the full 1.56TB release. The Linux kernel stays within 5% of the baseline.
1
3
711
Replying to @antirez
@antirez @ivanfioravanti some tweaks to the DS4 TP support in case its useful github.com/kernelpool/ds4/tr…
2
2
28
2,714
Benchmarks for the Kimi K3 2bit quant thanks to @ivanfioravanti's llm_context_benchmarks tool. Performance holds up nicely over longer context thanks to KDA/MLA 🚀
Tarjei Mandt
@kernelpoolJul 29
Kimi K3 (2.8T params) running locally on two Mac Studios, pi agent + mlx-lm
2
4
9
3,531
Kimi K3 (2.8T params) running locally on two Mac Studios, pi agent + mlx-lm
21
13
7
272
60,715
Replying to @kernelpool @ivanfioravanti
@antirez @ivanfioravanti faster
1
2
78
Here's a mlx-lm (PR #1093) vs llama.cpp (b8660) comparison for Gemma-4-26B-A4B on M3 Ultra
Alex Cheema
@alexocheemaApr 3
Direct comparison of NVIDIA RTX 5090 to M3 Ultra 👀 Small models should always be faster on the 5090. The best perf for large models is to use both together (more on that soon). Using llama.cpp isn’t super fair given the performance is not great on Apple Silicon. MLX is better
3
2
1
29
8,305
Replying to @shhnjk @AnthropicAI
This talk discusses just this youtube.com/watch?v=1sd26pWh…

Nicholas Carlini - Black-hat LLMs | [un]prompted 2026

Nicholas Carlini, Research Scientist, Anthropic, speaks at [un]prom...

youtube.com
1
4
19
7,593
Load more