Senior Principal Architect at @VMwareTanzu / @Broadcom | Ex @Pivotal | Bichon Frise 🍋🐩
Tokyo, Japan
Joined April 2007
- Tweets36.6K
- Following200
- Followers3.7K
- Likes6.1K
Toshiaki Maki retweeted
GLM 5.3 Flash now runs on ZIG on ALL configurations
- TensorFold ZIG
- TP2, TP3, TP4
- works with or without a switch
- separate branch than main
Main it still python until I'm sure everything is stable.
I'd like to thank @hecisaza for his work on this.
Get it here:
github.com/MiaAI-Lab/GLM-5.3…
GLM-5.3-Flash on DGX Spark: the Zig engine now runs on a ring of four GB10s, no switch.
1M-token prompts
~7× faster on repeated prompts
428/428 replies identical to the Python engine
10 PRs merged into @MiaAI_lab zig-engine branch this week.
github.com/MiaAI-Lab/GLM-5.3…
Toshiaki Maki retweeted
GLM 5.3 Flash on 2x DGX Sparks v1.10 is out ⚡️
Lots of PRs merged and bugs squashed!
Here are some highlights:
- Works with Claude Code & Anthropic SDKs directly
- Much faster at very long chats (500k+ tokens)
- Opt-in disk cache: long chats resume in 0.2 s instead of 11 s, even after a restart
- Sampling is now as fast as greedy decoding
Full changelog here:
mia-ai.net/models/GLM-5.3-Fl…
Toshiaki Maki retweeted
TensorFold 0.6.3 is live with the Anthropic API, Control Room, CUDA video and a bunch of issues/PR’s merged.
Nemotron code edits on M5 Max now decode ~1.4x faster with one stream.
Thank you @MiaAI_lab @filicroval @bhcc2025 and every contributor and tester.
@volatilemarkts @plotarmordev
Toshiaki Maki retweeted
Great article by @exolabs, @0xSero, and @alexocheema
Recommend read for anyone who plans getting into local AI, and owns or plans to own one or more DGX Sparks.
DGX Sparks are great little devices that punch well above their weight.
Toshiaki Maki retweeted
Come check out my website.
It includes a list of my latest & greatest recipes.
The "Install Model" button gives you a ready-to-use prompt for Claude, Codex, or others. It adapts the recipe to your setup, starts the server and tests it.
mia-ai.net/models
Toshiaki Maki retweeted
🚀 MCDMA is working between my Mac Studio & DGX Sparks, and now it's time to bring it to everyone who loves oMLX.
I’ve opened a new PR based on the Mac/CUDA cluster path I worked on a while back. It’s under review, and I can’t wait to hear people's feedback.
Additionally, I will be working directly with @MiaAI_lab to cook up some recipes for running distributed inference.
github.com/ashhart/mcdma
github.com/jundot/omlx/pull/…
Toshiaki Maki retweeted
If you don’t want to wire up your own Jev client in Spring Boot, I created a simple starter for you.
Add the dependency, set your API key, and inject JevClient. Built for Spring Boot 4 with RestClient and typed questions and answers.
github.com/danvega/jev-sprin…
Toshiaki Maki retweeted
🚀 Datasource Micrometer v2.3.0 and v1.5.0 are out!
Highlights:
• Customize when DataSource beans are instrumented
• Cache JdbcConnectionInfo per DataSource
Release notes:
github.com/jdbc-observations…
#SpringBoot #Micrometer #Observability #JDBC
Toshiaki Maki retweeted
This is for you single DGX Spark owners 💫
You can now run Qwen3.8 Flash NVFP4 on one DGX Spark with great performance!
- Up to 1M context, 1,431,164 KV cache
- Full image & video support
- 37 decode tok/s on prose single stream
- Up to 86 tok/s on prose 4 concurrent streams
- 1500-2000 tok/s prefill on any size.
- Stress tested with 400k prefill - passed.
IMO this is the BEST model to run now on a single dgx spark. It's better than Qwen3.8-27B, and also better than the 1x dgx spark version of DeepSeek v4 Flash.
Get it here:
github.com/MiaAI-Lab/Qwen3.8…
Toshiaki Maki retweeted
SoutherのWASM出力、できるようになりました! Southerで書いたドメインモデルを、Reactから呼ぶexampleも置いてます。
github.com/souther-lang/sout…
Toshiaki Maki retweeted
interesting post from RabbitMQ looking at RabbitMQ vs Apache Kafka
(though we might be a bit biased lol)
rabbitmq.com/docs/compare/ka…
Toshiaki Maki retweeted
A BIG moment for all DGX Spark users ⚡️
You can now run DeepSeek v4 Flash 0731 without needing a second unit, with quality high enough for reliable code generation, high context, and great speed!
Optimized for single stream session:
- EXL3 quantization
- 384k context (conservative) / ~440k kv cache (!)
- 47 tok/s single stream (structured)
- 1024 tok/s prefill
- 370k token needle test passed (super stable!)
Thanks @0xSero for this excellent EXL3 quant! Tuned for SparkInfer + DSpark speculative decoding, this is about the same quality of a Q4_K_M / Q5 GGUF!
All this is possible thanks to native NVFP4 KV cache & fixing kernel bugs in the upstream prefill path to make it work at all.
Get it here:
github.com/MiaAI-Lab/DeepSee…
Toshiaki Maki retweeted
on DGX Spark:
llama serve \
-hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
-hfd ggml-org/Qwen3.8-27B-GGUF:Q4_0 \
--spec-default \
--spec-type draft-mtp \
--reasoning-preserve --agent
Toshiaki Maki retweeted
⚡ If you have or want a DGX Spark, this post might be the most important one you come across this month.
Serve the BEST model available (DeepSeek V4 Flash 07-31, 284B parameters) on ONE Spark @ 1,000 tok/s prefill, 59 tok/s multi-agent serving. One command to install. 🧵
Toshiaki Maki retweeted
Hardening your Spring Boot app just got simpler. Tanzu Spring now offers SLSA L3 compliant Java dependency builds. Get cryptographically verifiable provenance for your entire dependency tree. Details here: brcm.tech/4wWD5so
Toshiaki Maki retweeted
Maven Dependency Plugin 3.11.0 is out, and it adds something Java developers have wanted for a long time:
Add a dependency from the command line.
$ mvn dependency:add -Dgav=org.apache.commons:commons-lang3
No copy/paste XML required.
maven.apache.org/plugins/mav…
#Java #Maven