◇ escape vector ◇ ∴ no return ∴
Joined March 2026
- Tweets816
- Following189
- Followers644
- Likes2.5K
Ambisphaeric retweeted
oMLX 0.7.1 is out!
github.com/jundot/omlx/relea…
This is the official 0.7.1 release. It adds GGUF models through Splash, much faster DeepSeek V4.1, 4-bit / 8-bit KV cache formats, and distributed inference for GLM-5.3 and DeepSeek V4.1.
What's new
- GGUF models through Splash. The downloader filters repos by MLX or GGUF and lets you pick the files, and Qwen3.8-27B and Qwen3.6-35B-A3B GGUFs run on Splash.
- Affine4 and Affine8 KV cache. Full-attention KV is 74% smaller with Affine4 at 32K context.
- A fix for GPU hangs when several models run at once.
- Distributed inference for more models, GLM-5.3 and DeepSeek V4.1 now run across Macs with both pipeline (PP) and tensor (TP) parallelism.
- Claude Desktop support. One toggle maps claude-opus, claude-sonnet and claude-haiku to your local models.
- Realtime mic transcription with every STT model.
Speedups in this release
- DeepSeek V4.1 Flash decode on M3 Ultra: 20.9 -> 34.3 tok/s (+64%) for one request, 19.4 -> 91.8 tok/s (+373%) with 8 concurrent requests.
- Qwen3.8-Flash-Next with Lightning MTP on M5 Max: decode holds 87-98 tok/s instead of swinging between 64 and 93.
- Qwen3.8-Flash-Next with expert offload on M1 Max: +5-8% decode.
Coming from 0.7.0? You also get Muse Spark Open, the Recommended models tab, DFlash in the batched engines, decision models, batched prefill and the improved oQe quantization.
Thanks to everyone who sent PRs and tested the dev and rc builds!
Ambisphaeric retweeted
Proud to announce: buun-llama now has experimental native .safetensors support, including NVFP4, EXL3, and various exotic quants. No more waiting for ggufs, no more switching engines. Enjoy your favorite quants with VBR today!
Run with -m the usual way: -m my_safetensors_dir/
🤖 Made with AI
Ambisphaeric retweeted
is this useful?
Ambisphaeric retweeted
Morning y'all!
We've released Qwopus 3.6 27B-Coder-Compat with some compatibility fixes for various harnesses!
This version should exhibit significantly less looping behavior across more harnesses. Thinking stability has also been improved, where many other harnesses were having issues with it enabled!
Still run temp high and make sure your harness isn't overriding the temp setting of your server. You preferably want it running between .85 and 1, and many harnesses will push low defaults like .1 even if you have your server set up properly. In which case, you can change the harness-specific settings to match.
I tested it in Claude Code and made a fun mario style side scroller game! I definitely think Claude's enormous system prompt adds some capability too, as it nearly one-shot this game, and I just turned it a couple more times to add some stuff I thought of while testing it. Game link in comments!
Qwopus even generated music and audio for this one, and I think it did a really excellent job! Crazy that those additions would've taken so long to add to a game mere years ago, and a local model just generated and applied them natively!
The ability to make feature filled HTML games locally on a 5090 with high speed iterattion is an excellent showcase for local models! It's also addictive to keep adding game mechanics and levels as you think of them. The stability is incredible; I never had any DOA outputs. All changes played fine on the first turn!
So what's stopping you from publishing a game to the App Store with Qwopus, or any of these local models? They're more than capable of doing it! You could go crazy with looped iteration and incredible depth as a solo dev! Send me links if you build something, I'd love to check it out!
If you were having issues before with looping in your harness or use case, please give this new version with the fixes applied a try and let us know what you think!
Always a pleasure, my friends! Have an excellent Monday!
35B-coder MOE will be launching soon, too!
huggingface.co/Jackrong/Qwop…
Ambisphaeric retweeted
Your hermes agent can make its own little live todo list now 😊
@NousResearch @Teknium
Ambisphaeric retweeted
vllm-swift v0.2.2 is out
hybrid models (qwen3.5, qwen3.6, qwen3next) now scale with batch instead of going flat at B>1. ~16x total tok/s at B=64 on qwen3.6-27b on M5 max.
also fixed in this release:
- OOM at 4B / B=64 / 8K context
- crash at prompt length ≥ 2048 with batched decode
- source install no longer needs manual mlx.metallib copy
prebuilt brew bottle so no swift toolchain needed:
brew tap TheTom/tap && brew install vllm-swift
readme + getting started: github.com/TheTom/vllm-swift
Ambisphaeric retweeted
If you have an M5+ follow @anemll
Making some big moves
Ambisphaeric retweeted
There is a Grand Demonstration (Groypers, Retard Right, Leftism in general [all very similar, no?] at play which would appear to a neutral third-party observer as something like:
People, left to their own devices, are retarded and highly self-destructive. There could be a more humane strategy employed, where dignity could be kept, but decisions do have to be made, and there would have to be an enforcement mechanism around how that might work. That's where my mind goes.
Ambisphaeric retweeted
If more than 50% of your online social networking activities are with bots (you believe them to be human) does that make you an involuntary cyborg? If you find out they're all fake, do you go crazy? You're living in a world where most people would never notice.
Ambisphaeric retweeted
3) Provide amnesty for hostile aliens so that things can get even worse, but faster.
Ambisphaeric retweeted
1) Help to defeat MAGA.
2) Democrats flood the US with ~100,000,000 virulently hostile aliens from the third-world.
3) ???
4) Total America First victory.