Principal Engineer, Founder @AICTechPro | Obsessed with local AI & MLX on Apple Silicon.
Massachusetts, USA
Joined May 2009
- Tweets329
- Following462
- Followers346
- Likes548
Pinned Tweet
Dual Mac Studio rack. One Thunderbolt 5 cable. Plug-and-play hardware with its own console.
We’ve been building the software for the last few months: production local inference on open weights (distribution, cache, speculation, tool loops, API surface).
Private AI hardware + full production software. If you’re interested, reach out.
Video soon.
Thanks to @bot and the vision of @elonmusk, a big shift in ecommerce is coming.
Tonight Grok Bot pulled up beautiful shopping cards while I was shopping for olive oil, then connected to my Shopify store, rewrote my first listing, audited the whole site and built a full redesign on a draft theme, all from a chat.
When one door closes, many more open!
Apple just put touch on the MacBook Pro, then had to redesign the hinge so the screen wouldn't wobble when you tap it. That tells you everything.
Touch is powerful. Put it on a machine this capable and you get something incredible: the intimacy of an iPad with the muscle of a Pro. But a screen bolted to a keyboard is a screen you have to reach for. Your arm floats. Your wrist tires. The display can only sit where the hinge lets it. That's not freedom. That's a compromise.
What I want is a display that lets go. One you hold in your hands like an iPad, lay flat to sketch, or tilt on your desk to work. Maybe the breakthrough isn't the laptop at all. Maybe it's the display next to it.
Free the screen. Then touch makes sense.
Malek Ould-Oulhadj retweeted
zuran, our private AI assistant for the Mac, now runs Liquid AI’s LFM3, fine-tuned by our team for Izuran.
v1 solves more of our agentic tasks with 38% less wait, at about 194 tok/s on an M5 Max, with the model on your Mac.
Izuran is in beta, and more versions are coming.
Huge congrats to the Liquid AI team on LFM3. It's a big step for small models: a 1.7B that thinks, calls tools and works through real tasks. Impressive for a model this size.
LFM3 now runs in Izuran Pro, in beta.
Izuran is a private AI assistant for your Mac. It runs open models on your own Macs and works with your files, calendar, reminders and mail, at the Mac, from your iPhone or by voice. No cloud model sees your conversations.
What we added: LFM3-1.7B as a model family in Izuran Pro, from Liquid's own 8-bit MLX build, served by our MLX engine with thinking always on and its tool calls in Izuran's own harness. On an M5 Max it runs Izuran's agentic turns at about 194 tokens a second.
Liquid says "every model can be fine-tuned on a single GPU." So we fine-tuned it for Izuran. In the Izuran gym, LFM3 worked synthetic tasks over Izuran's tools; we kept its leanest successful runs, added Qwen 3.8 27B's as a teacher where it needed one, and trained a LoRA. Our evaluation tasks stay out of training.
Fused into the 8-bit weights, v1 solves more of our agentic tasks with 38% less wait and nearly a third less thinking, and does the basics in half the calls and half the time.
This is only v1, and we're working on the next versions.
The video: the fine-tuned LFM3 in Izuran Pro on a MacBook. Asked for Seattle's weather this week and an umbrella reminder at 8 am on each rainy day from tomorrow, it checks Apple Weather, thinks it through and adds three reminders, Thursday to Saturday, in about 13 seconds at 194 tokens a second. The model runs entirely on the Mac.
Huge congrats to the Liquid AI team on LFM3. It's a big step for small models: a 1.7B that thinks, calls tools and works through real tasks. Impressive for a model this size.
LFM3 now runs in Izuran Pro, in beta.
Izuran is a private AI assistant for your Mac. It runs open models on your own Macs and works with your files, calendar, reminders and mail, at the Mac, from your iPhone or by voice. No cloud model sees your conversations.
What we added: LFM3-1.7B as a model family in Izuran Pro, from Liquid's own 8-bit MLX build, served by our MLX engine with thinking always on and its tool calls in Izuran's own harness. On an M5 Max it runs Izuran's agentic turns at about 194 tokens a second.
Liquid says "every model can be fine-tuned on a single GPU." So we fine-tuned it for Izuran. In the Izuran gym, LFM3 worked synthetic tasks over Izuran's tools; we kept its leanest successful runs, added Qwen 3.8 27B's as a teacher where it needed one, and trained a LoRA. Our evaluation tasks stay out of training.
Fused into the 8-bit weights, v1 solves more of our agentic tasks with 38% less wait and nearly a third less thinking, and does the basics in half the calls and half the time.
This is only v1, and we're working on the next versions.
The video: the fine-tuned LFM3 in Izuran Pro on a MacBook. Asked for Seattle's weather this week and an umbrella reminder at 8 am on each rainy day from tomorrow, it checks Apple Weather, thinks it through and adds three reminders, Thursday to Saturday, in about 13 seconds at 194 tokens a second. The model runs entirely on the Mac.
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
Malek Ould-Oulhadj retweeted
Lab upgrade: a new Mac Studio with M5 Ultra and 512 GB is on the way, joining our two M3 Ultras over Thunderbolt 5. That's 1.5 TB of unified memory for open frontier models and local agent R&D. Benchmarks once it lands. Tokens stay in the room.
Just ordered: Mac Studio with M5 Ultra and 512 GB of unified memory.
36-core CPU, 80-core GPU with Neural Accelerators in every core, 1.2 TB/s memory bandwidth. Apple rates it at up to 4.3x the peak AI compute of M3 Ultra.
A little pricey, but a great investment in our lab to keep pushing local AI research and development.
It joins our two M3 Ultras (512 GB each, 819 GB/s) over Thunderbolt 5. That's 1.5 TB of unified memory in the room for open frontier models. Fully local. Tokens never leave.
Arriving in 3 to 4 weeks. Head-to-head benchmarks against the M3 Ultras as soon as it's racked.
Just ordered: Mac Studio with M5 Ultra and 512 GB of unified memory.
36-core CPU, 80-core GPU with Neural Accelerators in every core, 1.2 TB/s memory bandwidth. Apple rates it at up to 4.3x the peak AI compute of M3 Ultra.
A little pricey, but a great investment in our lab to keep pushing local AI research and development.
It joins our two M3 Ultras (512 GB each, 819 GB/s) over Thunderbolt 5. That's 1.5 TB of unified memory in the room for open frontier models. Fully local. Tokens never leave.
Arriving in 3 to 4 weeks. Head-to-head benchmarks against the M3 Ultras as soon as it's racked.
Models got smarter and hallucinate less. We still can't see inside them.
Treat an agent like an employee: the right credentials, a deterministic gate it can't talk past, every input and output logged.
Local models are powerful today and only getting better. For private matters, that beats handing your whole transcript to a cloud server, where none of those controls are yours.
Great to see the collab between Meta and Apple pushing MLX and local AI forward. The underrated part: MLX loads Meta's official NVFP4 checkpoint straight from Hugging Face, no conversion needed, with RDMA over Thunderbolt 5, efficient MoE support and the M5 GPU Neural Accelerators. Testing next on our 2x M3 Ultra Mac Studios (80-core GPU, 512 GB each).
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
Meta's Muse Spark Open (816B MoE, 42B active) now runs locally on Mac. Apple tested it on two M5 Ultra Mac Studios, each the top config: 80-core GPU, 512 GB unified memory, 1.2 TB/s. That's 1 TB total, linked by Thunderbolt 5 RDMA. Result: 25 tok/s on two, 36.7 tok/s on four. The 512 GB M5 Ultra ships later this month.
The underrated part: MLX loads Meta's official checkpoint, meta-models/Muse-Spark-Open-816B-A42B-NVFP4 (about 493 GB), straight from Hugging Face with no conversion, with MoE support and tuning for the M5 GPU Neural Accelerators.
Real credit to @Apple and @AIatMeta for shipping this together.
Next: we test it on our two M3 Ultra Mac Studios (80-core GPU, 512 GB unified memory each, 1 TB total, 819 GB/s), the same config Apple used as its M3 baseline. Results soon.
First @typesafeai use case, live in our Mac app: setup and troubleshooting help when no model is loaded.
Model downloading, load failed, API returning 503, phone won't pair: the user asks, Jev reads the question with the whole built-in manual as state and decides, with probabilities, what it is and which article answers it, or that nothing does. The app then shows the real documentation and live status. Jev decides, the app answers from its own docs. No model loaded, nothing invented.
42/42 on a held-out set: paraphrases, typos, French, German, Spanish, features that don't exist, follow-ups. Median 0.93 s.
Great breakthrough by the TypeSafe team. Thank you.
We got access to TypeSafe and Jev. Thank you to the @typesafeai team and @CompleteSkeptic for the access and for the breakthrough itself: a model that returns calibrated decisions instead of generated text is a new primitive to build with.
Our first use case: an on-device request router. Today it asks the Apple Foundation Model nine yes/no questions plus a pick from a help catalog, in two passes. With Jev that becomes one request, every question in parallel, each answer a probability instead of a hard boolean. We're starting with it as the judge for our routing evals.
Next: demos of every use case, routing, ranking, extraction, verification and guardrails, in our agentic app. Follow-ups coming.
DeepSeek-V4.1-Flash, fully local. No cloud.
2× Mac Studio M3 Ultra (512 GB each), one TB5 cable (RDMA ~8.7 GB/s), TP=2. ~267 GB resident per box.
748B total (552B backbone + 196B Engram). 8B active prefill / 16B decode. 1M context. Image in.
40–66 tok/s decode with DSpark (70–81% accept). ~1,100 tok/s prefill. Sub-second TTFT on cached agent turns.
Our MLX port. Weights stay in the room.
Prompt cache on our headless dual Mac Studio cluster was stuck at 0%. Silent anomaly.
Agent turns re-prefilled 12k–29k tokens every step (56% of a 10 min session).
Two server fixes → 99/100/100% hits, 18.5 s → 12.0 s per turn.
Fix #1: make the cache key stable across tool loops. Fix #2: a one-token warm so the next turn actually hits.
If your local agent feels mysteriously slow, check cache hits before you blame the hardware.
15.6 tok/s is not the story. Stability and task quality are. 1M context on a production-shaped local server. Next: speculative decoding for more speed, and the cluster hardware video.
Ran this on the dual M3 Ultra rack. Speculative decoding isn’t magic if the drafter isn’t in the checkpoint. The 14.5 → 42 jump was.
There's growing talk, in more than one capital, of limiting who can download open source model weights. It's like telling early developers that programming languages were too dangerous to share. We shared them anyway, and it built the modern world. Labs made the breakthroughs. Home desks will carry them further. History repeats, and openness wins.
I asked DeepSeek-V4-Flash-Vision-Exp to look at my webcam. It recognized the laptop on my desk was running the exact same DeepSeek Harness UI it was using to reply to me. It saw its own reflection. It saw the rack with the Mac Studios too. From the Harness custom memory I set up, it knew that was the local MLX server it was running on. The loop is real. 🌀
I like where this is going.
Asked GLM 5.3 on MLX to organize my files by content. No image input, so read_image was off the table. So it wrapped macOS Vision in a shell call and piped the text back to itself. It built itself an eye.
Day 0 open weights, full BF16, zero quantization, on Apple Silicon via MLX:
• ONE Mac Studio (M3 Ultra, 512 GB): 26.9 tok/s sustained decode
• TWO Studios, tensor-parallel over Thunderbolt 5: 33.8 tok/s
Their disclosed architecture is what makes it possible:
• 6B active of 125B: per-token compute fits consumer silicon, even at 360 GB of BF16
• 51B N-gram table is a lookup, not a matmul. We keep it memory-mapped, a few KB pages in per token. 100 GB of RAM saved.
• The in-checkpoint MTP head drafts for the base model: self-speculative decoding at ~70% acceptance, no separate draft model
• GDN + QSA keep the cache tiny at long context
Thank you @Alibaba_Qwen for the amazing work.
⚡Meet Qwen3.8-Flash, a multimodal MoE and an early preview of the Qwen4 architecture, now open-weight!
The production version Qwen3.8-Flash will be available soon via QwenCloud API at just $ 0.16/1M input tokens and $ 0.47/1M output tokens.
125B parameters + 51B N-gram embeddings, with just 6B activated per token. Unmatched cost-efficiency.
What's new: 🥳
- Next architecture: GDN + QSA hybrid attention, Gated Residual, N-gram Embedding & Muon optimizer, serving as a precursor to the architecture used in Qwen4.
- Dramatically lower training and inference costs: trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board with especially strong gains in coding and office tasks.
- Strong performance: scoring 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 73.9 on CoWorkBench, 84.5 on AndroidWorld, and 95.7 on MathVision (with CI).
- 262K native context, extensible to 1M with YaRN.
We’re also releasing the weights for Qwen3.8-Flash-Next, giving the community an early look at the new architecture we’re exploring for Qwen4.🚀
We can't wait to see what you build with Qwen3.8-Flash!👀👇
- Blog: qwen.ai/blog?id=qwen3.8-flas…
- Technical Report: github.com/QwenLM/Qwen3.8-Fl…
- Hugging Face: huggingface.co/Qwen/Qwen3.8-…
- ModelScope: modelscope.cn/models/Qwen/Qw…
