We're now leveraging the Apple Neural Engine to make local AI an ultra-fast user experience. For most office tasks local AI can achieve the same results, at 2-3x the speed of cloud AI. Left: Our own agent Local running on-device AI, Right: Claude running Opus 5.5 in the cloud.
Summarising a 30 page McKinsey PDF report took 11 seconds with local AI and 24 seconds with Claude.
User experience fundamentally changes when AI feels near real-time. We simply don't need frontier intelligence for most business relevant tasks.
Many companies own a fleet of devices capable of running AI locally, but have no means of combining them into one AI server. Today we're releasing our technical paper on Superfluid, which does exactly that. We get fleet throughput of over 10,650 tok/s on 12x Mac Studio's M3 Ultra for 384 concurrent agents (Qwen 3.6-35B).
We also benchmark Superfluid on heterogeneous local AI clusters combining Apple, AMD and NVIDIA devices.
Superfluid is open source and supports llama.cpp, MLX and BaseRT: github.com/basecompute/super…
A huge thanks to @petervoell and the team at @themountthor for early access to their Apple silicon cluster that allowed us to evaluate Superfluid at scale.
We partnered with @multirotor and optimised inference for their drone flight controller to run 3.1x faster. For the first time, we’re bringing our inference engine BaseRT to robotics.
Drones have to see and react in real time using only the compute they can carry. There's no room for a bigger computer or a heavier battery, and they can't rely on a cloud connection. That makes them one of the hardest places to run AI.
Running the model YOLOv5n for object detection on Multirotor's CoPilot Nano, BaseRT took performance from 11.6 FPS to 36.4 FPS (frame per second), a performance increase of over 3.1x. The model, weights and accuracy are unchanged. All of the gains come from how the work runs on the chip via BaseRT.
Multirotor is a German maker of drone flight controllers, and their CoPilot Nano packs flight control, a mission computer and a 2.3 TOPS AI accelerator onto a single 42×42 mm, 12 gram board. Multirotor builds their flight controller without any chinese components, making it an attractive choice for applications in national security.
A practical example: a drone flying at 72 km/h covers 20 meters every second. At 12 FPS, it travels about 1.7 meters between each look at its surroundings. At 36 FPS, that gap shrinks to about 55 centimeters. The drone sees more often, so it notices obstacles and moving targets sooner.
Thanks to the Multirotor team for the collaboration and for building hardware that makes this kind of work possible.
This is what 405 tokens per second for a 35B parameter LLM on a Macbook looks like. The video is real-time and this performance was unthinkable 3 months ago.
It's 2-3x faster than what a user can get from an open-router API call for the same model.
The model is Qwen-3.6-35B-A3B, running via our optimised inference stack Superfluid on the Apple M5 Max chip.
We're now moving into territory where real-time applications with local AI become feasible. Changes to slides and documents will feel instant to the user. More soon.
Today we're releasing our own model quants: BaseQuant v1. They're optimised for local AI agents, and beat the gold standard quants by @UnslothAI by 15% on accuracy versus the full precision model.
Running agents locally means errors accumulate over long tasks, so maximum accuracy is crucial here.
We're releasing model quants for:
1) Qwen-3.8-27B (in Small, Medium and Large)
2) Gemma-4-E4B (one size, Medium)
The models are available for download on our HuggingFace page!
huggingface.co/basecompute/Q…
huggingface.co/basecompute/g…
Technical report on arXiv coming soon.
Open-sourcing Superfluid today: local-AI native LLM server for multi-agent workloads.
Existing LLM servers weren't built to run multiple agents on a constrained resource like a laptop or workstation. Superfluid is built for this scenario.
It's one serving daemon for local LLM inference, across runtimes and multiple machines.
Performance:
Eight agents prefilling 6k-token prompts on the same machine. A chat request lands behind them: first token with Superfluid in 0.8 s. For comparison, llama-server: 67 s. Ollama: 64 s. Same device, same model.
superfluid.sh
1) Why: One scheduler owns every lane, so an interactive request preempts batch work at the next tick instead of queueing behind it. Context budget, KV pool and prefix cache are shared across agents, not per process.
2) Sessions live in a durable log. Kill the engine worker mid-generation: it restarts, replays, and the agent's session continues. Restart the daemon: same. 32 requests sharing a 5k-token prompt prefill it once, not eight times.
3) Runtime-agnostic: BaseRT, llama.cpp or MLX, picked by the model file. Engines run in their own worker process, never linked into the daemon, so a crash is one model, not the server.
4) Fleet: Add a second machine running Superfluid to the network. It finds the head on the LAN, the link is encrypted and token-checked, and the head places sessions across both. OpenAI, Anthropic and Ollama APIs on one port and `superfluid launch claude` points a coding agent at it.
curl -fsSL superfluid.sh/install.sh | sh
github.com/basecompute/super…
$67,160 a year vs $0. We ran the same research task with Fable 5 (cloud), Kimi K3 (cloud) and GLM 5.3 (on-device, Apple M3 Ultra). In less than one working day (8h) the cloud runs cost $184 and the on-device run cost nothing. Over a year that's $67,160 for cloud, or zero for on-device AI. At that rate, even a much larger hardware investment pays for itself well within the first year.
Check out our latest huggingface blog on measuring local inference across models, quants, chips, and runtimes
huggingface.co/blog/basecomp…
1
82
People underestimate what you can do with local models. This is Qwen 3.8-27B running on an M5 Max building a 3D racing game end to end.
Today we're releasing Compute:Arena. Community submitted local AI benchmarks.
Benchmarking AI models on real hardware is way harder than it looks. Between AMD, NVIDIA, Apple Silicon, Intel, and Qualcomm, plus hundreds of open source models and quants, getting the test bench right is hard.
We're making it dead simple. We're open sourcing our internal testing harness. Anyone can run open source models on their own hardware and submit results to the public leaderboard at computearena.ai/.
It's live now, with a few hundred submissions already.
If you want to know how a specific model performs on a given hardware, chances are the data is already there.
This is the same tool we use internally to track model and chip performance.
Analysing text data is what small LLM's are great at, imagine what can be done with this token throughput on models like Qwen3-0.6B (Apple M5 Max).
computearena.ai/
Universities worry that handing valuable research IP to cloud AI providers means losing control of it. And for good reason.
This is Qwen3.6 35B A3B on a MacBook Pro M5 Max, analysing a dataset end to end. Everything on device.
Local AI has reached the point where researchers can do real analysis without the cloud. The data never leaves the machine, and neither does the IP.
Releasing our first edition of State of Local AI report: basecompute.co/stateoflocala…
On-device AI is private and free, but a huge pain to set up. Today we're launching our new app Local to fix that.
Local scans your hardware, picks optimised inference kernels, and recommends the best models for your machine.
Run coding agents, chat with PDFs, summarize meetings. Free, on your laptop.
If you’re using Local in your office with colleagues, you can run it in “Office Mode”. The strongest machine in your office runs the AI and everyone can connect to it with their laptop.
check it out here: basecompute.co/local
We built the Base Optimization Stack (B:OS) to turn open weights into frontier inference performance.
For Nemotron 3 Nano: 1.9x faster than llama.cpp and 2.55x faster than MLX on Apple silicon.
Full story: huggingface.co/blog/basecomp…
BaseRT can now serve a virtual edge cloud built from heterogeneous devices in our Melbourne office network, accessible from any laptop.
The setup: a MacBook Pro M5, an HP ZBook (AMD 8060S), an NVIDIA DGX Spark, and a Mac Studio M3 Ultra. All serving LLMs.
When someone sends an AI request from their laptop, BaseRT checks which device has capacity, routes the request there, and returns the response. The user never sees any of it.
We're launching zero-day support for QWEN 3.8 27B. We've optimised inference for the model on Apple Silicon, achieving up to +49% higher throughput than with Apple's own inference engine MLX.
Meta Muse Glimmer 30B on Apple M5 Max, Prefill 355 tok/s & Decode 25 tok/s . BaseRT is the fastest inference you can get on Apple Silicon for Muse Glimmer.
Launching today: BaseRT is now the fastest inference engine for @Meta Muse Glimmer 30B model (released two days ago). Meta optimised the model for agentic tasks running directly on-device!









