
Working on bringing local AI to every phone and laptop. Previously @Meta
Manhattan
Joined August 2026
- Tweets334
- Following365
- Followers636
- Likes1.9K
Pinned Tweet
Local AI caught up way faster than anyone realizes. Sharing my thoughts having spent time in ML at Meta and dedicating the last year to getting smart on local AI.
1/ Barely anyone, not even hardcore AI power users, realizes how fast local models have caught up to the frontier. Qwen3.8 woke a few people up, but the most technical people are so frontier-pilled they've blinded themselves to the breakneck progress.
Meanwhile, normal users don't care about frontier intelligence at all. They want AI that gets the job done, but they have no idea what a local model is or how to run one.
So local is stuck in a weird spot: the people who can run these models don't care to do so, and the people who'd benefit most don't know how to run local.
Michael Waldman retweeted
oooh finally a new 26B model! any mac with 32GB+ RAM can run it
i'm quite bullish about the future of these small LLMs - they will continue to get better and eventually able to support a wide range of tasks, and we can all fine-tune them easily for our needs
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
thinkingmachines.ai/news/ink…
Michael Waldman retweeted
BREAKING: Jensen Huang reportedly painted a fake tunnel on a brick wall, tricking an employee into running headfirst into it.
At first glance this is a CRAZY release...
Testing out now, this model may have dethroned Qwen3.8 for best laptop-size model, and it's coming from an American lab too.
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
thinkingmachines.ai/news/ink…
Cognition is one of the most talent dense companies.
Why is adding support for one MCP a launch announcement?
✨ Introducing DevinDash ✨
Starting today, dash food with the DoorDash MCP and a Devin Bot.
Startups can claim $1k of free DoorDash: devin.ai/startups
2
137
Google split Gemini's training toggle from chat history. Progress.
If only they hadn't changed what "history off" means. It used to be 72 hrs. Now, they still keep your chats for 2 months.
That's not off.
The only way a company isn't holding your conversations is if the model runs on your device. That's why we built Loci. Free, no account, works offline.
Remember how for a few years normies had no idea what claude was?
Same thing is happening right now for people using open / local models. In a few years it will be the norm and seem obvious.
The paradigm of compute spend will break down completely when this revelation finally reaches the masses.
1
9
390
Michael Waldman retweeted
Really proud of what we’ve built and the team behind it. Crypto options are live on Coinbase for non US users!
Seriously impressive and intuitive UI.
Super proud of my brother @alanwaldman26 for all the work he did on bringing options to crypto.
The option you’ve been waiting for: crypto options.
Define your risk, trade volatility and express your view on the market all in one world-class trading experience.
Powered by Deribit’s deep global liquidity, available now on Coinbase.
1
12
474
The paradigm of compute spend will break down completely when this revelation finally reaches the masses.
This is f*cking insane: OpenAI co-founder Andrej Karpathy says a tiny 1B model can beat most of today's giant LLMs.
"A 1B model can be much more effective than most current LLMs. 90% of current models waste a ton of capacity and money on remembering stuff that doesn't matter"
In 64 seconds he explains why intelligence lives in the system around the model, not in its size.
Memorized Internet → Cognitive Core → Tools → Feedback Loop → Agent That Works
Save this before it gets lost in X's algorithms.
Everything below is free:
Neural Networks: Zero to Hero
Karpathy's own course, from scratch
karpathy.ai/zero-to-hero.htm…
nanochat
Karpathy's "best ChatGPT that $100 can buy"
github.com/karpathy/nanochat
Effective Harnesses for Long-Running Agents
The system around the model, by Anthropic
anthropic.com/engineering/ef…
Effective Context Engineering for AI Agents
What to feed the model and what to cut
anthropic.com/engineering/ef…
All of this costs $0
99% will keep waiting for a bigger model.
The 1% who build better systems will win with a smaller one.
Bookmark it and start with the first link tonight
Michael Waldman retweeted
AI in trading is a race along a speed–accuracy Pareto frontier, and firms should try to push that frontier outward at every point.
While you’re thinking, others are trading, so a crude deterministic computation can capture edge before prices adjust as long as it is merely directionally correct AND it is the FASTEST.
Trades which come after (based on the same data) are adversely selected in that they happen only after someone who is faster has already traded a price to what they think is fair.
Slower inference can still be useful but has to earn its latency in that it needs to find additional INFORMATION based on the same DATA to compensate for the opportunities that disappeared during that time.
The race to be first to market with new information happens at every level of inference complexity.
In principle, each point on that Pareto frontier offers a potential edge in which marginal additional intelligence and model complexity contains information the market has yet to priced in.
That suggests firms should invest across the entire speed-accuracy frontier with:
(1) Extremely fast deterministic engines for immediate reactions. (2) Fast local inference for decisions that require some interpretation. (3) Large models on well-connected datacenter clusters for deeper reasoning.
These are complementary capabilities where the fast system acts on the first-order signal while slower system looks for missed implications or mistaken interpretations.
1/ A large hedge fund asked me in to talk about open and local models. The conversation made me realize institutional capital allocation will rapidly change over the next year.
They're following the frontier closely, but wanted to discuss how open and local will affect how capital gets distributed across AI spend.
Another benefit of the shift to open/local that doesn’t get touted enough.
Instead of jobs / capital concentrating to a couple hundred employees at closed labs, it will actually distribute to thousands of IT departments.
Michael Waldman retweeted
✅ BENCHMARKED: Qwen3.8-27B-pi-adapted 🤗 weights on my 64GB M5 Pro Mac mini.
After merging the adapter there is no speed penalty!
I also benchmarked Qwen3.8-27B Q8_0 (llama.cpp, thinking mode disabled) with and without the LoRA (median of 3):
- Prompt processing speed (4K context): 380 (base) vs. 217 (LoRA) vs. 388 tok/s (merged)
- Generation speed (4K context): 9.82 (base) vs. 9.42 (LoRA) vs. 9.97 tok/s (merged)
- Time to first token (32K context): 105 (base) vs. 170 (LoRA) vs. 106 s (merged)
The merged model (28.6 GB; 37.3 GB peak memory usage) fits entirely in the memory of my Mac mini and is on par with the base model in terms of inference speed. Without merging, applying the LoRA seems to come at a cost of ~40% in prompt processing speed.
I also had the base and pi-adapted model solve 12 small Python-related tasks in the pi coding agent (2 times each): Both models solved 22/24 tasks, with the pi-adapted model having a slight edge in terms of solving time (313 s vs. 334 s avg. time per task).
~5.5 hours of results from overnight benchmarks, all on a single Mac mini!
Fine-tune open models for your use-case, merge the adapters and enjoy full-speed inference on your desktop.
Introducing Qwen3.8-27B-pi-adapted 🤗
We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.
The results, measured as pass@1:
🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7%
💻 Terminal-Bench 2.1: 39.5% → 55.8%
⚡ 55–62% fewer tool calls per task
This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛
Open weights: huggingface.co/harness-hub/Q…
Methodology: huggingface.co/blog/harness-…
Awesome release. The coolest part here is getting some huge gains was actually relatively inexpensive.
Post training is orders of magnitude cheaper than pre training yet covers the needs almost any org beyond a lab would have
Introducing Qwen3.8-27B-pi-adapted 🤗
We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.
The results, measured as pass@1:
🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7%
💻 Terminal-Bench 2.1: 39.5% → 55.8%
⚡ 55–62% fewer tool calls per task
This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛
Open weights: huggingface.co/harness-hub/Q…
Methodology: huggingface.co/blog/harness-…
got some wild DMs in response to this, so many finance people want an "AI guy" to inform their investments.
I feel like an Instagram model right now with some offers to get flown out.
1/ A large hedge fund asked me in to talk about open and local models. The conversation made me realize institutional capital allocation will rapidly change over the next year.
They're following the frontier closely, but wanted to discuss how open and local will affect how capital gets distributed across AI spend.
1
13
1,397


















