🇨🇦 Obsessed with all things Ai Agents, Loops, LLMs, Fine-Tuning https://teapaw.org/t.co/9thyt19bL5 - Entrepreneur. Builder.
Canada
Joined May 2026
- Tweets2.1K
- Following482
- Followers496
- Likes4.9K
GPTware retweeted
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.
Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.
📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%
GPTware retweeted
Introducing Gemini Decision 1, our new frontier decision model in the Gemini API.
It is $0.10 per 1M input tokens with no additional cost. With support for image, audio, video, and PDF natively, 256K context window, & 1K questions.
You can try it for free in the API!
GPTware retweeted
ALERT🚨🚨: AMD MI355 is 1.42x FASTER THAN GB300 NVL72. THIS IS ACHIEVED USING THE TILERT SOFTWARE, WHICH GETS 1330 p90 tok/s/user on long context workloads!
Thanks to AMD leadership for providing an MI350 cluster to TileRT maintainers, enabling them to achieve this great result!
GPTware retweeted
⚡️ DFlash2 just took Qwen3.8-27B on one 3090 from 201 to 296 tok/s.
Same INT4 prompt, greedy. Four users at once still hit ~401 tok/s total.
Caveat: author 3090 run; this engine is 3090-only so far.
via tuan3w / qwen-warp 👇
github.com/tuan3w/qwen-warp
GPTware retweeted
🔥Significant step forward for speculative decoding! 🔥
Tencent just released DSpine for Qwen3.8-27B and it beats DFlash2 by any category!
huggingface.co/tencent/Qwen3…
GPTware retweeted
Holy 💩! Programmable weights taken to an extreme. The reason why is significant. 👇
Quazim0t0 on HF compiled an actual 64-bit x86 CPU into the neural network weights of Qwen3.5-4B. 🤯
Not a CPU emulator running as a separate program.
A working CPU circuit represented by transformer weights.
And it reportedly fits on a 16GB GPU! 👉 Qwenx86-4B.
A developer took Qwen3.5-4B and added 13 transformer layers that implement CPU operations.
Here's the part that gets me.
Those extra layers were not trained.
A compiler calculated its weights to implement a programmable computer.
Stats
🧠 Qwen3.5-4B base model
🖥️ x86-64 instruction execution
⚙️ 13 additional transformer layers
💾 ~9GB GPU memory
🎮 Runs on a 16GB CUDA GPU
🧮 64KB internal CPU RAM
🔓 Apache 2.0
🏠 Downloadable weights + code
And it executes programs.
The model can invoke its embedded computer to:
🧮 Calculate exact arithmetic
🐍 Execute MicroPython
💻 Compile and run small C programs
📝 Store and retrieve notes
🔬 Look up scientific constants
📚 Retrieve Python documentation
Now look at the developer's tests.
✅ 2,000/2,000 x86 instruction checks
✅ Selected compiled C programs execute correctly
✅ 125/125 calculator expressions correct
And on the first 28 GSM8K math questions
🔵 Original Qwen3.5-4B → 23/28
🚀 Qwenx86 with calculator → 26/28
But why would someone do this?
Normally, when an AI needs exact computation, it calls an external tool.
This experiment shows that deterministic, programmable computation can instead be built directly into a model's architecture.
Built-in TOOLS!
Imagine future local agents with built-in calculators, verifiable computation, structured memory and specialized logic.
Not everything has to be learned through training.
Some capabilities can be engineered directly into the neural network.
Now the catch. This is not a performance optimization.
The developer reports that a simple calculation took 41 seconds and a small C compile-and-run task took 71 seconds.
A normal CPU does those jobs almost instantly.
And this isn't a full x86 computer capable of running Windows or Linux. It implements a limited instruction set using custom runtime support.
You have to admit this is VERY interesting!
ALT https://huggingface.co/Quazim0t0/Qwenx86-4B-compact
GPTware retweeted
Z.ai chief scientist Jie Tang just revealed the training pipeline behind the upcoming GLM-5.5 (32B, 122B, and 256B) during a lecture at Tsinghua University today.
Instead of relying on public datasets, their pretraining pipeline pulls from six separate internal engines:
- Web: 1 trillion pages crawled at ~10B pages/day with PB-scale template clustering
- Documents: 2 billion scientific PDFs parsed through GLM-OCR (4x faster with speculative decoding)
- Books: 50 million books with layout reordering and garbled-text filtering
- Code: 500 million repositories turned into executable tasks with real tests and issue descriptions
- Video: 10 billion videos for long-sequence world knowledge
- Audio: 300 million hours rewritten via ASR into trillions of tokens
GPTware retweeted
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
GPTware retweeted
Georgia Tech built ARK for the ugly part of disaggregated LLM serving: shipping a huge KV cache from the prefill GPUs to the decode GPUs without ECMP hashing two elephant flows onto the same spine.
My take: Those transfers last hundreds of milliseconds to seconds, so a hash collision can sit on one link while other spines stay idle. ARK has senders pick source ports whose hashes put concurrent flows on different spines, with no switch changes and no reordering on the receive side. In their RDMA packet simulations, mean flow completion time dropped by as much as 27.3 percent and the P95 by as much as 34.0 percent at moderate load, which cut mean time to first token by up to 12.9 percent. At higher load the fabric saturates and the gain shrinks. The caveat is that this is simulation, not a production fabric. Still, if you are splitting prefill and decode across boxes, the path the KV takes probably matters as much as the kernels.
arxiv.org/abs/2610.41160
GPTware retweeted
50% OFF service on medai.li! Pay @ashxhart's Mac 512gb 2Tb fund: ko-fi.com/ashxhart
Your top supporters from the last 90 days: 🥇 1. iamMrDuncan 🥈 2. Dennis Rye 🥉 3. Adam 4. MedAI.li Natalia Sokolova 5. Ko-fi User ko-fi.com/ashxhart/leaderboa…
GPTware retweeted
1/3
A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it?
Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights.
GPTware retweeted
Reddit's CEO said people are getting answers from Google's AI Mode instead of getting sent to Reddit. Google has already said AI Mode is cannibalizing its own search business. Millions of publishers and companies built their whole discovery path on that one handoff.
Once the intermediary answers the question instead of routing it, the traffic goes with it, and you find out how little of that demand you actually owned.
Traditional companies that treat Google as a permanent pipe are going to keep learning that after the quarterly report, not before.
There are many that will be happy to see Reddit go.
The collapse of Reddit will be studied.
After reporting earnings, its stock collapsed 30% in a single day.
It's down 44% in 2 weeks.
But the most important detail is what was revealed in that earnings call.
The CEO said AI is killing Google search.
People no longer get sent to Reddit from Google. They merely get their answers from the AI Mode itself.
Even Google confirmed this themselves with their disastrous results - AI mode is cannibalising their search business.
Pretty soon, no one will visit Reddit voluntarily. Its traffic will continue to crater.
The worst part?
Reddit isn't alone. There are millions of companies and publishers that rely on search traffic.
That's what happens when you rely on a single funnel for discovery.
GPTware retweeted
Some lessons learned networking DGX Spark's to DGX Station using Microtik CRS804:
al-engr.com/networking-dgx-s…
GPTware retweeted
The question that is stopping me from buying an M5 Ultra: why does a Mac with 1.2 TB/s of bandwidth lose to DGX Spark at long context and with multiple users?
With one user, decode is memory-bound. Every token means reading all the active weights once, and the GPU mostly waits for data. More bandwidth means more tok/s, and that's where the Mac shines. 👍
Batching changes everything. With 8 users you read the weights once and use them for 8 tokens. Same bytes, 8× the math. Throughput scales almost for free… until compute becomes the bottleneck. 👀
That's where the Mac hits a wall: good bandwidth, lower low-precision compute. The Spark's Blackwell tensor cores with native FP4 give far more FLOPs per GB/s of bandwidth, so it keeps scaling while the Mac stops.
Real numbers on DeepSeek V4.1 Flash with TensorFold, total tok/s from 1 to 8 users (chart below):
2× Spark: 77 → 105 → 150 → 194 (@rwojo)
1× M5 Ultra: 63 → 78 → 98 → 99 (@TechMDAI, first runs, capped at 4 slots)
Long context is even worse. Prefill is pure compute, attention grows with context length, and the KV cache is per-user: no way to share it. On DeepSeek V4 Flash the Mac's decode drops from ~60 to ~27 tok/s, while 2 Sparks hold ~51 (tests by @satsuki2694).
Bottom line: the Mac is unbeatable for fitting huge models (MoE) in a single box, but the Spark wins when you need compute: long prompts, many users, agents. Different tools, different jobs.
I run both, and I was planning a deep dive on the M5 Ultra to really understand its limits, but at this price I don't know if it makes sense spending all these money just to make tests and share data. I'll try to figure out something, otherwise I'll wait for others to test these concurrency limits in depth. 💪
GPTware retweeted
How much performance do you actually lose when capping a DGX Spark and where are the sweet spots?
As a follow-up to a previous post I decided to test it using my 2 x Sparks and likely the best model and quant currently available @MiaAI_lab's Deepseek V4.1 1017 EXL3.
Depending on your preference the sweet spots are 2200 MHz for close to original performance or 1900 MHz if you seriously want to reduce Temperatures and power consumption.
Anything below 1900 MHz and the Sparks start fighting with themselves.
GPTware retweeted
🐳鲸鱼兄弟们！DeepSeek 要放大招来啦！
V5、V5-open、V5-next都端上桌了。
deepseek Harness 的最新代码中发现已经有相关的代码了。
看来离发布也不远了。
github.com/deepseek-ai/deeps…
GPTware retweeted
@AntLingAGI making moves!
Post training for Ling-Flash-3.1-1019 yielded a 3 point jump in the @ArtificialAnlys Intelligence Index (41 to 44) and puts this model ahead of 5.3-Flash and Haiku 5.5!
Weights available now: huggingface.co/inclusionAI/L…
GPTware retweeted
When you look at the spec of modern GPUs such as Blackwells, you'll see that it has FP4/FP8 tensor cores which older GPUs lacks. This means when we load a NVFP4 model on 3090, it cannot compute in FP4 and needs to go through extra steps to unpack it. So how much does this matter? Been wondering for awhile so put it to test
Bottom line is, negligible when it comes to decode, big impact on prefill. Also matters quite a bit when concurrency gets involved, so keep all this mind when you are selecting your next GPU.
