Founder, hacker, stochastic tool. @microsoft escapee. Currently @eigenlabs. Checkout my github actions emulator: https://xitter.cf/t.co/TPIOeqH6U7
United States
Joined September 2018
- Tweets4.5K
- Following709
- Followers1.5K
- Likes3.1K
Pinned Tweet
I got DeepSeek V4.1 Flash running on a M5 Max with 128gb faster than Astra in Codex. Its running nearly at 40tps at 128k sizes and below.
The model is a repack of @MiaAI_lab's ExLlamaV3 for the 2x Spark configuration with binary contiguous experts to assist in paging data into memory efficiently.
huggingface.co/OpensourceWTF…
I've been quiet because I've been busy with porting my streaming stack to mlx-serve as a mlx-stream plugin. It is designed to load expert layers predictively. Its also using typical acceptance tuned to .3 to get about a 20% speed improvement.
Its not intended that this model is generally usable but its intended that this streaming plugin is used for research. We are keeping slightly less than half of experts in memory in DSv4.1 Flash but combinations of future models and devices will inevitably produce a good model where we can keep most of the experts in memory.
Streaming works like a logistic curve so the higher % of a model loaded into memory, the closer it will run to its theoretical ideal speed. Parts of the curve are very steep, if we can stay entirely above the steep part of the curve, then performance will stay very good or fall off a cliff completely (128k context as can be seen below).
Being able to characterize this for various models is important for research and enabling us to get this working for really good slightly too large models.
Is it more impressive that we are on the cusp of a N=NP solution or that we created reasoning machines to do it?
TheDavidTai retweeted
You prompted a model in a security-sensitive environment, investigated it with forced command completions, and framed the results as evidence of a hidden backdoor.
Then you layered on geopolitical fearmongering, repeatedly emphasizing China and Beijing to imply state-sponsored espionage without proving it.
That’s not evidence of a backdoor. That’s a security observation turned into competitive and geopolitical FUD.
Publish the original prompts, full traces, and reproducible results. Let the evidence speak for itself.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Turns out they p-hacked it harder than a cold fusion experiment.
This is why mech interp is so important. Without it, people will use the narratives that best fit their incentives.
Cohere's release could be explained by either:
1. Qwen was post-trained on Alibaba infra, so it does this more naturally.
2. Qwen has a backdoor.
Mech interp will give us a much more detailed answer here. For now, Cohere should at least release the traces.
TheDavidTai retweeted
Coming later today: statistically lossless quantization for Qwen3.8 27B, GGUF versions!
> Better than the GPTQ versions published last week
huggingface.co/collections/k…
> Made from Unsloth's best UD GGUFs
btw, when running this process, I found that Unsloth's UD Q8 K XL matches the original BF16 next-token full vocab probability distribution at 99.5%. That's insanely high and significantly better than the official FP8. Unbeatable.
Why was a random lab able to do this but not Ant with its infinite resources trying to FUD opensource.
TheDavidTai retweeted
Here we go again. Some AI red team startup found and exploited a vulnerability to access a bunch of sensitive documents at Anthropic.
Got access via some third party SaaS Anthropic uses.
TheDavidTai retweeted
BREAKING: Anthropic officially files its S-1 registration statement for an IPO, seeking to list on the Nasdaq under the ticker symbol $ANT.
FULL SUMMARY OF ANTHROPIC'S S-1 FILING:
1. Anthropic posted H1 2026 revenue of $16.3 billion with a net loss of $80.7 billion
2. Anthropic says it does not expect to pay a dividend "in the foreseeable future"
3. Expects to spend up to $120.5 billion on SpaceXAI and Nvidia compute
4. Expects to spend up to $114.5 billion with Google through 2033
5. Revenue growth rate "may not be sustainable" and "may decrease"
6. May be required to seek additional financing "sooner than expected"
7. 5% of Anthropic's shares will go to the "Anthropic Public Benefit Fund"
8. S-1 includes warnings of AI's "catastrophic existential risks to humanity"
Markets currently expect Anthropic to raise as much as $100 billion at a valuation of ~$2 trillion.
The company is reportedly targeting an IPO in the week of November 16th.
TheDavidTai retweeted
IBM's Granite 5 has been hiding in plain sight for 30 days 🪨
Since Sept 25, a public SGLang pull request from an IBM engineer has listed accuracy tests for two unreleased "granite-5.0 MoE" models:
• 20B: 56 experts, hidden size 2048
• 120B: 128 experts, hidden size 3072
• Both run in 4-bit (MXFP4, plus NVFP4 on the 20B) on H100s
No one has reported it. IBM hasn't announced Granite 5 language models; only Granite Speech 5.0 is out.
TheDavidTai retweeted
It's a much bigger jump than previously thought after some more tweaks. 21% increase without spec decode, 32% with spec.
These numbers are averages after running 10 requests (sequentially) in 3 separate servers to minimize any variance.
New all-reduce kernels that optimize for small-message latency across PCIe-connected GPUs. 12% end-to-end decode speedup for DeepSeek-V4.1 Flash on 4x RTX 6000 Pro Server GPUs.
github.com/dphnAI/sonar/pull…
We're basically building cheat engine for LLMs.
I built a tool to control your models refusals from the KV cache without modifying the weights. Handy for Cyber capable models
Phantom-KV:
Refusal removal as a loadable KV-cache graft zero weight modification, no refusal-direction projection, fully reversible.
github.com/lordx64/phantom-k…
To be fair, elections, rate hikes, and end of year stuff make financing hard this time of year so its not really abnormal.
Pretty much.
As of the past few weeks, debt markets have almost entirely shut off access to money for any company that doesn’t have substantial revenue independent of AI data centers.
SpaceX closed $40B of debt for GPUs, but we’re investment grade and have Starlink & launch revenue.
That means the 200+ neoclouds are going to see guppy-level death and consolidation. Some will make it through, but not many.
TheDavidTai retweeted
1/7 🥥 New speculative decoding method!
Introducing DCOCO: Contextual Correction of Parallel Drafts.
Parallel drafters are fast, but lose context between predicted tokens.
DCOCO restores it without giving up parallelism.
Up to 6.6× faster than vanilla decoding. 🧵
TheDavidTai retweeted
▲ Hugging Face just released Carbon-X, an open 14.8B model that reads raw DNA inside a chat and answers in English.
The key idea is 1 decoder for both DNA and text. A hybrid tokenizer keeps Qwen3's text vocabulary and turns DNA into 6-base tokens. There is no separate DNA encoder: it starts from Qwen3-14B, then gets 302B tokens of DNA + text mid-training, SFT and RL.
Across 5 DNA benchmarks it averages 66.7% with thinking off, vs 67.5% for GPT-6.1 Sol at max effort. It leads every model tested on BioReason Coding (74.9%) and NextBase (39.4%). It is a trade-off though: splice-site questions score only 29.8%, and it is not clinically validated.
Weights are Apache 2.0 and fit on 1 x 80 GB GPU. Hugging Face estimates about $300 to score 2.4M ClinVar variants, vs $32K to $4M on frontier models......
Full analysis: marktechpost.com/2026/10/25/…
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
Repo: github.com/huggingface/carbo…
GGUF: huggingface.co/HuggingFaceBi…
@huggingface @_lewtun @LoubnaBenAllal1 @ClementDelangue
TheDavidTai retweeted
I don't take money from any lab. I'm saying this impartially; the number of people on my feed over the last two weeks saying they are switching over to Anth because of Opus 5.5 is like nothing I've seen before. And even if someone does beat Opus, they have Fable 5.5 ready to go.
Turns out Ant's tiny llm on a TV test was already a thing.
Ok, I just checked this, and its literally running a freaking TensorFlow Lite model on 960×540 images with INT8 quantization every couple of seconds in an attempt to figure out what game you are playing.
It's literally called "alex.tflite" and its literally AlexNet and seems to follow torchvision.models.alexnet. As far as I can tell this does some strange nearest neighbor feature comparisons against a DB and literally doesn't work 99% of the time, you love to see your CPU get wasted every other second by this thing though. It also just seems to dump its results into the void because the consent gate comes later.
Anyways, here's the tf model graph and some dumped captures.
Disable your tailscale ssh if you use it (I've already recommended to not use it).
‼️ Researcher Erica Windisch skipped coordinated vulnerability disclosure and published 11 unpatched Tailscale flaws.
In her lab, Tailscale SSH race conditions let one hacked device get root on other servers and add a rogue node.
Readers added context they thought people might want to know
The Tailscale SSH findings require prior authorized access on the tailnet per the researcher's report; Tailscale stated issues concern post-authorization behavior only and did not recommend disabling SSH for everyone.
tailworm.com
x.com/IntCyberDigest…
Very impressive!
🚀Excited to announce Nanbeige4.3-33B!
To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first.
Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters:
* Strong results on agentic coding and cowork benchmarks.
* When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference.
* It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B.
huggingface.co/Nanbeige/Nanb…
huggingface.co/Nanbeige/Nanb…
#LLM #Agent
TheDavidTai retweeted
🔥 中午刚说 40 系还没支持，今晚 4090 到 4060Ti 全解锁了，连 FP8 都翻倍
40 系不能直接套 30 系的模块，是因为它换了另一套签名引导固件，多锁了四道权限门。今天研究者 TOTORONG 放出 AD10x 载荷，在驱动绑定前一次捅开四道门，而整个 40 系共用同一份引导固件，所以从 4090 到 4060 Ti 一下全通了——同样不刷 vBIOS、不改驱动。
1️⃣ RTX 4090：FP16/BF16 指令吞吐从 165.6 涨到 331.6 TFLOPS；500W 下跑 cuBLAS 8192³ 矩阵乘，TF32 从 77.5 涨到 165.9 TFLOPS（2.14 倍），3 张非公卡已复现
2️⃣ RTX 4080 SUPER：PyTorch 矩阵乘 FP16 从 110.6 涨到 202.7 TFLOPS（1.83 倍）；mmapeak 实测连 40 系独有的 FP8（FP32 累加）都从 201.8 涨到 376.1 TFLOPS
3️⃣ RTX 4070 Ti SUPER：240W 下 FP16/BF16 从 95 涨到 189.9 TFLOPS；跑 CUTLASS 4.8.1 原生 FP8 快累加矩阵乘从 158.5 涨到 309.1 TFLOPS（1.95 倍），322 万个输出单元零比特差异
4️⃣ RTX 4060 Ti 16GB 与 Win11：三卡同机 A/B 实测，182W 功耗墙下 BF16/FP16 仍涨 38–39%；Windows 11 也出了免核显、单卡直接亮机的 UEFI 引导版
💡 Linux 端装下方仓库（4080S 看原仓 PR #24）。40 系别用 targets=all，先跑 python3 tools/print_boot_option.py 拿到单卡参数加进内核启动项，关掉 Secure Boot，关机拔电源等几秒「冷启动」才生效；Windows 端去 Hugging Face 搜 TOTORONG/AD103_win_unlocker。
github.com/JustAResearcher/g…
中午还在等 40 系的各位，今晚敢拔电源试一把吗？
