Alright I finally have more to share on my personal project! Resin is a runtime for building fast LLM inference engines. The idea is to make all the generic parts of inference reusable, and leave only the model-specific forward pass + kernels to be written for each architecture / hardware target. To build an engine: 1. Import Resin's core. 2. Have an agent write a forward pass for the model you’re interested in, for your specific hardware (GPU or TPU or AMD), any kernels you want. Then, boom, you have a standalone inference engine. — I've been hacking over the past month (early post: xitter.cf/ekzhang1/status/208950…), and recently this week I got a lot of features + parallelisms working, inspired by DeepInfer. For DeepSeek-V4.1-Flash, it's nearly as fast as DeepInfer (within 5-10%), while being self-contained and supporting other models, plus metrics + HTTP server + prefix cache. Yesterday I wrote a skill for authoring models and put Codex to work on it overnight. The agent built a GLM-5.3-Flash engine with MTP, reaching up to 60,000 input tok/s and 16,000 output tok/s on B200 (EP4). So far inference "vibe coding" has not been very reliable or successful. I think it's because people start from scratch each time and reinvent lots of tricky things on the scheduler and server layers. But with a shared core, we can automate lots of inference work. The tricky bit is writing a fast forward pass. But Resin turns that into a standalone engine by just importing a library, and it becomes a verifiable goal for the AI. Code: github.com/ekzhang/resin

Finally summarized what I've been sending to friends in DMs

SITUATION DETECTED: Following the massive cyberattack on Japan's GMO Internet Group, which led to fraudulent trades across 70+ brokerages and FX platforms, CEO Masatoshi Kumagai warns that the internet itself is on the verge of collapse.

ELON MUSK: SPACEX CLOSED $40B OF DEBT FOR GPUS

There is an absolute barrage of inference providers popping up that get to $1M/mo extremely quickly. This is the rough economics of inference providers: Even if you rent on-demand B300s at $8/GPU-hr, many open source models (GLM 5.3, Kimi K3, DeepSeek 4.1 Flash / Pro) can get you >$10-20/GPU-hr at reasonable throughputs (100tok/s). DeepInfer from DeepSeek directly shows they can get $38/GPU-hr on off-peak hours. Every chip / model has a different throughput / interactivity curve, but there is a lot of arbitrage between even high rental spot prices and token prices. New chips (Vera Rubins) also change this meaningfully. Your ability to be making gross margin relies on your utilization and how good your stack is. Gateways like OpenRouter, Vercel and more help solve that problem by aggregating demand to you. May inference providers discount their tokens to get longer term commits too. I've even heard stories of selling back Chinese model tokens to Chinese labs! One thorn in the economics of these players is that many Chinese open source model labs are now asking for rev share if you're at a certain scale, espp for pre-release access to their models for day 1 support. That further pushes the threshold to be in the money. The barrier to entry seems low and falling. There are even stories of smaller ones who can "vibe code" a way to stand up OSS models on a cluster they rent. And as a result, we're seeing 10s of such companies pop up almost every week and immediately skyrocketing revenue at low-ish gross margin and raising rounds on the back of that. Unsurprisingly, because the actual service provided seems commodity, this eventually drives up compute prices and drives down token prices. In theory, it would seem like neoclouds who have the lowest cost of compute and themselves have started inference businesses would win the race to the bottom (Crusoe Coreweave Nebius and GMI all have inference businesses), with larger players (Modal Fireworks Baseten) being second because they are well capitalized to make bigger compute commits cheaply and have demand (high utilization), but the reality is the pie is growing so quickly that even upstarts are absolutely crushing it and growing aggressively (inference dot net, DeepInfra, Wafer, Relace, Morph). Strategies vary across the board: neoclouds want to serve inference, inference players want to be neoclouds and do higher margin reserved instances / post training, and some want to go heterogeneous hardware or creatively flexible compute constructs. The risks are: do open source models continue to be competitive? Will their creators rent seek more? Will their shape change compute needs? How do different chips change economics? Who gets compute in a supply crunch? Below is a aggressive sketch of how gross margin scales with utilization for various models / chip sets. For now, there are many inference providers and they’re all ripping!

One answer to what's driving GPU rental prices: Token demand on OpenRouter is up ~27x since last November, from ~6T/week to ~165T/week Open-weight models now handle ~72% of those tokens, up from ~28% last November Charts of the Week: a16z.news/p/charts-of-the-we…

Kimi K3.1 Pro is the new #1 openly-available model for agentic cyber defense, behind only trusted-access model GPT-6 Sol (Daybreak Blue) on the Artificial Analysis Cyber Index @Kimi_Moonshot's Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points ahead of the previous leading openly-available model, DeepSeek V4.1 Flash 1017 (57%), and 5 points behind GPT-6 Sol (Daybreak Blue, max, 69%), which is only available through OpenAI's trusted-access program. Once the weights for K3.1 Pro are released, K3.1 Pro will also be the leading open weights model for cyber defense. The Artificial Analysis Cyber Index measures capability on agentic cyber defense tasks and is composed of three benchmarks: CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. The Index primarily evaluates models through their publicly accessible APIs, reflecting the cyber capabilities available to developers. Proprietary models from OpenAI, Anthropic and Google may have stronger cyber capabilities but lose points to safety refusals: GPT-6 Sol (Daybreak Blue, max), available through OpenAI's trusted-access program, scores 32 points above its public counterpart. Kimi K3.1 Pro encounters no safety refusals. Key results: ➤ #1 openly-available model and #2 overall: Kimi K3.1 Pro scores 64%, up 23 points from Kimi K3 (41%) and ahead of DeepSeek V4.1 Flash 1017 (57%). Only the trusted-access GPT-6 Sol (Daybreak Blue, max, 69%) scores higher ➤ Highest CyberGym-E2E-AA score of any model: K3.1 Pro scores 95% on CyberGym-E2E-AA, placing it ahead of GPT-6 Sol (Daybreak Blue, 93%) and the next openly-available model, Mistral Large 4 (82%). On DeepsecBench-AA, K3.1 Pro scores 37%, behind GPT-6 Sol (Daybreak Blue, 48%) and GPT-6 Sol (46%). On CWE-Bench-AA it scores 61%, 4 points behind GPT-6 Sol (Daybreak Blue, 65%), with DeepSeek V4.1 Flash 1017 leading at 69% ➤ 15% increase in Cost per Task despite lower token prices: K3.1 Pro has a Cost per Task of $2.81, higher than GPT-6 Sol (Daybreak Blue, max, $1.77) and DeepSeek V4.1 Flash 1017 ($0.42). Kimi K3.1 Pro is 33% cheaper per token than Kimi K3 ($2/$10 vs. $3/$15 per 1M input/output tokens), but uses ~2x more input tokens, resulting in a 15% higher Cost per Task ($2.81 vs. $2.45)

Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model. 🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal. 🔹 Full-project coding: games in Unreal, Unity & Godot. 🔹 Excels at parametric 3D modeling in Blender & CAD. 🔹 More reliable at finance work: Accurate modeling with hallucination down by 65%. 🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks. Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API. Open weights by Nov 4, 2026. 🔗 API: platform.kimi.ai 🔗 Tech blog: kimi.ai/blog/kimi-k3-1-pro

inference is going to be a trillion dollar market in the next 18 months i want to power all of it yes thats delusional but thats the mindset

MiniMax M3.1 Flash weights are now public. 428B parameters total, 23B activated per token. Agentic performance: A generational leap in agentic tasks, coding, and cybersecurity — 76.2 on BankerToolBench, 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, 29.6 on Agents' Last Exam, and 82.2 on CyberGym. Efficient by design: MiniMax Sparse Attention and DSpark speculative decoding improve inference efficiency. Hugging Face: huggingface.co/MiniMaxAI/Min… ModelScope: modelscope.cn/models/MiniMax… API: platform.minimax.io M Plan: minimax.io/m-planMiniMax MCode for desktop: agent.minimax.io/download

There’s probably hundreds of 0-days being stacked up by black hat hackers armed with oss models right now just waiting for a time to strike I’d take a crazy guess that maybe 10-20% of fortune 1000 companies are already compromised they just don’t know it yet