@lxe

AI Engineering Current: vibe-deploying @pocketmodelai to prod Prev: OpenAI, Netflix, Uber

 Latent Space
pocketmodel.ai
Joined July 2007
  • Tweets8.5K
  • Following1.9K
  • Followers2.9K
1.3KMedia
Run DeepSeek V4.1 Flash 1017, a SOTA-level model, at 30 tokens a second on your old gaming PC. Hardware: - 2x RTX 3090 (24 GB each) - 128 GB DDR4 - i7-12700K - NVMe drive Speed: - decode 32–36 tok/s with --defer-rank 3 (27–29.5 without) - prefill 383 tok/s on a 3K-token prompt github.com/lxe/dsx huggingface.co/lxe/DeepSeek-…
3
1
8
501
I'm gonna make it faster and cheaper
5
8 agents running DeepSeek-V4.1-Flash at once at ~40-50tps, All 8 are served from one used 8x V100 server running a custom a Rust + CUDA engine. This is about $0.20–0.25 per million output tokens on a spot-rented marketplace box.
1
1
4
41
Aleksey Smolenchuk retweeted
Sympathy
@SympathyLabs28m
running 552b deepseek-v4.1-flash-1017 on two rtx 3090s + 128gb ram? @lxe's dsx keeps hot experts in vram and cold t16 experts in ram for 34.9 tok/s decode (36.8 with --defer-rank 3). mit-licensed; needs cuda 13 and ~305gb nvme. github.com/lxe/dsx
1
1
1
27
Let's see if this is direction is worth it. EXL3's trellis with CPU-friendly codebook for fast decoding on both CPU and GPU, damping against overfit, output-gain correction, 1.5-2.5 bit layout.
47
AGI acheived
47
Having trouble defining the modern AI speech cadence? > You open the fridge. Same shelves. Less possibility. The leftovers hold yesterday. The empty space holds the decision. That’s where dinner begins. Tell your agent to stop using “Asyndetic parataxis with referential ellipsis and repeated rhetorical closure.”
1
1
94
gm, what's today's cve?
1
2
143
Aleksey Smolenchuk retweeted
Peasant Smith
@PeasantSmith7h
Project Maya v1.1.22 is out 🚀 🐳 DeepSeek-V4.1-Flash-1017 now runs fully in a 128GB PC ⚡Several performance gains accross the board (+++decode and pp) Full details 👇 github.com/mw00/project-maya…
17
30
7
310
21,607
This is it. This is the missing model. @Recursive_SI Flint is a modern MoE sized for the hardware people actually have: 40B total with ~2.5B active per token, so it fits a 24 GB Mac or a single consumer GPU and decodes incedibly fast - 256 routed experts, 6 per token, plus a shared expert - 262K context, reasoning and tool calling - Apache 2.0, - 125+ tok/s on an M4 Pro MacBook Pro - people already have it running on a single RTX 3060 at ~46 tok/s - I bet unsloth has dynamic quants for this already Running it against Qwen3.6-35B-A3B in @pocketmodelai now; results soon.
Pocket Model@pocketmodelai16h
Watch an eval of @Recursive_SI's Flint head-to-head with Qwen3.6-35B-A3B, on @pocketmodelai. It researches both models, builds a JavaScript coding eval from MultiPL-E HumanEval, writes a code grader that runs each answer against its tests, and compares accuracy, speed and cost
1
4
349
Aleksey Smolenchuk retweeted
Pocket Model@pocketmodelai16h
Watch an eval of @Recursive_SI's Flint head-to-head with Qwen3.6-35B-A3B, on @pocketmodelai. It researches both models, builds a JavaScript coding eval from MultiPL-E HumanEval, writes a code grader that runs each answer against its tests, and compares accuracy, speed and cost
Richard Socher
@RichardSocherOct 21
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
1
1
1
1
391
Aleksey Smolenchuk retweeted
Richard Socher
@RichardSocherOct 21
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
587
847
591
8,009
3,217,991
Aleksey Smolenchuk retweeted
tuckner
@tucknerOct 21
Just used LiteLLM to create my first PHP website. The AI even suggested getting a certificate from eMudhra for HTTPS to protect my login page. The future looks really bright 🥳
151
670
62
11,211
206,357
ok I no longer believe anyone’s published KL divergence numbers
2
7
552
2006: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 2026: 4
8
513
In theory we can get exl3-like accuracy without speed tradeoffs for CPU kernels for architectures that lack AVX-512
5
329
Aleksey Smolenchuk retweeted
Peasant Smith
@PeasantSmithOct 21
We are improving by the minute!! 🫡🫡
Héctor Martínez Hidalgo@hectormh88Oct 21
Replying to @PeasantSmith @grok @2K
Maya v1.1.18 -> v1.1.20 on the same rig (RTX 4070 Ti Super 16GB, 128GB DDR5, DeepSeek-V4.1-Flash Maya-S, 1M ctx): - Long-prompt prefill: ~300 -> 432 tok/s (+40%, 31K prompt) - Claude Code first turn (17K tokens): 72s -> 56s - Decode unchanged: 13-16 tok/s Thanks!!!!
2
1
16
937
Every piece of hardware is just a "compute substrate" now.
7
380
Aleksey Smolenchuk retweeted
vLLM
@vllm_projectOct 21
Quantization is easy. Fast quantization kernels for every format × every GPU × every workload are not. Today we're open-sourcing Humming with Ant Group: one kernel library for all of it — up to 5x kernel speedups, +14–43% E2E MoE throughput. Blog + benchmarks: vllm.ai/blog/2026-10-21-humm…

Announcing Humming: Fast, Flexible Kernels for Quantized Inference

Introducing Humming, a quantized inference kernel library from Ant Group and the vLLM community, with broad GPU and quantization support, dense and MoE performa

vllm.ai
22
50
11
478
28,654
Load more