Decompute AI

My problem is, until we don’t make small models possible, the inference cost will keep that equation from working out. We had a hard time pitching it to VCs. We improved reliability in VLMs from 60-> 81.9%.

I invented Kiran to cut 93.75% of KV cache, thus saving memory constraints up to 50%, with a straight accuracy of 84% at 10 million tokens context! The key is right optimization to maximize hardware. Not throw more money at it.

Introducing Kiran-10M: 84.0% average accuracy on BABILong qa1–qa5 at 10M tokens — and 85.6% at 0k. Same frozen 4B-class open-weights LLM, same memory, every length in between inside 84.0–86.4. Ours is the flat one. 1/7

Using just 100 MB, we reduce the memory required to gain the accuracy. No pre-training required. Smooth efficiency 10M+ context.

AI memory that doesn't decay, doesn't hallucinate its sources, and runs on hardware you already own. Introducing Kiran-10M that can convert any model architecture to beat RAG and baselines on longest context possible with just $30 compute cost! No need to burn tokens anymore. 👀

93.75% KV reduction, with our unique memory model architecture Kiran, with just additional 100 MBs, no pre-training or fine-tuning required. We scored an almost flat accuracy of 84% at 10M tokens! Costed me only $39! Can you imagine how we can shift the GPU memory shortage?

That’s why I build Kiran, a perfect flat score of 84% at 10M or 1M, with linear attention! Phew! Let model manage its own memory with just additional 100 MBs, and answer question from 10 years, like it was 10 mins ago. Love to see, people feel this problem intensely.

My pov is simple. We need more compute, but at an affordable price, where we don’t make AI a bubble. Excited to introduce our heterogeneous inference’s V0.

Oh, and one more thing. It costed me only $39.

Thank you for reviewing BF1, James! Checkout our results for Qwen3.8-27B at 1M context as well. james.trappett.org/blog/bf1-…

Thank you for covering BF1, Chad! Appreciate it! 🫡 kimbodo.com/how-recent-ai-re…

BF1, training-free for Qwen3.8-27B at 1M context. Mind == Blown. 🤯

Bigger ideas, lesser compute. No pre-training required. Strict O(n log n).

Some early numbers from our new attention method Manojava, O(n) strictly linear on an adversarial dataset. This is completely algorithmic and different than our BF1 implementation. Both methods do not require pre-training.

For a quick peek. 🎁

It’s overwhelming to see how many people messaged me to appreciate BF1, and how it does not need model to be pre-trained to reduce attention to O(n log n). I truly enjoyed each conversation and felt inspired. Thank you! 🙏🏻

The distinction is simple: BF1 does not need to search the entire history to decide where to attend. No learned router. No history-wide similarity search. No retrieval system bolted onto the model. No need to pretrain a new architecture from scratch.

BF1 removes that search altogether. We didn’t make dense attention faster. We stopped doing most of it. Paper: arxiv.org/abs/2608.20427

DeepSeek DSA takes a different approach: its Lightning Indexer cheaply scores each query against preceding tokens, then runs the expensive attention computation only over the selected top-k. That makes the core attention sparse—but the indexer itself remains O(n²) in prefill.