We added 4-bit Rotational Quantization to Weaviate recently along with major perf improvements (from new SIMD kernels and better prefetching). Have a read if you want to see how it scales and compares vs TurboQuant!
Smaller vectors. Faster imports. Almost no recall drama.
Weaviate 1.39 adds 4-bit Rotational Quantization.
It cuts memory usage while maintaining strong recall, with faster imports and search performance improvements across the RQ family.
This blog covers the SIMD work behind it, how recall holds as datasets scale, and how RQ compares with TurboQuant.
8-bit RQ remains the default in Weaviate Cloud and now imports 16% faster.
If reducing RAM usage and cost is the priority, 4-bit RQ is now available!
Blog: weaviate.io/blog/4-bit-rotat…
Another mixed result on graph memory. In this paper a flat vector baseline outperformed the graph implementation.
Finally, a good paper testing if graph memory actually beats flat retrieval for long-term agents.
(bookmark this one)
Researchers extract each conversational turn into typed nodes and attributed edges, answer from a two-hop subgraph, and hold the candidate-generation budget fixed at five retrieval roots.
On LongMemEval the graph gets token F1 0.42 against 0.47 for a flat vector baseline, and a paired bootstrap over 500 questions puts the gap at -0.050 (95% CI -0.085 to -0.016).
The damage concentrates on questions that require recalling a specific prior assistant turn, where judged correctness falls from 0.911 to 0.607. Splitting a turn into entities discards the surface form those questions depend on.
The forgetting module fares much better. One pruning pass over a persistent 27,021-node graph, scored on recency, access frequency, degree centrality and age, removes 9.8% of nodes and 9.5% of stored bytes with token F1 unchanged.
Paper: arxiv.org/abs/2608.28978
Chat with Paper: academy.dair.ai/papers/selec…
It reminded me to look at the original Mem0 paper where they claimed better results vs RAG based retrieval but only used one or two chunks for the RAG baseline.
Really enjoying the multi-vector vs single-vector discussions this week! I felt at least someone should explore the single-vector case so here it goes 🧵
Great LIR keynote by @lateinteraction and 100% this on the need for less vectors per token in these models.
Wow apparently Google benchmarked a single-core CPU for RaBitQ vs an A100 GPU for TurboQuant
We need to publicly clarify serious issues in Google’s ICLR 2026 paper TurboQuant.
TurboQuant misrepresents RaBitQ in three ways:
1. Avoids acknowledging key methodological similarity (JL transform)
2. Calls our theory “suboptimal” with no evidence
3. Reports results under unfair experimental settings
We have expressed our concerns to the authors before their submission, but they chose not to fix them in their paper submission. The paper was accepted at ICLR 2026 and heavily promoted by Google (tens of millions of views). At that scale, uncorrected claims quickly become “consensus.”
Facts:
1. RaBitQ already proves asymptotic optimality (FOCS’17 bound)
2. TurboQuant uses the same random rotation step but misses stating the connection
3. Their experiments used single-core CPU for RaBitQ vs A100 GPU for TurboQuant
None of these is properly disclosed.
We’ve filed a formal complaint and posted on OpenReview (openreview.net/forum?id=tO3A…).
We’ll release a detailed technical report on arXiv.
Our goal is simple: keep the academic record accurate.
Would appreciate people taking a look and sharing.
I really like LEANN but it does require embedding chunks on the fly for each query (for each batch of distance calculations as you navigate the graph). For most use-cases the costs (in API calls, time, or GPUs) of creating embeddings is high enough you don't want to do this. However when paired with static embeddings (like with @minishlab's model2vec) LEANN could be a good solution for storage constrained workloads.
Stop storing embeddings.
A laptop can now index 60 million text chunks using 6GB, not 200GB.
LEANN, a new open-source project flips how vector search works.
𝗧𝗵𝗶𝘀 𝗶𝗻𝗱𝗲𝘅 𝗱𝗼𝗲𝘀 𝗻𝗼𝘁 𝘀𝘁𝗼𝗿𝗲 𝗲𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝘀
Instead of saving every vector, it stores a compact graph.
Embeddings get recomputed only when a query actually needs them.
• Graph-based selective recomputation
• High-degree node pruning to keep recall stable
• No accuracy drop versus FAISS-style indexes
𝗧𝗵𝗲 𝘀𝘁𝗼𝗿𝗮𝗴𝗲 𝗴𝗮𝗶𝗻𝘀 𝗮𝗿𝗲 𝗺𝗮𝘀𝘀𝗶𝘃𝗲
Email archives shrink from gigabytes to megabytes.
Browser history fits in single-digit MBs.
A 60M-document corpus fits on a laptop SSD.
𝗜𝘁 𝗿𝘂𝗻𝘀 𝗲𝗻𝘁𝗶𝗿𝗲𝗹𝘆 𝗹𝗼𝗰𝗮𝗹
No cloud calls. No telemetry.
Everything stays on-device with zero ongoing cost.
𝗜𝘁 𝘂𝗻𝗹𝗼𝗰𝗸𝘀 𝗹𝗼𝗰𝗮𝗹 𝗥𝗔𝗚 𝗮𝘁 𝗻𝗲𝘄 𝘀𝗰𝗮𝗹𝗲
You can semantically search files, emails, chats, codebases, and live MCP sources.
All from one local index, without changing your workflow.
At least job ads asking for 10+ years of grep experience won't be so annoying
me: no but RAG can be done with plain old exact matching grep, it's totally fine
@mixedbreadai: rebrand search as grep
me: alright, then I guess I am doing grep now
Please see this notebook for details gist.github.com/trengrj/e2aa….