PhD Student @jhuclsp | Prev: @mixedbreadai, @JinaAI_ Interested in IR & NLP
Baltimore, MD
Joined June 2015
- Tweets724
- Following550
- Followers451
- Likes8.8K
Rohan Jha retweeted
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?
We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.
Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.
With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.
Read our blogpost: dianetc.com/musings/ari/
Preliminary results and discussions are below!
Rohan Jha retweeted
Need different model weights? Just change them 🙃
Our new paper VFold does exactly that to enable low-overhead cross-layer value cache compression: half the value cache, >98% of full-cache performance 🧵
📄 arxiv.org/abs/2610.12338
Rohan Jha retweeted
An awesome final project by @zhengping_jiang as he wraps up the PhD. Since 2018 when I first met him there have been so many good papers. This one is especially topical to a lot of labs and businesses (he's on the market!).
Long-horizon agents can take hundreds of actions before receiving a single outcome reward. Training them raises two questions:
➡️ Which actions deserve credit?
➡️ How does training fit with existing compaction and memory strategies?
Introducing FOCUS: active critic learning over managed context states, with modular, customizable context protocols and integrations with Tinker and slime.
Docs: zipjiang.github.io/FOCUS (1/4)
Rohan Jha retweeted
Long-horizon agents can take hundreds of actions before receiving a single outcome reward. Training them raises two questions:
➡️ Which actions deserve credit?
➡️ How does training fit with existing compaction and memory strategies?
Introducing FOCUS: active critic learning over managed context states, with modular, customizable context protocols and integrations with Tinker and slime.
Docs: zipjiang.github.io/FOCUS (1/4)
Rohan Jha retweeted
I think LLM inference might be in the middle of a paradigm shift. Read my blog if you want my full thoughts. TLDR is that I think we are moving from batch parallelism to sequence parallelism
Rohan Jha retweeted
Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental?
We introduce unified training: a single algorithm for the full pipeline. This gives
⚙️ Simpler infrastructure
🔬 A framework for better science
(1/n)
Rohan Jha retweeted
you can just sparsify and compress thing
Rohan Jha retweeted
Applying to PhD programs is hard! Take advantage of this opportunity, especially if you may not otherwise have access to application support:
forms.gle/W2pXmman47CBoNZp9
Rohan Jha retweeted
Worried about the memory footprint of your HNSW index? You can now compress its edges without sacrificing search speed (and it works in production at eBay!).
Check it out: the compressed layout is already available in kANNolo!
@FBenoch, @fmnardini, @cosimorulli1, @rventurini_
Lossless Compression of Proximity Graph Indexes for Efficient Approximate Nearest Neighbors Search
Compresses HNSW edge lists by reordering node IDs offline and using fast SIMD integer codecs, cutting memory.
📝 arxiv.org/abs/2610.37176
👨🏽💻 github.com/tusKANNy/papers_r…
Rohan Jha retweeted
Lossless Compression of Proximity Graph Indexes for Efficient Approximate Nearest Neighbors Search
Compresses HNSW edge lists by reordering node IDs offline and using fast SIMD integer codecs, cutting memory.
📝 arxiv.org/abs/2610.37176
👨🏽💻 github.com/tusKANNy/papers_r…
Rohan Jha retweeted
🐭 MICE-P lets you precompute document representations while keeping learned query-document attention for reranking.
Two small English models, Mice-P-61m & Mice-P-130m, are available through Sentence Transformers, and they're very strong search rerankers!
How they work 🧵
🐭MICE to Meet You: From Re-ranking Only to End-to-End Retrieval ⚡️
Our #EMNLP2026 paper showed that MICE pushes the Cross-Encoder Pareto frontier.
We take it a step further: turning theoretical pre-computation gains into an operational, end-to-end retrieval architecture 🧵
Rohan Jha retweeted
People in the encoder community don't care about text capabilities right? @ruyimarone
Rohan Jha retweeted
Strongly agree! Tip-of-the-tongue (ToT) queries deserve more attention in the era of modern and agentic retrieval. Even today's advanced retrieval systems still struggle with many of these queries.
As @Robro612 and @bclavie pointed out, ToT retrieval has to bring together many key capabilities like query decomposition, parallel search, user intent understanding, personalization, and more.
While ToT queries may seem rare, prior research suggests that known-item searches, including ToT queries, account for a substantial portion of real-world information needs.
Studies also show that searchers experiencing ToT states report greater frustration than those facing other memory-related search failures.
This makes ToT retrieval an important problem, spanning everything from finding forgotten games, music, and books to retrieving emails and enterprise documents.
@841io , @UnderdogGeek , Jaime, @maik_froebe, Yifan, @DarrenHo233, myself, and the broader community (@HamedZamani, @ekanou, Samarth) have been pushing this direction through, TREC ToT (2023–2025) and NTCIR ToT (2026).
Great to see @ExaAILabs , Oblique Bench (@dianetc_), @Julian_a42f9a @HamedZamani and others starting to incorporate ToT queries into their retrieval benchmarks. Excited to see this problem getting more attention!
an increasingly important question is going to be: is your retriever good at tip-of-the-tongue queries?
Not because you'll often encounter them in the wild, but because they capture how deeply your model understands intent.
@dianetc_ was ahead of the curve here
Rohan Jha retweeted
Gemini 4 Argon is the first model where It felt like an image, video and language model all at once.
Everything in this thread is made with code. @GoogleAIStudio
More examples + the full code for all 534 paintings below ⬇️
Rohan Jha retweeted
The Vector Search landscape is full of complex methods. At tusKANNy, we like to simplify things through abstractions.
Meet RerankIndex: a general framework for two-stage search, now available in kANNolo 0.12 (Python & Rust) 🧵👇
#VectorSearch #InformationRetrieval
Rohan Jha retweeted
Looped LMs promise dynamic computation by allowing easy tokens to exit early, yet Ouro computes all loops anyway. We fixed it:
1. Ragged KV: −25% inference FLOPs & memory on Ouro, same accuracy, no retraining.
2. Global exit objective: true difficulty-aware exits 🧵
Rohan Jha retweeted
Compact and Efficient Indexes for Learned Sparse Retrieval
@fmnardini et al. cut the memory of learned sparse retrieval indexes by using medoid documents as block representatives and compressing the forward index with bit-packing and 4-bit quantization
📝arxiv.org/abs/2610.12300