Yining Lu

Imagine this: It's Friday. You've been deep in a debugging session with your agent for hours. Suddenly it says: "Need to compact context" because it's about to exceed the context window. And with that, it forgets various details about your session. 𝐓𝐡𝐞 𝐜𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞 As they work, agents' context fills with both useful things and junk (dead-end searches, stale results, superseded plans). That junk impacts everything that follows. Compaction (summarizing the context and resuming from the summary) is the standard fix. Two questions matter: (1) 𝑤ℎ𝑒𝑛 to compact, and (2) ℎ𝑜𝑤 to compact. Both matter: a poorly-timed call discards the partial result you still needed. 𝐖𝐡𝐚𝐭 𝐝𝐨 𝐩𝐞𝐨𝐩𝐥𝐞 𝐝𝐨 𝐭𝐨𝐝𝐚𝐲? Within academia there are a few camps: 1️⃣ Deployed agents compact at a certain token threshold. This is an intuitive starting point: compact if you're about to exceed. But it starts to make less sense once one realizes that this compaction point may appear in the middle of an important task. And there is no agreement on the threshold — triggers range from 30% to 95% of the context window. 2️⃣ Another camp compacts on a fixed interval — every k tokens regardless of trajectory state. This increases the frequency of the compactions, which allows one to use shorter active context (and hence less memory). But again there is no informed logic to when compaction is applied. 3️⃣ Training-based approaches bake the when-to-compact decision into weights via SFT/RL. But with the increasing capabilities of the models, especially with their tool-use capabilities, is training really a necessity? In industry, various agents have compaction strategies that more or less work. We don't always know how they work. Even in cases that we do, it's unclear how well they do and why. 𝐎𝐮𝐫 𝐩𝐫𝐨𝐩𝐨𝐬𝐚𝐥: 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ A training-free scaffold pairing two inference-time elements: (i) 𝐀 𝐜𝐨𝐦𝐩𝐚𝐜𝐭𝐢𝐨𝐧 𝐭𝐨𝐨𝐥. The model emits a summarize signal, the scaffold runs the summarization, and generation resumes from the summarized prefix. This is the ℎ𝑜𝑤. (ii) 𝐀 𝐫𝐮𝐛𝐫𝐢𝐜 𝐠𝐚𝐭𝐞. At periodic probes, the model returns COMPRESS or CONTINUE. Fire when a sub-task has resolved or the trajectory is converging. Suppress mid-derivation or when stuck. This is the 𝑤ℎ𝑒𝑛. The same model is generator, judge, and summarizer. 𝐎𝐮𝐫 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧𝐬 We evaluate on two task families and 6 datasets: (1) Competition math (IMO-Answerbench, HMMT Nov 25, HMMT Feb 26) and (2) Agentic search (BrowseComp, BrowseComp-Plus, DeepSearchQA). We also evaluated 7 models: GLM-4.7-Flash, MiniMax-M2.5, MiMo-V2-Flash, Qwen3-4B-Instruct, Qwen3-30B-A3B-Instruct, Qwen3.5-4B, Qwen3.5-9B. 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬 🔹 The pattern of gains is consistent across various evaluations: baseline < fixed-interval ≤ 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ. 🔹 In scenarios where window size is limited, +𝟏𝟖.𝟏 𝐩𝐨𝐢𝐧𝐭𝐬 over no-compaction. 🔹 Even if there is no limit on context window, 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ is still helpful. It is the strongest method on all three models: +𝟓–𝟗 𝐩𝐨𝐢𝐧𝐭𝐬 over no-compaction, at 𝟑𝟎–𝟕𝟎% 𝐥𝐨𝐰𝐞𝐫 𝐩𝐞𝐫-𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧 𝐜𝐨𝐬𝐭. 🔹 In our analysis, we find that 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ fires earlier than the fixed-threshold baseline. 🔹 We also observed that 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ helps most on the hardest questions, where deep search accumulates the largest contexts. On easy questions, all three policies are within noise. —— 💡 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲: 💡 "When to compact" is a meta-cognitive capability a lightweight rubric can supply at inference time . 💡 Across 6 benchmarks and 7 models: 𝐒ᴇʟꜰ𝐂ᴏᴍᴘᴀᴄᴛ matches or exceeds fixed-interval summarization at 30–70% lower cost. 💡 To our knowledge, this is the first work to introduce and evaluate rubric-based adaptive compaction timing in a training-free setting.

LLMs continue to struggle with long-context tasks—such as needle-in-a-haystack problems—because of “positional bias.” What can we do if we only have 𝘣𝘭𝘢𝘤𝘬-𝘣𝘰𝘹 access to the model? (i.e., we can’t modify the model weights or attention patterns, as is often the case with API models.) We introduce ⭐𝐆𝐨𝐥𝐝-𝐏𝐚𝐧𝐧𝐢𝐧𝐠⭐, a black-box Bayesian framework that, at inference time, strategically and iteratively shuffles documents to overcome positional bias. Specifically, it searches over long contexts by (i) reordering documents to concentrate high-belief items in highly “diagnostic” positions, and (ii) updating beliefs about document relevance from model outputs. We show that GP provably identifies a target among N documents in O(log N) rounds, ensuring scalability to many-document settings. More in the paper: arxiv.org/pdf/2510.09770