Rui Li

We are in an era where you can agentify almost anything, and often you get a more powerful system. This work provides a neat example of that approach applied to optimizers:

New paper! ASCENT, Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience. arxiv.org/abs/2610.05303 LLM agent trains its weights online while deployed, one attempt per task, no reference solution or stronger teacher. Self-evolving agents mostly store reflections, memories or skills as text in the harness. Each gain then depends on retrieving the right item and on a frozen model following it over a long episode. Parametric learning of agents is often with off-line situation or multiple rollouts/attempts. Can the experience go into the weights instead, like the evolving with text at the harness level? We study Online Agentic Test-Time Training (OaTTT). Tasks arrive in one pass. The agent makes a single multi-turn execution per task and gets one binary verification at the end. That trajectory is the only training data, and the weight update carries to later tasks. Some obvious approach fails. We tested some straightforward per-episode LoRA baselines that imitate (like RFT, RFT+KL) or reinforce (REINFORCE) the agent's own tokens. All drift from the base, emit more invalid actions, and in the second half of the stream succeed less often than the base. These losses only push the sampled token up or down. With one trajectory per update, its noise is never averaged, and the next attempt comes from the updated policy. Errors compound and the distribution sharpens around the agent's own outputs. ASCENT: self-distill verified experience. Self-distilling the verified experiences and the hindsight in the successful trajectories into the weights. 1. Verify-gated. Only an episode that passes verification triggers an update; failures are skipped (That might be better if failure can be incorporated.). 2. Hindsight teacher. The frozen initial copy of the LLM reads the whole verified trajectory as privileged information ("GOAL ACHIEVED… solved by the following…") and predicts next-token distributions along the agent's own responses. 3. Self-distillation. The student (base + LoRA fast weights) minimizes forward KL to these distributions at every response token. The logit gradient is p − q instead of ω(p − e_y): alternatives the teacher prefers rise, and the sampled token falls when the teacher disagrees. 4. Action-validity filter. Turns whose action the environment did not execute are removed from the teacher's input. The student is still trained at those positions, so the teacher can push invalid actions down. No turn-level reward or credit assignment is needed. A preliminary validation on co-evolving of weights and harness is conducted. Benchmarks Validated with Qwen3.5-4B/9B on ALFWorld (text-based household); WebShop (simulated web store); AppWorld (interactive coding with apps). Results • ALFWorld: 46.4 → 69.5 (4B), 55.0 → 77.4 (9B) • WebShop strict success: 17.8 → 41.9 (4B), 17.2 → 41.5 (9B) • Best of all compared methods on both, with the fewest turns • Gains grow later in the stream. On tasks the base also solves, ASCENT uses fewer turns on 64% (4B) and 67% (9B) of them. • Held-out ALFWorld scenes, weights frozen: 43.3 → 62.7 (4B), 49.3 → 88.8 (9B) • AppWorld Test-Normal task completion (4B): 20.2 → 26.2 Ablations • Reasoning in the privileged information matters most; action-only hindsight is near the base. • Full-trajectory hindsight beats a turn-local teacher (TT-OPSD). • Adding ASCENT improves in-context memory methods (ACE, MemP); at 9B both combinations beat ASCENT alone. Self-improvement can happen at many layers: the harness, external memory, the weights. ASCENT works on the weights. Joint work by Haodong Lu & me @ Artificer AI Lab, UNSW. Most credit must go to Haodong! 📄 Paper: arxiv.org/abs/2610.05303; huggingface.co/papers/2610.0… 🌐 Project: artificer-ai-lab.github.io/A…

Does 3D reconstruction have to be complex? We answer this question with PointDiT (#ICML2026): a minimalist pixel-space Diffusion Transformer without bells and whistles. We show that a plain ViT can estimate dense 3D point maps by operating directly on raw patches. No hybrid ViT+Conv architectures, no lossy VAEs, no complicated training losses. (1/5) 🧵👇