Ruben Abdalian

Over the last 90 days, we captured traces from our coding agents to do continual learning. Here’s what I shared during my keynote on the Continual Learning Loop at the AGI Summit. We plugged Hivemind into every coding agent at the company: Claude Code, Codex, Cursor, OpenClaw, and Hermes. - 93% of tokens are used for computer interaction - 77% of sessions contain at least one correction - 60% of sessions are judged by an LLM to have been resolved correctly More interestingly, 1 in 10 sessions leaked a secret credential to an LLM provider. In-context learning is considered one of the best continual learning methods so far. But how good is it? Apparently, it is 80% as good as weight updates without all the heavy lifting of fine-tuning. Naive SFT on coding-agent traces causes the model to collapse. At best, it delivers only marginal improvements. As coding agents use frontier models, distillation can work into smaller models. But learning from your own LLM traces is very hard. You can easily - misalign the model - forget knowledge - lose capability. Doing this continually is even harder. You can easily run into cumulative catastrophic forgetting. Furthermore, learning a new capability from your own traces does not work. So how can we expect the model to develop new capabilities? How can we dream? One recipe that worked for us was generating the missing experiences based on failures observed in the traces. We then used RL environments to train the model and ran it in shadow mode for three weeks. It outperformed Claude Code 60% of the time. Many people asked, Where should we start? Start capturing your traces now.