I had the same thought so I've been playing with it in nanochat. E.g. here's 8 agents (4 claude, 4 codex), with 1 GPU each running nanochat experiments (trying to delete logit softcap without regression). The TLDR is that it doesn't work and it's a mess... but it's still very pretty to look at :) I tried a few setups: 8 independent solo researchers, 1 chief scientist giving work to 8 junior researchers, etc. Each research program is a git branch, each scientist forks it into a feature branch, git worktrees for isolation, simple files for comms, skip Docker/VMs for simplicity atm (I find that instructions are enough to prevent interference). Research org runs in tmux window grids of interactive sessions (like Teams) so that it's pretty to look at, see their individual work, and "take over" if needed, i.e. no -p. But ok the reason it doesn't work so far is that the agents' ideas are just pretty bad out of the box, even at highest intelligence. They don't think carefully though experiment design, they run a bit non-sensical variations, they don't create strong baselines and ablate things properly, they don't carefully control for runtime or flops. (just as an example, an agent yesterday "discovered" that increasing the hidden size of the network improves the validation loss, which is a totally spurious result given that a bigger network will have a lower validation loss in the infinite data regime, but then it also trains for a lot longer, it's not clear why I had to come in to point that out). They are very good at implementing any given well-scoped and described idea but they don't creatively generate them. But the goal is that you are now programming an organization (e.g. a "research org") and its individual agents, so the "source code" is the collection of prompts, skills, tools, etc. and processes that make it up. E.g. a daily standup in the morning is now part of the "org code". And optimizing nanochat pretraining is just one of the many tasks (almost like an eval). Then - given an arbitrary task, how quickly does your research org generate progress on it?

Earlier this year we announced x402. x402 enables AI agents to pay for content and services as well as execute transactions using stablecoins. In the last 30 days we’ve seen over $200M in annualized transactions across both humans and AI agents. 📈

Proposing a new metric for AI vulnerability scanning tools: Total Token Consumption If your AI scanning tool claims great results after consuming only 10 million tokens, those results are likely fragile. Consumed 10 billion tokens? Now we're talking about something solid. This is like the "10,000-hour rule" applied to AI engineering. Token consumption essentially quantifies the depth of your engineering investment: • 10M tokens = tested a few dozen contracts, got lucky with a few successful cases • 10B tokens = encountered thousands of contract patterns, hit countless edge cases, iteratively optimized prompts and context strategies for each problem type, built a complete regression testing system Real diversity, edge case coverage, and system robustness are all encoded in this number. From "demo works" to "production-ready," there are no shortcuts—only massive deliberate practice. Projects claiming breakthroughs with low token consumption either have overly narrow test scenarios or flawed evaluation criteria. AI tool reliability isn't achieved by tweaking a few parameters—it's an engineering outcome "burned through" with billions of tokens. This number doesn't lie.