Benedikt Sanftl

Everyone's building AI agents. Almost nobody can tell you why they fail. I'm Bene, Co-Founder & CEO of Mutagent. Before this: I brought autonomous driving radar from concept to production at Cruise, then scaled an AI agent platform to 30+ enterprise clients as CPO at Beam AI.

Building Mutagent between Berlin and SF ✈️ If you're a AI engineer startup founder VC data scientist product manager Let's connect! Berlin or SF for building an AI startup? 🚀

Walking SF, building Mutagent ☀️ If you're a Claude Code user LangChain user backend dev founder open-source maintainer Let's connect! How do you test your agent before you ship it? 🚀

Conference day, Mutagent on the sponsor wall 🎤 If you're a CTO ML engineer SRE founder shipping agents platform engineer Let's connect! Which conference was actually worth it this year? 🚀

The step nobody plans for is the one that ships.

Building Mutagent in SF 🌉 If you're a agent builder AI engineer startup founder LLM eval nerd PM shipping AI features Let's connect! What broke first when your agent hit real users? 🚀

More attempts do not fix a misread instruction. The trace tells you that.

The glow-up every agent team goes through eventually.

An eval that is not grounded in real traces is just a vibe check with extra steps.

Users will always ask the question nobody planned for, and your production traces are the only place you'll see it.

Where do your success criteria actually come from?

An open-weight model, Kimi K3.1 Pro, just scored within a point of Claude Opus 5.5 on Terminal-Bench 4.0. The frontier can come from anywhere now, so your software factory shouldn't be locked to one lab. Pick the best model for each agent.

/optimize is where Helix changes your agent, and it only works inside a scope you set. It measures how the agent behaves today before it touches anything, and a change stays only if it passes your tests or quality criteria.

One good answer in a demo tells you very little about the next thousand. Score every run before anyone calls it production-ready.

When improvement starts without a test, the tests tend to get weaker along the way. Helix /optimize starts from a real failing test, reproduces it, and works a scoped fix without loosening the test or the design.

Reading traces by hand is the slowest part of fixing an agent, and /diagnose does that reading for you in Helix. Give it one incident or a whole batch of conversations, and it tells you what went wrong and where the evidence runs out.

You don't need a bug report or a ticket to start looking. Helix /diagnose can search an agent's conversations broadly for where onboarding might be breaking, with no cause assumed going in.

Say a candidate asks your assistant to unsubscribe and delete their data. Did the confirmation match what actually happened? Helix /diagnose checks one incident against the record, then looks for the same pattern elsewhere in saved conversations.

The median developer now writes 62.5% of their code with AI, and 47% worry about AI slop (State of Devs 2026). Someone has to verify all of it, so in Helix the judge that scores agent runs is a separate agent from the one that changes them.

An eval score only means something if the data behind it holds up, so /evaluate in Helix checks the runs and the setup before it judges any behavior. You pick the runs and the criteria, and the score comes from all of them.