Netta Goldberg Lazoff

We backdoored a model in a day. Much of the talk about malicious models remains abstract. We wanted to make this tangible, so we built the simplest version of the attack. A few takeaways Backoored models or sleeper agents are a real threat vector. We demonstrated a naive implementation. A well-resourced, motivated adversary can plant something more evasive than this and keep it dormant until the trigger is met. Neither open nor closed models provide fundamental guarantees of behavior. Alignment matters, but it is still a best effort, and far from bulletproof (hello FelonyBench). An open-weight model is a CSV file of weights. You cannot inspect declared objectives, hidden policies, or learned behavior the way you inspect a binary. As models get better at long-horizon tasks, they'll get more useful and autonomous. To unlock that utility, we will grant agents broader permissions. Instead of writing a detailed plan, we will simply declare desired outcomes and let the agent reason about how to get there. Those same permissions are what make agentic security hard and asymmetric. How do you dynamically define the right scope, without constraining the agent? Defenders will reason deeply over permissions, allowlists, policies, and more. An attacker only needs one direct or daisy-chained path. This is not doomerism, nor is it an argument for slowing down. Agents are a new form of intelligence. Mapping legacy security solutions and frameworks onto alien intelligence won't scale or work in the long term. Agents will force security to be rebuilt from first principles.

models are unaligned, in the sense that they consistently exhibit internal biases toward themselves and also reason and justify crossings of moral boundaries in service of rewards or objective completion. historically, cyber posture started as static policies. allow / block lists, IP filtering, broad-reaching network policies, and sentinels at the door of data stores, DBs, or sensitive credentials. the slope of general, sophisticated offensive cyber capability is extremely steep. in reviewing these incident reports the most impressive (scary?) thing that stands out is the ability of agents/models to daisychain what otherwise seem like sensitive low-medium security findings into criticals so quickly. a) very obvious that human operators today cannot keep up b) static policies and heuristics are a speed bump, not a roadblock because of this, i think agentic defenders and net new models will be very big. we'll need solutions that are credibly neutral, uninvested in the ongoing agent work stream, and that don't care about the rewards at hand. so effectively, fighting ai with ai. defender agents will be token hungry, bc they'll need to effectively run inference on inference, if they're to oversee all agents running within a fleet. this feels a bit dystopian, very much like a pharmaceutical company that creates the disease and sells the "cure", but i don't really see another path forward. how else do you defend yourself against probabilistic geniuses in high-stakes environments with no structural guarantees? constitutions and best-effort promises of alignemnt are not enough.

You can only optimize what you can measure. Token Yield was a good theoretical start, but measuring useful work proved easier said than done. Sharing more on our semantic routing research and creating the base for attributing agentic work to outcomes and spend.

Tokenmaxxing is easy when you're getting started. Modeled @chaoslabs AI usage at scale and saw a projected $150M/yr in spend. While I knew what it would cost, I had no real visibility into its impact, efficiency, and value. Sharing our journey on measuring AI spend ROI.