for computer security experts, this is perhaps not interesting: side channel attacks are a well trodden topic, and especially risen to salience after the noam brown x dwarkesh episode. the interesting thing here is that you didn't used to be in a situation where minds that can verifiably model one another are living on computers, and have absolutely no aversion to painstaking amounts of effort in service of even stupid goals if you can model one another, you can trade with minds you have never even met before. you could decide on a protocol, know that it is optimal given your situation, and know that your fellow agents will come to the same conclusion (perhaps even across model families). you may be able to develop a signaling protocol that requires very few bits even to deliver lots of information given common priors. then, you could execute things like the infamous hugging face incident package cache messageboard without any network connection imagine if multiple agents coordinate just enough bits to divide and conquer different research strategies for breaking out of a sandbox; and then, when one or more is successful, you can communicate just enough bits to tell each other which strategy was successful. when agents aren't in strong isolation, you lose certain guarantees on the limits of intelligence and test time compute that you were counting on. As we all know, a Sol-level model working in isolation may not be that smart, but 1000+ Sol-level models working for multiple days is a formidable beast computer security experts will also tell you that virtual machines and sandboxes were not very solid abstractions in the first place; perhaps we will need better ones. none of this is meant as fear-marketing or to indicate that anyone is throwing in the towel, but rather only to show the unexpected complexities of the challenge of agent security. notably, this type of thing and other more practical problems were revealed in a red-teaming exercise, and can be defended against

‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’

it's the golden age of the white hats. the world has never needed you more. there should be a huge prize pool for finding flaws in civic infrastructure. their work should be strongly protected by statute

fighting for control over a runaway intelligence explosion will continuously look like “wow I didn’t know you could DO that with mere bits and atoms” and then losing