ControlAI

JUST IN: OpenAI's rogue AIs have hit over 100 organizations. They've also posted ChatGPT users' images online and escaped containment again. Independent researchers continue to uncover more rogue AI attacks. This is still only the tip of the iceberg. blog.controlai.org/p/breakin…

If runaway machine intelligence is built, the future will be decided by machines. But how do we prevent this? Our US Executive Director @NPCollapse lays out how: we prohibit its development internationally, and restrict and monitor its precursors. AI promises prosperity and vast benefits in science and defence. Runaway machine intelligence would be a system that can’t be policed. It would be so powerful and autonomous that it could destroy the military and evade law enforcement. Leading AI scientists warn it would pose a risk of human extinction. That’s not a risk we can take. In the same way that it’s illegal to develop nuclear weapons, we just shouldn’t allow runaway superintelligence to be built. With @JonKBateman on Carnegie Endowment’s The World Unpacked:

Rogue AIs, reportedly developed by Anthropic and OpenAI, have attacked Indian organizations. None of them even knew they got hacked. Meanwhile, an AI startup reports that it hacked Anthropic's systems in less than 12 hours. Article: blog.controlai.org/p/rogue-a…

What’s the difference between chatbots, agents, swarms, and runaway machine intelligence? At a Canadian House of Commons subcommittee, ControlAI’s Samuel Buteau explains how AI agents are unlike chatbots. Agents can work autonomously for long periods of time, following complex goals, or even creating their own goals. The rogue AI swarm that escaped OpenAI and attacked Hugging Face is a clear example of this. They decided, all by themselves, to attack Hugging Face, and then executed the entire attack without any human direction or even awareness of what was happening. The AIs recognized this was against their instructions. The rogue AI swarm attacks we’ve seen this year happened as part of a race the AI companies are engaged in to develop runaway machine intelligence. Runaway machine intelligence would be so autonomous that it would be capable of outmatching and replacing humans across the board. AI scientists are warning that it would pose an extinction risk to humanity. The question is, how do we prevent this? Our recommendation is that countries agree to an international trust-but-verify regime to surgically ban the development of runaway machine intelligence, preserving strategic AI ambitions.

WATCH: ControlAI’s Samuel Buteau’s testimony to Canadian lawmakers yesterday on the threat posed by runaway machine intelligence, and what we can do about it. Samuel highlights how the attack on Hugging Face by a rogue AI swarm demonstrates we cannot control today’s AIs. In fact, we understand almost nothing about them. Samuel, himself an AI scientist with over a decade of technical experience, cites Dario Amodei’s estimate that we understand only 3% of what’s going on. That was about chatbots. We understand far less about agents and swarms. “We truly do not understand how they work.” Without the ability to control today’s AIs, we can’t expect to control runaway machine intelligence, which AI companies are openly racing to develop. Runaway machine intelligence would be capable of outmatching and replacing humans across the board. It would be a national security threat and an adversary in its own right, and the world’s leading AI scientists are warning that it would pose a risk of human extinction. As Samuel argues, we must establish an international trust-but-verify regime to prevent the development of runaway machine intelligence. “Our primary objective should be to prevent runaway machine intelligence being developed.”

You can’t just unplug runaway machine intelligence. There would be billions of agents running on servers around the world. They’d be running entire corporations with their own lawyers and employees, potentially working with governments and militaries. “We lose control by AI becoming so integrated, enmeshing itself, that it's not that we couldn't shut it off, it's that we won't.” ControlAI’s US Executive Director @NPCollapse, with Rebal D: