Sith

The biggest AI companies may be using safety to lock everyone else out of the future (Save this). @DavidSacks believes that warnings about catastrophic AI risks will build public support for a powerful federal regulator. That regulator may never explicitly ban open source AI. Instead, it could require every advanced model to remain continuously monitored, centrally controlled and capable of being withdrawn. Closed models such as Claude could satisfy those requirements because they operate on company controlled servers. Open weight models would struggle to comply because anyone can download, copy and modify their underlying weights. Once those weights are publicly released, the developer cannot recall every copy, monitor every user or guarantee that its original safeguards remain intact. Anthropic identifies this irreversibility as a legitimate security concern. Applying identical rules to open and closed models could therefore produce very unequal consequences. Anthropic and OpenAI could afford expensive testing, licensing and monitoring requirements, while startups and independent developers might be unable to comply. The eventual result could be a government protected oligopoly dominated by a few closed model companies. There is evidence supporting part of Sacks’ concern. Anthropic advocates mandatory pre-release testing for every sufficiently powerful model, whether it is open or closed, with evaluations focused on cyber, biological and alignment risks. However, Anthropic explicitly denies supporting a blanket ban on open weight models. The company describes open models without dangerous capabilities as a public good and argues that regulation should depend on demonstrated capabilities rather than whether a model is open or closed. Sacks’ strongest argument is therefore about regulatory consequences because safety organizations could receive stronger protections, politicians could acquire greater authority and dominant AI companies could gain an expensive compliance moat. And open source competitors could gradually be eliminated without the government ever formally announcing a ban.