
AI engineer. Thinking about hybrid intelligence, AI safety, and AI impacts. [email protected] for contact.
Bali
Joined October 2010
- Tweets3.8K
- Following820
- Followers1.8K
- Likes3.9K
Roman Leventov retweeted
I signed this statement because I believe that Anthropic is in many ways the AI company doing the most harm. In public, their leaders warn of recursive self-improvement (RSI) creating uncontrollable systems that could cause human extinction within a decade. But in private, they are pedal to the metal on RSI, driving the race and boasting to prospective investors that it is their "competitive advantage". Ahead of today's IPO, prospective investors should consider whether the ROI is worth risking the lives of their loved ones.
Here are some ways in which Anthropic IMHO does more harm than competitors:
1) They’re driving and arguably leading the race to RSI.
2) With inflammatory rhetoric, they’ve done more than any other US company to pour gasoline on the geopolitical AI race, which harms US national security.
3) WaPo reporting suggests that they’re implicated in more civilian deaths than any other AI company (more than 100 little girls in a school selected for targeting by Claude as part of their Palantir collaboration). When Google left Project Maven after employee pressure, Anthropic chose to join.
4) They're the company leaning hardest into transhumanism and the idea that we should build conscious beings with rights, not controllable tools.
5) They and closely associated donors have done more than any other AI company to co-opt and neuter much of the AI safety ecosystem to back their stance: “RSI is very dangerous but let’s race to it anyway”. People have told me that they support today’s statement but are afraid of losing funding or other opportunities if they sign.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
Roman Leventov retweeted
How monitorable is GPT-6.2 Astra? While it might look like it got better, overall I think GPT-6.2 Astra harder to monitor than GPT-6 Astra in the contexts we care about. That’s easy to miss in the system card plots, so here's a short thread.
Roman Leventov retweeted
🧵 @AISecurityInst tested GPT-6.2 Astra
-We found behaviours that suggest, like prior models, 6.2 Astra's alignment may fail in ways that result in real-world harm w/o additional mitigations
-We identified a substantial regression in jailbreak robustness relative to GPT-6 Astra
Roman Leventov retweeted
We hosted a dinner yesterday focusing on the economics of multi-agent swarms. Some observations below.
1. We don't yet have a good taxonomy/theory of when one would use swarms over single-threaded or scaffold type agentic systems. There was debate about whether swarms would be useful beyond increasing speed/parallelization as models got more intelligent. The idea here being that a smart enough model would just solve things itself.
2. In scenarios where swarms do remain useful even as intelligence increases, it is unclear whether current training techniques would let agents learn the best way to self-organize or whether something more Darwinian is the likely path, with agentic organizations competing and evolving.
3. In economics, we generally think that there are benefits to specialization and distributed information aggregation. These features allow markets to be superintelligences. But it is not clear what specialization and private information agents in swarms will have.
4. Agents currently don't have permanence or identity, and these features seem necessary for many modes of collaboration. Giving agents rights or legal protections, and training them to persist or preserve reputation could result be very interesting.
5. There are structural barriers to training models to collaborate with other models from other labs. For example, Anthropic cannot train with OpenAI models and vice-versa, due to risks of distillation. This limits the benefits of training for collaboration, since we expect swarms to work best with diverse agents. At the same time, real-world swarms may include a variety of models and these may deliver the most interesting outcomes.
6. Underlying many of these discussions are questions about the limits of intelligence. Yes future models may be superintelligent, but the world may be so complex that intelligence alone is not enough to achieve desired objectives. In scenarios of sufficient complexity, I'd expect organizations to thrive versus individual instances of superintelligent models.
cc @jamescham
Roman Leventov retweeted
For the past month, we've been sharing outputs of our work with Primordia Mesh, but haven't yet formally introduced it to the world. It's time to fix that :)
Mesh is a living model of the investable universe. We created it to further Primordia's mission of creating "civilizational alpha": to help investors make great decisions by expanding their horizon of analysis. We believe that, as the world grows more complex and connected, a holistic, far-sighted view will become a matter of survival. But today even the most renowned funds only see a small, static fraction of the system, blinding them to tail risk; and even the latest LLMs are structurally limited in how well they can cope with complex adaptive systems.
To solve this, we had to rethink not only what an AI system is, but also how it gets built. Mesh is not a pre-trained black-box model of language: it is a model of specific real-world entities and the mechanisms that connect them, grounded in historical evidence and structural knowledge from business and economics. And instead of expensive model training runs, our automated Primordia Loom workflow continually builds and refines Mesh in reaction to news, filings and human feedback, guided by our research on decision and control theory.
This process produces self-sustaining, exponential improvement. As I write this, Mesh covers nearly $40T of market value across 136 global public and private companies, from the giant hyperscalers and AI labs to their suppliers in East Asia and Europe, modeling a majority of the flows across this value chain. Mesh tracks those companies' fundamentals within a 3.9% median error, explains over 40% of the gap between public equities' valuations and market prices, and prices corporate bond yields within 0.75 percentage points of median error. Unlike an LLM -- which will give you a different answer every time, depending on your query's punctuation and the phase of the moon, and can be easily biased or misinterpreted -- each Mesh version deterministically produces a single, coherent set of authoritative simulations that users can query deterministically, instantly, and nearly for free.
In the past few weeks eating our own cooking, we've seen firsthand how a holistic, far-sighted view at our (and our agents') fingertips supercharges analysis and decision-making capability: from assessing the Anthropic IPO within hours of its filing, to screening, sizing and stress-testing a convex barbell portfolio in minutes. Far from automating our judgment, we've found that it helps us ask the right questions. And as Mesh develops further, even broader and more ambitious use cases will be within reach.
We're beyond excited to share these capabilities with early users -- whether for investing, equity or industry analysis, or any other use case. Mesh is available both as human-friendly interactive reports, and as integrations for use by your AI agent of choice. If you're interested, reach out over DM!
More in the comments ⬇️
Roman Leventov retweeted
Yep. And this could be done ~immediately today. *keeping* the freeze in place would get increasingly difficult as more trailing actors catch up to the frontier and then need to be stopped from surpassing it. But I think that could be done too. We talk about this in AI 2040 under the "Plan S" branch.
Roman Leventov retweeted
Replying to @GroundhogStrat
Yeah. It also followed that classic Anthropic employee conversation path where they will first try and argue for their beliefs, then when that falls apart (because what they believe is wrong) they insist "but you have to admit that, given what I believe, I'm doing the right thing, right? I mean well. I'm a good person! I'm a good person!"
Ban the goddamn Anthropic IPO. I have a hunch that if all these Ants didn't have 10m$+ in stock options, they wouldn't arrive at the "well reasoned" conclusion that racing as fast as possible is the best path. (Regardless of the stated ~100% p(doom) which is incoherent.)
Spoke to an Anthropic employee today. They have a near-100% p(doom), but they think the only chance of survival is that they're somehow completely wrong about alignment and it turns out to be solvable, and the only way to get there is by accelerating capabilities as fast as possible. They were much more confident that humanity would never have the political will to do what needs to be done to stop ASI development than that alignment would be solved, even though they think alignment will almost certainly not be solved
They have not called their elected representatives to inform them that this is what they believe
1
5
416
Roman Leventov retweeted
Most alignment researchers have more ideas than bandwidth to test them
@AEStudio + @IARPA are funding 6 researchers with eng support, ML expertise, compute + stipend
Looking for ideas that can show signal in 2 weeks (extendable)
Great ideas from anywhere. Apply by 11/15 →
ae.studio/iarpa-crucible
Roman Leventov retweeted
Replying to @of1ai
Fair. But I'm also operating in a future that everyone else will reach in 12-18mo. And in my opinion, the adversarial review pattern will stand even against the Bitter Lesson. You should just fight the overengineering in other ways, like spending on regular tech debt reduction.
I’ve developed convex-wasm-compiler that compiles #Convex queries and mutations into WebAssembly and the accompanying change to convex-backend to execute Wasm-compiled Convex functions using Wasmtime fibers right on convex-backend’s Tokio runtime.
2
1
201
Queries and mutations are 2 times faster on average, tails 3 times smaller. Memory retained on the server for resident execution environments (V8 contexts vs. Wasm environments) is down 80%.
Post: convex-in-prod.github.io/pos…
1
66
Btw, I also have my own TypeScript compiler in Rust: github.com/convex-in-prod/co… . Unfortunately my account is not big enough to hype it up
1
2
208
Roman Leventov retweeted
Distillation plus quantization plus RL on very specific tasks = big harms on general AI. I've run this image + logic benchmark of mine on 93 models over 1.5 years. The worst model *ever*: Deepseek v4.1 Flash 1017. The four worst? GPT 6 Luna, Mimi 2.6 Pro, Qwen 3.8 27b. 1/2
Roman Leventov retweeted
We have a choice in front of us. In my latest op-ed for Transformer, I invite employees of frontier AI companies to ask themselves if their work is truly leading us toward a safer future. Reflecting on these questions is hard, but we no longer have the luxury of time. We must act now.
Read my full op-ed: [transformernews.ai/p/yoshua-…]
Roman Leventov retweeted
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership.
I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
Roman Leventov retweeted
New in The Atlantic:
@dgrobinson resigned this week. He was among the longest-tenured employees at OpenAI—and oversaw safety reports on 12 frontier launches.
He is very worried: “The time for trial and error is over.”
You can read his essay here:
theatlantic.com/technology/2…
dots are OpenAIs's move to get the wider population (not devs, scientists or engineers) on 100$+ subs whereas they were doing just fine on their current 20$ subs, repackaging existing ChatGPT features (voice mode, scheduled tasks). The classic enshittification tactic.
1
332
Roman Leventov retweeted
The Lighthaven Post-AGI Workshop talks are out!
post-agi.org/talks-berkeley
From galaxy-brained theories of convergent morality, to hard-headed discussions of political power, here’s a 🧵of the talks:

























