徐森凱 | research @iapsAI | prev @RANDCorporation @CSETGeorgetown @OpenAI &c | 🗣 中文 русский … | 横眉冷对千夫点赞
Cool Gray City of Love
Joined March 2011
- Tweets1.9K
- Following3.3K
- Followers1.9K
- Likes5.6K
Pinned Tweet
The risks of AI have become increasingly unmistakable this year. As the situation becomes even more pressing and complex, regular dialogue between the US and China is all the more important.
In a new report, my team suggests how to do this well.
Karson Elmgren retweeted
If just a few lab staff took on verification work full-time, verifying a US-China pacing deal could get much more feasible within months.
Here are 8 specific things lab staff could start on today to help make international verification possible: 🧵
Karson Elmgren retweeted
We’re Fangcun AI (方寸跃迁), China’s first lab dedicated to independent third-party evaluation & technique research on frontier AI safety.
Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
The US and China have agreed to set up an AI incident notification channel, but how exactly might it be implemented? @SarahJGodek @Clarissa_Koh7 & I have just published some recommendations iaps.ai/research/operational…
We also discuss formalizing and strengthening the channel. An MOU could give it a bit more force. Over time, verification technology could also be used to make claims about incidents more reliable than just "trust me, bro."
You can find the full memo and our suggested notification template here: iaps.ai/research/operational…
Karson Elmgren retweeted
[1/7] China’s AI labs are also talking about “recursive self-improvement” (RSI). A big question here is: Is Beijing coordinating a race to achieve RSI?
@ZilanQian and I looked at Chinese AI companies, investors, and policy docs.
We found a more bottom-up, commercial story.
Interesting article in the Journal of the Party School this Sept on national security risks from AI, incl "technology fetishism" and "pan-entertainmentism" as well as "carbon-based life being eliminated by silicon-based life" among more mundane items
(Screenshots are AI, some minor errors)
Pair this with TC260's AI Safety Governance Framework 3.0, as well as the (over-reported, not very interesting) recent article from MSS head Chen Yixin and (slept on, more interesting) speech from CAC Deputy Director Wang Lihong for a sense of the scope of thinking on AI risks in the Chinese party-state
Full article here: mp.weixin.qq.com/s/ltbfMDjWG…
Karson Elmgren retweeted
🚨New paper (possibly the most important one I have written yet).
We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios.
Led by @KamacheeMax.
Karson Elmgren retweeted
Tomorrow’s the last day to register to vote by mail in SF!
Lots of important stuff going on inc. @Scott_Wiener (track record of getting stuff done on AI safety, YIMBY) v Chan (misrepresenting Wiener’s record, NIMBY). It’s gonna be close
Info here: ca11.info/
Karson Elmgren retweeted
New from ByteDance: it plans to invite 10 "leading researchers with a strong sense of AGI mission and responsibility" to join the new department.
Notable that the JD includes working with "external partners", universities and research orgs to jointly improve AI model safety. Also mentions interest in folks with prior experience working on CoT monitoring, mechanistic interpretability and CBRN risks.
Location: Beijing, Shanghai, San Jose
mp.weixin.qq.com/s/YauSE1KRB…
ByteDance has set up a dedicated frontier AI model alignment and preparedness department, Chinese media The Paper reports.
The department will work on model/agent alignment and frontier risk mitigation including CBRN (one of the first Chinese companies to say publicly that they are working on this).
Worth noting that ByteDance has always had a team working on model alignment, led by Yan Lin (严林). What's new is that this work is now being elevated to the level of a standalone department, while still being led by Yan.
thepaper.cn/newsDetail_forwa…
Karson Elmgren retweeted
Sources have anonymously briefed the South China Morning Post on latest US-China AI talks:
• Regular discussions to iron out framework
• AI cooperation could be a joint deliverable by next months APEC summit where Xi and Trump will be together
• This may include a basic framework for addressing AI-related crises “with operational details to be worked out later”, and “an announcement establishing a continuing institutional mechanism for AI cooperation, involving government officials, research institutions and industry representatives.”
• Clear distinction between an AI safety and incident-management agreement, which is already under discussion, and a broader AI economic cooperation framework, which would be “considerably more ambitious”, per one source.
Karson Elmgren retweeted
ByteDance has set up a dedicated frontier AI model alignment and preparedness department, Chinese media The Paper reports.
The department will work on model/agent alignment and frontier risk mitigation including CBRN (one of the first Chinese companies to say publicly that they are working on this).
Worth noting that ByteDance has always had a team working on model alignment, led by Yan Lin (严林). What's new is that this work is now being elevated to the level of a standalone department, while still being led by Yan.
thepaper.cn/newsDetail_forwa…
Karson Elmgren retweeted
Semianalysis said yesterday that “Beijing Will Not Pace the Frontier,” citing “China’s Speed-First AI Safety Regime.” imo, I think that it is more accurate to say that “Beijing Thinks About AI Safety Differently and Has Growing Concerns About AI Risks.”
newsletter.semianalysis.com/…
