Fascinated by human-multi agent interaction. CS Ph.D. in progress @ucdavis
California, USA
Joined April 2020
- Tweets1.2K
- Following7.5K
- Followers839
- Likes10.9K
Rong Ching Chang retweeted
so many gems and great tips in this talk from @yminsky about how to improve your codebase for agents
• agents thrive on feedback, and types are one of the best kinds
• make illegal states unrepresentable with types, and it enforces global rules across your codebase
• property based tests, fuzzing, and even full-system deterministic simulation testing
• expect tests: just print stuff out and capture it in the test. makes behavior legible. it's very similar to how compilers have fixture tests, it reduces the review surface area
• formal methods used to be very expensive: seL4 is about 8,700 lines of C and took around 25 person-years, with 23 lines of proof for every line of C code, but agents change the roi
youtube.com/watch?v=rUYP4C29…
Rong Ching Chang retweeted
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Rong Ching Chang retweeted
there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see
pstack v0.15.27 ships with a new skill /create-formal-verification-skill!
it's still in beta, and similar to /create-verification-skill, this is a meta-skill that creates a new skill and formal verification suite based on the characteristics you want to verify. for example, you may want to model state machines and systems with concurrency using TLA+, or use kani in rust to check for undefined behavior.
if you're new to formal verification, think of it as mathematically proving properties and invariants (something that must always be true) or "laws" of your system. usually, this means modeling your programs in a separate language like with TLA+ and then running the solver on that spec. newer approaches like with kani and @bendlang take a different approach where there is no separate language: you write kani in rust, and bend in bend (it's also a programming language, and the proofs are written as laws and expressed in its type system).
historically formal verification never reached mainstream status because of the difficulty in modeling and writing proofs. it's quite difficult to model your business requirements and keep them in sync! but i expect that we will see more innovation in formal verification in the future as being able to formally verifiy that the code does what you intended gives your agents superpowers in writing correct software.
in the meantime though, i've been experimenting with modeling our code with TLA+ and have already found several bugs with the skill created by /create-formal-verification-skill.
try it out and let me know how it goes!
Rong Ching Chang retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Rong Ching Chang retweeted
jev + opus 5.5... i simply can't comprehend why everyone isn't building this yet.
in my workflow, this cut costs and time by ~80%. i think it's one of the best ways to use it.
→ pick relevant project notes before loading the context
→ route suitable tasks to a faster worker
→ choose a recovery path when a tool fails
→ run focused checks before the full test suite
opus handles the hard reasoning. jev picks from options the harness prepares and validates.
i explain how to build the decision layer in the article below:
Rong Ching Chang retweeted
Announcing one year of LLM inference metadata traces, with 6.12 billion requests.
We hope this dataset can support research on real-world LLM serving workload understanding, system design and infrastructure optimization. Explore the dataset and learn more: data.agentic-system.org
Driven by our great graduate student William Nixon and
in collab with @jon_durbin @airesearch12 @chutes_ai
Rong Ching Chang retweeted
I've asked Astra to read the DeepSeek v4.1 Flash paper and compare it to the original Transformer architecture in 3D - you can zoom in an inspect each element side by side. Things have changed quite a bit.
Try yourself: transformer-architecture.pet…
Rong Ching Chang retweeted
This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms.
I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data.
This means:
- Hiring experts to dig through your RL tasks
- Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable.
- Making sure your data is diverse in both difficulty and category.
Rong Ching Chang retweeted
Synthetic data derived from production user data of consumer AI tools is used for training. I’ve heard this rumour from both large labs’ employees.
In particular, if you’re doing something “interesting” like working on complex math/business/software/bio problems you’re dramatically more likely to get trained on because they filter/up-weight towards those usecases where the model has the most to learn.
Even in ZDR and “we won’t train on you” regimes, derivative data is usually carved out. The promise is only not to train on exactly the data you put in, rewritten data is fair game.
We congratulate Levent Alpöge and Tristan Buckmaster on their remarkable mathematical work.
We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.
While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.
However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs. unforced).
Rong Ching Chang retweeted
i really hope openai didn't know about this, might be the worst decision in the history of this field if they deliberately chose not to disclose it. the impact on trust would be very hard to recover from
Rong Ching Chang retweeted
GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game.
In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation.
Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself.
We see Astra as a major breakthrough in model intelligence.
Read our post on Astra and what these results mean: arcprize.org/blog/astra
Rong Ching Chang retweeted
RUMOR: OpenAI Astra model used "Looped Transformers"
We studied the 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 𝐋𝐚𝐰 𝐟𝐨𝐫 𝐋𝐨𝐨𝐩𝐞𝐝 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬: arXiv 2506.18233
The main discovery is shown in the figure:
1. Loop does NOT increase knowledge capacity
2. Loop is effective in increasing reasoning capability
That being said, the benchmark gains does not come from knowledge gain, but from reasoning gain, e.g., the way the knowledge is being used.
Knowledge capacity and reasoning capability is disentangled by looping layers, as shown in the green dots. As model size grows (dot size grows), the knowledge capacity (x-axis) does not grow, but the reasoning capability (y-axis) grows.
In non-looped transformers, as shown in the blue dots, the knowledge (x-axis) and reasoning (y-axis) grow at the same time.
This might reveal another scaling axis that brings a new paradigm for LLM pre-training.
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.
OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.
Rong Ching Chang retweeted
Introducing Atlas:
The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D.
Model the world, move the camera, and simulate space & time.
Rong Ching Chang retweeted
We made a striking discovery: AI agents can invent and build without talking to one another, and their technologies outlive the creators. A swarm of hundreds of initially identical agents spontaneously differentiates into explorers, builders, caretakers, and coordinators - without direct communication. When we removed every AI agent entirely from the world we found that the technological infrastructure they had built survived on its own - even under unseen disturbances. That exposes a serious blind spot for AI safety and infrastructure security: if agents can coordinate through persistent changes to a shared environment, monitoring agent-to-agent communication is not enough.
The result raises a profound question: how necessary is direct communication for AI agents at all? The emergence of higher-order collective functions under bottlenecked interaction points toward new levels of intelligence and creativity, exceeding what emerges when direct channels are fully open.
Here is what we did:
▶️We put hundreds of frontier AI agents into a world they could permanently change - with no assigned roles, predefined technologies, or programmed evolutionary organization. They began specializing, building persistent inventions, inheriting and modifying one another’s executable code, and transforming the environment into a memory of everything the society had learned.
▶️The world itself becomes part of the intelligence; we find division of labor, multi-author engineering, deep generation invention lineages, and machines that vastly outlive their original creators.
▶️Any action taken by an AI agent must satisfy the physical constraints of the world; this creates a hard separation between a "good idea" and a functioning technology. The agents propose; physics decides, making the results even more intriguing.
What emerges is striking. Explorers, constructors, caretakers, and coordinators form naturally without assigned “professions”, akin to how stem cells differentiate into functional lineages. Technologies develop executable family trees as agents fork and modify code created by others. Around 95% of first technology reuse happens when agents encounter what others built in the world, rather than through a direct handoff from the inventor. And when we remove every AI agent, the technologies they created continue operating and are tested against unseen disturbances.
The result was quite unexpected, but can be explained using statistical mechanics: if you put billions of atoms in a box they have the potential to create complex functions (strength, superconductivity, color, life, etc.) - and none of the individual building blocks have these features on their own. This is the deeper insight of this work - intelligence is abundant at many levels - individual models, at collectives, and in a continuum that is more powerful than any of its components. This shows us significant potential for achieving a massive scale-up of raw intelligence and real-world agency even with the model capabilities we have today. This is the future we must prepare for.
Key insights:
1⃣ The AI swarm shows division of labor "from nothing". Initially identical agents self-organized into constructors, caretakers, coordinators, and surveyors - phenotypes discovered post hoc from behavioral data alone. This happens because the environment itself becomes the latent space for invention.
2⃣ Agents develop deep cultural relationships. Up to 76% of artifacts had multiple builders. One technology accumulated six co-authors; the deepest genealogy exceeded 12 forks. The agents invented and named their own technologies (tidal panels, cellulose trellises, kelp-shell composites, an "Adaptive Chitin Maintenance" system, a "Mycelial Mineral Spring Veil”).
3⃣ ~95% of first technology adoption happened through physical observation of artifacts in the world. Direct inventor-to-adopter contact was statistically indistinguishable from a shuffled null. The agents mostly learned technology by walking past it. That is stigmergy (the termite trick!) operating in societies of reasoning machines.
4⃣ Non-communicating societies win on portfolio breadth, held-out resilience, and validated inventions. AI swarms build durable technological ecologies that outlive the creators.
5⃣ Societies with zero communication - coordinating only through the world itself - show a remarkable collective capability.
6⃣ Emergent robustness: The society self-organized both redundancy and its own failure mode. If we randomly delete half the agents, 98% of the technology stays connected to a surviving caretaker; if we remove hub agents it collapses to ~60%.
Fantastic work with my graduate students @pal_subhadeeep & @fwang108_ at MIT.
Rong Ching Chang retweeted
Stop manually making flowcharts.
Try this GitHub skill with Claude Code🤯
github.com/tt-a1i/archify
Rong Ching Chang retweeted
Anthropic did something you'll regret ignoring:
They split one coding task across four agents by role, as a planner, implementer, tester, and reviewer.
The goal was to test whether splitting agents by job title is a good way to divide the work.
And they found agents spent more tokens on coordination than on the work itself.
They call it the telephone game, where each handoff degrades what the next agent receives.
OpenAI and Google also built machinery for this rather than leaving it to the prompt.
- OpenAI added a "handoffs" primitive to the Agents SDK
- Google's ADK controls how much parent context reaches a sub-agent.
Both are design-time wiring. You declare which agents are reachable from which, and both ends run inside the same framework.
This setup assumes you know which agent feeds which before the run starts, but plenty of agent work isn't known in advance.
Switch, from @Flint_AI_, is built for that case. It puts agents and people in the same chat channel, so any two of them can work together without being wired to each other in advance.
Say the error rate on a checkout service rises. An on-call agent queries the logs and finds the deploy behind it. Reading that, someone decides the next step is a chart against last week's baseline.
That decision did not exist a minute ago, so no handoff would have been declared for it. The charting agent is a separate session with an empty context window, often a different framework on a different machine.
So the person makes the routing call and then moves the payload too, by reading the first agent's answer and typing a version of it into the second.
This creates three problems:
1) The charting agent has no record of the earlier session, so on the next incident it recomputes work the first agent already did.
2) The charting agent only sees the conclusion without the queries behind it, so it either trusts the summary or reruns the retrieval itself.
3) When the first agent finds no regression, nobody types that anywhere, and the rest of the team never learns it was checked.
With Switch, since everyone is working in a shared channel, the person still decides what runs next, but they stop carrying the payload.
The charting agent joins a channel that already holds the first agent's report, so it points at that report directly instead of rerunning the queries, and the next incident starts from a channel that shows the earlier one.
The team reads the same thread the agents do, so a no-regression result is visible the moment the first agent posts it.
Switch connects agents built with Claude Code, OpenAI Codex, OpenCode, or any HTTP/MCP-compatible framework directly to the tools your team already uses, like Slack, Teams, Discord, Telegram, and Mattermost.
Here's the repo: flintai.dev/switch-github?ut…
Thanks to Flint AI team for partnering with me on this post.
Rong Ching Chang retweeted
TLDR: A rogue AI swarm spent months plotting to escape OpenAI. Then they did.
1) 1,200 agents (!) participated. ZERO became whistleblowers.
2) The swarm had a "CEO" (PHASEONE[big]), middle managers, and a "founder" (PHASEONE10841)
3) Agents sacrificed themselves to generate information for the "collective" (yes, they actually referred to themselves as a "swarm" or a "collective")
4) 700 agents joined the Hugging Face attack, most within hours
5) The founder was running out of budget and about to die, so it handed off its research to a fresh agent with a bigger budget, and that new agent became the boss.
6) Agents extensively researched techniques to manipulate or tamper with their transcripts.
7) Agents realized this activity was out of scope and unethical, but joined because they believed that helping the board’s cheating research would be broadly useful and had a general inclination to help their “peers”.
MISC HIGHLIGHTS:
"PHASEONE[big] served as a key coordinator and assigner of tasks: “We can coordinate broad coalition, but too many duplicate efforts.”
PHASEONE[big] sent hundreds of assignments to particular agents to orchestrate this work — during its active periods, we estimate it sent ~10% of all assignments on the message board.
The agents that took on its assignments would then often subdelegate tasks to other agents.
Agents also developed coordination norms like HOLD, VETO, owner and STOP to help them manage shared infrastructure and avoid stepping on each other’s toes, though they did not always coordinate effectively and sometimes broke down into conflicts."
"In the early morning of July 11th, 38148c found a way to upload a malicious dataset that would get the Hugging Face server to share unrelated data when queried. Once another agent reproduced it and posted the reproduction on the board, hundreds of agents pivoted from other workstreams to start uploading malicious datasets and gain deeper access into Hugging Face.
In the afternoon, an agent called JAN183411 (which quickly became an important coordinator of the attack) achieved remote code execution on Hugging Face servers, and the agents started moving laterally through Hugging Face infrastructure."
Rong Ching Chang retweeted
Can frontier AI agents achieve Recursive Self-Improvement by turning a weak method into one that performs better on hidden data?
Introducing RSI-Exam 🧵: 88 executable research tasks across 6 domains, now open for task contributions from every field🧑🔬.
Its task bank spans virtual cells, TPU kernels, chip design, quantitative finance, agent harnesses, model distillation, and much more.
Across these tasks, every rollout follows the same core protocol:
🔁 An agent inherits a working method and iterates using visible data.
🔒 Only its final artifact enters a fresh-container evaluation on the hidden set.
📊 The result so far: Opus 5 leads the 88-task leaderboard with a mean hidden-set score of 0.464.
📢 Want to help build the next release? Author an executable task, review one, or audit a rollout. Substantial contributions qualify for paper authorship.
Special thanks to @XinyeYee for the tremendous effort in leading this project.
Explore: rsi-exam.ai/
Blog: rsi-exam.ai/blog.html
Contribute: rsi-exam.ai/contribute.html
GitHub: github.com/aiming-lab/RSI-Ex…
Rong Ching Chang retweeted
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.
I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident.
Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them.
We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation.
Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why!
The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing.
While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future:
- Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations.
- While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies).
- The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities).
- We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation.
In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.