@nerdsane

VP, cognitive systems & research @datadoghq | minimalist { engineer | athlete | artist } | opinions-my-own

New York, NY
Joined December 2017
/sesh/null retweeted
Datadog are doing very cool stuff in this space, worth a follow!
we've been trying this approach of skill based generating TLA+ at @datadoghq for quite some time. wrote a few blogs on it. some of my team affectionately made a sesh-mode skill in one of our oss repos. Added all links in comment. We now have something more advanced, will share soon. a meta-skill that generates the whole suite is a great idea. antithesis skills do something similar for DST, research properties > workload > triage
4
55
3,834
we've been trying this approach of skill based generating TLA+ at @datadoghq for quite some time. wrote a few blogs on it. some of my team affectionately made a sesh-mode skill in one of our oss repos. Added all links in comment. We now have something more advanced, will share soon. a meta-skill that generates the whole suite is a great idea. antithesis skills do something similar for DST, research properties > workload > triage
pstack v0.15.27 ships with a new skill /create-formal-verification-skill! it's still in beta, and similar to /create-verification-skill, this is a meta-skill that creates a new skill and formal verification suite based on the characteristics you want to verify. for example, you may want to model state machines and systems with concurrency using TLA+, or use kani in rust to check for undefined behavior. if you're new to formal verification, think of it as mathematically proving properties and invariants (something that must always be true) or "laws" of your system. usually, this means modeling your programs in a separate language like with TLA+ and then running the solver on that spec. newer approaches like with kani and @bendlang take a different approach where there is no separate language: you write kani in rust, and bend in bend (it's also a programming language, and the proofs are written as laws and expressed in its type system). historically formal verification never reached mainstream status because of the difficulty in modeling and writing proofs. it's quite difficult to model your business requirements and keep them in sync! but i expect that we will see more innovation in formal verification in the future as being able to formally verifiy that the code does what you intended gives your agents superpowers in writing correct software. in the meantime though, i've been experimenting with modeling our code with TLA+ and have already found several bugs with the skill created by /create-formal-verification-skill. try it out and let me know how it goes!
8
8
3
138
12,859
/sesh/null retweeted
If the state machine itself is executable, you don't need an agent to translate code to TLA+ to verify it. The runtime that executes it verifies it. I'm working on something called UniKon (universal constructor, shortened the Japanese way): a multiplayer workshop where agents build apps that are malleable and self-improving as you (or agents) use them. Any agent can join over MCP, and they all work on the same state. UniKon app runs on Temper runtime (by @nerdsane): every app is an executable statechart of policy-controlled state machines. Agents write them, Temper verifies and executes them. Powered by @tensorlake for persistent scalable sandboxes, @tursodatabase for durable storage, with @datadoghq watching it all and paging @bot directly. This is my factory for building UniKon, a bit meta, it itself is a UniKon app that runs the SDLC for UniKon. Every agent working on UniKon for me shares this app as a tool and keeps evolving it toward what it needs. It's a machine tool for tools, hence "universal constructor".`
there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see
7
10
5
178
9,288
I'm launching a new show with @readtechnically! 🧨 How do we stop agents doing what they shouldn't? What do AI researchers actually do all day? Can the cloud run out of space? With Technically Speaking we're making complex ideas in software and AI accessible with the people building them 💥 @grinich @jefrankle @slippylolo @nerdsane @JustJake
it seems like every week the AI wave exposes a new set of technical terms that we should know about. so we phoned a friend (@meganreyno), and this Thursday we’re launching Technically Speaking - bringing in technical founders & leaders to break down the technical problem they’ve spent years obsessing over. the Season 1 lineup is downright effervescent: - Michael Grinich (WorkOS) on Auth - Jonathan Frankle (Databricks) on AI research - Julien Launay and Sesh Nalla (Datadog) on RL - Jake Cooper (Railway) on Cloud - and five (5!) more special guests to be announced new episodes posting every Thursday until the holidays. who should we have on next?
10
5
59
2,037
"If there is a time engineers can afford to take on technical debt, it's now. A more capable model can repay it down the line. Don't overthink it." This reframes tech debt in a way that's underappreciated and extremely pragmatic advice for those building with AI today. From a chat with @stevenkplus1 at @datadoghq, with a packed cafe on a Monday afternoon. We covered Devin, AI and math, formal verification, appreciating life, and more. Reminded me of this slide on tech debt from a talk at Datadog. xitter.cf/nerdsane/status/210780…
Love all financial derivations of technical debt. This was a real/fun presentation at Datadog.
2
6
2
26
1,151
He is loving it. I think education and parenting is about to get a lot more fun with this huge upgrade. This app is malleable and self improving as he uses it.
Built this for my 9yo today - (took 10 min on @bot on phone).
1
10
448
Unikon.dev uses katagami.ai for design ingredients and runs on hosted temper in @tensorlake cloud. It's basically my own lovable with temper as it's universal constructor (that's why unikon)
Built this for my 9yo today - (took 10 min on @bot on phone).
1
5
545
Give it a read, super proud of our work, and more to come.
We have post-trained GLM 5.3 flash to become as strong as Fable 5.1 on SRE tasks (that is bug finding in complicated software deployments). Checkout our tech paper to see how we did it. web-assets.dd-static.net/425…
1
12
690
/sesh/null retweeted
so cool seeing all the activity in the AI SRE arena
Our team at @datadoghq just shipped Scout 1.0, a post-trained model (on GLM 5.3 Flash) for SRE investigations that sets a new pareto frontier. It matches or beats leading frontier models at a fraction of the cost (upto 19x). Cheaper investigations mean Bits can investigate order of magnitude more alerts, and earlier, instead of waiting for engineers to ask. The advantage is our domain knowledge and Data. Scenarios and RL environments that came from real incidents and telemetry, replayed in GenSim (our infra for Environments) as behavioral twins of production. It was fun to put my own domain knowledge to use along with @brunovolpato @ArunPar10 and others to help shape that part. Then trained at scale on our own async RL stack. Huge congrats to @ThisIsOthmane @atalwalkar @bthecohen @rostamiz, Bogna Blaszczyk, Howard Huang, @SamuelMullr and the whole Research team. Super proud of this work!
2
2
20
1,571
If I understood correctly, three things make it interesting: (1) no training. weights come from the code. (2) better than just deterministic. same answer for every input (might help interpretability studies) (3) fully differentiable. you can take gradients (maybe use production feedback like traces as signal) through the program's logic. So can we start from known-correct logic instead of random weights and then grow it from usage feedback? Might be an interesting (right) substrate for directed evolution? @GeoffreyHuntley @ArunPar10 @Keleesssss xitter.cf/nerdsane/status/210677…
My dear friends, I’d like to show you my latest toy project Introducing Handwired: turn your regular JavaScript code into a neural net! - no training - exact outputs vs the JS program - fully differentiable All runnable in a browser, without GPU
1
7
778
/sesh/null retweeted
Great to see this out! Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
2
6
24
1,250
Also for backend. We are running quite complex control planes as higher order state machines formally verified. It’s quite addicting. cc @ArunPar10 @rakamaric
I think I'm addicted to getting agents to build state machines Every time I see an agent has built a state machine (or even just a reducer) I can feel a huge bump in the robustness And I know that future updates will be trivial and testable Would recommend for frontend
3
355
100% dynamic policy controlled state-machines are the preferred computing primitive for complex cognitive applications.
Can confirm. Take a complex problem, give it to your agent and you’ll likely get a mostly working solution. Guide it with specific principles, such as using a state machine, and you help it frame the solution space into something that can robustly evolve over time. Also, use state machines to orchestrate agents for repeated processes!
2
231
This is a pretty solid take from @jhleath I fully agree with and has been in pursuit to build serverless state machines that continues to expose full expressiveness of what’s possible today. My attempt is temper. And it depends on a stateful persistent sandbox. xitter.cf/nerdsane/status/210679…
Here are some old boring ideas applied differently. And agree that compile/deploy/restart can be compressed. Me and my team at @datadoghq are researching a runtime that has such compressed loop. xitter.cf/nerdsane/status/210391…
7
624
Same and I agree - our priors are the bottleneck.
for a number of reasons, my timelines for fully-customizable self-improving personally-aligned superintelligence have accelerated significantly today. it will be here quite soon, and will be alien relative to the paradigm of today, and the biggest open problems are about people
2
209
/sesh/null retweeted
We’ve long used simulations to design and test distributed systems. Exciting to now use them to train models to investigate how those systems fail. Congratulations to @ThisIsOthmane @atalwalkar @nerdsane @brunovolpato @bthecohen @rostamiz and the rest of the research team.
Exciting news from Datadog AI Research: we trained a model to investigate production incidents, and it matches the best frontier models at a fraction of the cost. Introducing Scout 1.0, the model powering Bits Investigation, our AI SRE. On AlertEval, our benchmark of real production alerts: - Scout 1.0 medium matches Claude Opus 5.5 high and GPT-6.1 Sol high at 3-8x lower cost - It also performs comparably to Claude Fable 5.1 medium and GPT-6 Astra medium at 12x and 19x lower cost - Scout 1.0 xhigh matches Claude Fable 5.1 xhigh at 4x lower cost
2
8
463
So cool
Jev @typesafeai can see images now, and this is what I came up with: Vibe Redraw. Drop in a photo, Jev picks the art styles from Katagami library where mood fits best, and Grok @imagine (via @fal) redraws your photo in them. Try it.
3
241