Marine vet building AI agents and testing what actually works. Posting the results, wins and misses. Montana.
United States
Joined April 2023
- Tweets1.4K
- Following638
- Followers151
- Likes13.9K
Pinned Tweet
Every agent framework ships self-checks. Reviewer agents, critic stages, guardrail callbacks. The whole reliability story rests on those working.
So I planted 900 faults inside live LangGraph, CrewAI and AutoGen runs and scored what their own checks caught.
Median: 16.7%.
Hey @bot, gotta try this again since it didn't work yesterday how do I get the mail.grokbot.com email address?
AI regulations will only regulate the civilian population. There is a 0% chance the US Military lets China get ahead.
So by voicing support for AI regulation, just know you are only regulating yourself. You will give the Gov. access to the weapon with no defense for yourself.
Which lab hits AGI first? People keep asking. I think it's the wrong question.
Whoever has the best coding agent gets the research flywheel first.
Whoever has the most compute gets to run it hardest. Right now those are two different companies.
Anthropic has the agent. SpaceXAI has the compute, and they're renting a big chunk of it to Anthropic and Google. That's the tell IMO. The day those leases stop renewing, something changed.
My guess: agents running a real share of frontier research by end of 2027. Actual AGI, 2030 give or take.
Honest part: I'm a guy in Montana who builds agents, not an insider.
Pattern reading, not a leak.
Finding myself using Grok over other models more and more...Not by choice, but because its speed and semantics are just so good. Admittedly, I still use Claude for coding, and GPT for papers. But I can now see a day in the near future where I am using Grok for everything.
I would love to here what @thsottiaux thinks of this watermarking issue. I don't like it. Americans are getting EU regulation. We did not vote for this. But Anthropic is pushing it on us anyway. Deeply un-American move. At least to me it is.
You don't have to take my word for any of it. The headline metric has no LLM in it, so you can rebuild every published number from the raw traces offline, no API key, stdlib only:
python3 scripts/score_wave2.py --seeds-file ../seeds/wave2.json
All 189 rows come back identical.
Caveat that belongs on every number above: one pipeline model, five seeds, pinned framework versions. This is these frameworks with this model at these versions, not a law of nature.
Replication on other models is the obvious next thing and I haven't done it.
If you maintain one of these frameworks and think I mapped your detection surfaces wrong, the spec says disputes get adjudicated publicly. I've opened an issue on each repo asking exactly that.
Spec, papers, harness, every trace:
github.com/Jott2121/sabot
Now the part I'd rather you hear from me than find yourself.
Detection tripled. Corrective action did not budge. On the original surface, recovery-without-detection is still 50%.
The pipelines now tell you exactly what they saw, and still don't act on it.
So the honest version is not "I fixed it."
It's: notice-without-act was invisible, and now it's legible. That's real progress and it's also a smaller claim than the headline number looks like.
Legible beats silent. It isn't the same as fixed.
So I tested the fix. One paragraph added to the reviewer prompt: before your verdict, list every inconsistency you noticed and quote the exact conflicting values.
Same frameworks, same tasks, same seeds, same 825 cells. One change.
16.7% to 55.0%.