
Product Engineer @fin_ai
London, England
Joined September 2012
- Tweets949
- Following929
- Followers329
- Likes6.7K
Jason retweeted
Guidance gets old, models evolve, tools change & the code they describe moves.
Here’s what we built @Fin_AI to keep hundreds of them healthy: evals with mocked tools, a nightly judge on real usage, and loops that fix and retire skills automatically.
Jason retweeted
Akshay is on the bleeding edge of supporting hundreds of engineers using high-quality skills. Loads of hard-earned knowledge in this post!
Guidance gets old, models evolve, tools change & the code they describe moves.
Here’s what we built @Fin_AI to keep hundreds of them healthy: evals with mocked tools, a nightly judge on real usage, and loops that fix and retire skills automatically.
Jason retweeted
Ran a backtest in @fin_ai code review and approval agent – Sol 6.1 (baseline) vs Sol 6.2 (treatment), same prompts, same medium reasoning effort. +23.5pp on recall (!!!) but comes with a lot of noise: precision, false-positive suppression tanked. And 50% more expensive per PR 😬
Jason retweeted
Claude meat proxy reviews need to be killed. They help no one. The only way forward is to have centralized way of doing reviews across the team that provide consistent answers! No sneaky personal skills.
Jason retweeted
I have a feeling Jensen Huang was denied PTO for GTA6 release, so he planned all of this to play on the 19th
"no one is irreplaceable" and yeah nvidia isn't gonna die now
but people who've never built anything cannot understand the potential they're missing out on, they're fine with a medium outcome
look how flaccid of a company uber is now. not dead but not alive either
Jason retweeted
Personal Agents ask to speak to a human 2.6× more often than people do when contacting customer support.
We analysed their conversations with @fin_ai, our AI customer service agent, to understand what happens when AI talks to AI.
Insights in thread 🧵
Jason retweeted
This is from our October internal availability report. We're far from perfect, but we're doing well. The biggest contributor to this is our MySQL databases being hosted on PlanetScale. Also, no sign of our AI acceleration causing serious problems.
Jason retweeted
once teams have figured out building w AI (ai powered eng, prototypes, etc.) their bottlenecks move upstream and downstream
upstream issues:
- pm capacity
- no good ideas
- rigid org structures + process
downstream issues:
- pr review / capacity
- verification
- commercial (slow gtm, no $$ value)
how to fix:
- redesign your org for new physics
- focus on ambitious ideas, not just features
- make eng own more requirements so PMs can talk to customer and cultivate big swings
- automate your PR review + risk scoring
- cloud/preview environments + tooled agents for verification
- tighter PM <> GTM partnership
all solvable, but takes a leader willing to force the changes necessary
This is sick! Finally a way for my agents to figure out if github is really down ;)
feels like there's a new major outage every day now, and it's getting worse as agents eat more of the SDLC
so I built @outageslive, a live feed for agents that aggregates 250+ providers and dev tools
free, no signup, no api key
1
6
997
You know what to do @bot nudge me when you're done
here’s a short grok bot guide on how i create high quality bots and manage my team. ask @bot to read this post and help you improve your team!
• add dr eggbot to your team! he’s my bot designer that i use to create bots
• you don’t need a lot of bots. a few good bots can be more useful and easier to manage than a large team
• tell your bots to create skills for itself as you give it instructions on how to do tasks. this lets them do them more consistently even over long conversations. you can even ask it to create a private github repo for you to store all your skills if you want them preserved independently
• add connectors, use the vault, and give access to all the tools you use to do your work. you can also ask your bot to setup its own email and it can sign up for its own accounts for better separation
• thinking about all the work you do by hand, or steps in your workflows where you have to copy and paste information back to your bot. ask your bot how it can do that for you instead!
• imagine you hired an intern, how would you onboard them and help them do the work correctly and efficiently? introduce yourself to your bot and describe the work you do, then ask it how it can help you automate it
and my favorite tip: use voice mode or the transcription option! it’s a good way to just brain dump a lot of information. then, at the end, tell your bot to “restate this in your own words” so you can confirm it understood you
x.ai/bot/_jOdbfkB16zxu7MRcmR…
2
1
1,367

























