Staff engineer focused on agentic engineering and Developer Experience @datadoghq · Father to two small humans
Surrey, England
Joined October 2008
- Tweets1.6K
- Following93
- Followers208
- Likes787
Dottie, my main @bot helped me finally redesign my personal site
I'm not a designer and it won't win any awards, but after a lot of iterating and sharing inspiration with her I'm so happy with how it came out 🥰
Couldn't have done this on my own
fxndev.com
I'm a fan of the Hemingway bridge, stopping mid task so it’s easy to pick up tomorrow
But agents mess with this
Leaving them idle as I walk away feels wasteful but if I set them off I’ll keep thinking about next steps with out of date state
I need Hemingway Tasks 😅 something that sets up tomorrow better
Anyone have good ones?
“No model did best in its maker’s own harness” 🤔
Model Benchmarks Are Worthless
How can a model score 24% and 59% on the same 113 coding tasks?
It happens when you vary the harness that wraps it.
I’ve increasingly ignored most of the recent model leaderboard results. Partly from exhaustion and partly because improvements in a ranking don’t show up in the actual work.
So I asked the 8090 Labs team to measure what the leaderboards skip: the model and the harness together, on real bug fixes and features from open-source projects.
8090 Labs ran 7 models through 5 harnesses.
Here are a few results I didn't expect:
- Anthropic's Claude Fable 5.1 scored 41% in Claude Code and 61% in OpenAI's Codex. It made 28 tool calls at the median in one and 66 in the other.
- The cheapest runs weren't the cheapest work. GLM in Codex cost $0.22 a run and $0.93 per success. In Pi it cost $0.25 a run and $0.43 per success.
- GLM passed a task on both attempts 51 times in one harness and 16 times in another.
- GPT-5.6 Sol and Grok 4.6 barely cared which harness they were in.
The bottom line is that if you're picking a model/harness combo off a model-only leaderboard, you're making a critical decision based on only half the information.
8090.ai/blog/same-model-diff…
Is this rude?
I called my grok @bot Dottie
It felt like a good name to go with my Hermes agent Betty (who identifies as a friendly fox 🤷♂️)
🤖 Made with AI
My new favourite thing to do with LLMs is to help me read fiction
I’m a literal reader, maybe cause of English as 2nd language or maybe because of decades of only reading non fiction
Being able to read a book and then discuss the “underlying” meaning or whatever is awesome!
Having recommendations based on what I’ve just finished and how I found the experience is super helpful.
I feel like I’m being guided to explore titles that expose me to different devices and styles I can then discuss and interpret with no pressure or judgement
I worry that with all of the great examples of these very capable engineers shipping a ton of work in short periods of time is that so many engineers esp young ones are very impatient or don’t know what quality looks like so they can’t build the system to ship quality fast
I see a lot of “multiplayer” agent stuff
Does anyone want it or is it just something to build?
If it’s for anything beyond initial intent or maybe planning phases I don’t get it, what am I missing?
Yikes first run of Haiku 5.5 against our ADEEP evals and compared to 6 Luna
Mean score was 70.7% which is 3.94 points lower than Luna and at the same time almost 2x the cost and 27% slower per eval 😬
Haiku on Claude Code and Luna on Codex both with default reasoning
Shipped a Model Guide today to help our engineers answer: What model should I use?
Using the data from our internal ADEEP eval platform to give recs for models on the Pareto line or ones that score higher than avg && cost less than avg
Well received so far 😅
🤖 Made with AI
With models getting faster the pause is disappearing
For q&a use cases I wonder if we’ll get the same BS artificial slowdown comparison sites use
You know so it feels like it’s really thinking
Coding is different ofc
How’s everyone else handing over credentials to their Hermes agent?
We’ve just implemented something using cloudflare tunnels but feels like something that should be solved ootb?
Or is everyone just putting stuff in telegram
How much do you care about the model latency? I never think about it but apparently a lot of people do
If you do care, what would you say is your average interactive turns per pr landed? Single digits or more?
I’ve been thinking about the different ways to use agents, how prompts change, and how trust is at the core.
Pairing → micromanaging → delegating
When pairing, you prompt very specifically: “Implement a function that reads from this table in the database; use columns xyz”
As trust increases, you move to micromanaging: “Let’s create a new endpoint for the API. Call it blah, and use JSON API spec like the other endpoints.”
Finally, when delegating: “Our users need to be able to …”
Does this fit? Curious what unlocked the next level of trust for you?