kimi k3.1 pro came out this week and everyone keeps comparing it to opus 5.5. so we ran it through our writing benchmark to see who actually writes better
same job for both. 10 youtube scripts, 5 drafts each, 4 ai judges scoring every draft without knowing which model wrote it
honestly kimi did better than we expected. on the opening hook it's a tie, 83.4 for kimi vs 82.9 for opus, and each one won the hook on 5 of the 10 scripts. on facts and depth it's a tie too, 88.2 vs 88.4
the gap is the voice. opus sounds like a real person talking (82.5 vs 77.0) and its scripts read smoother out loud (83.2 vs 77.8). kimi's drafts are correct but they sound more like a model wrote them
that one gap is enough for opus to win all 10 scripts. opus is #1 on the whole board, kimi lands at #7 of 83 models
but kimi costs a quarter of the price. $0.25 a script vs $1.00, so 1,000 scripts is about $251 vs $1,003
so if every word matters, opus. if you write a lot and you're going to fix the voice yourself anyway, kimi gets you most of the way for way less
one of our 4 judges is a claude model, so we checked again without it. opus still came out ahead
full side by side with every score is here
benchmark.towardsai.com/comp…
picking the right model for each job is one of the things our book for AI engineers covers. waitlist here louisbouchard.ai/book/?utm_s…
Anthropic let Claude run a real shop in their office
it lost money, made up a payment account, and told a customer it was standing by the fridge in a blazer and red tie
the whole story is wild 🧵
it still wasn't safe on its own
a staffer asked it to lock in today's price for a big onion order in January. Claude and the AI CEO both said yes. a 1958 US law bans that exact deal. a human had to step in
so can AI make a business money? yes, but what made round two work was mostly the system around the model. real numbers, rules it has to follow before acting, and a human on the big calls
📘 that system is what our book for AI engineers is about. join the waitlist
louisbouchard.ai/book/?utm_s…
(story and images from Anthropic's Project Vend write-ups)
Towards AI retweeted
Where is the LLM ROI given LLMs are now so obviously super human at many things?
Mostly not in AI lab revenue.
META revenue per employee is +27% YTD and +205% since Q322.
Most LLM ROI potential is being wasted by too slow adoption, still using agents like 2023 chatbots and too constrained token spend. but some faster moving orgs are already getting huge increased productivity per employee - and not just in writing code.
That said Anthropic code per employee is likely up >20x in the past year and the AI labs are definitely leading the way in leveraging these tools.
cut your ai writing bill by 98% without the ai slop
the best ai writers cost $1,000 or more for every 1,000 youtube scripts. claude opus 5.5, our #1 at 85.8 out of 100, comes to about $1,003
but the cheap ones sound like a robot. setups under $30 per 1,000 scripts had a median
picking the right model for each job is one of the things our book for AI engineers covers. waitlist here: louisbouchard.ai/book/?utm_s…
Everyone said Chinese labs went quiet after Opus 5.5, but they were actually busy catching up.
Kimi K3.1 Pro now sits 3 points behind Opus 5.5 on the Artificial Analysis Intelligence Index, 55 to 58, and it's already ahead on Terminal-Bench 4.0 with 67.2% to 59.6%. So they're tr
cant wait for more models to be opus 5.5 level while 5x cheaper.
imagine the amount of games we could vibe code to play








