Pinned Tweet
gpt-6.1 sol has the slowest tps and finishes a short code question first
in my testing, start to exit, typical time over 6 runs each:
gpt-6.1 sol: 9.3s
muse wonder: 15.8s
opus 5.5: 16.7s
gemini 4 argon: 17.0s
tps doesnt mean a task finishes first
Fable 5.5 WorkloadBench is waiting and ready for you
workloadbench.brenhq.com/
i'll give meta credit, i have normies talking to me about muse.
none of them talk about dots or grok bot.
lots of folks leaving Anthropic in past few weeks. timing is interesting just ahead of IPO.
or they already vested and the lockup holds them either way.
Dario is going to post on open source being a national security threat coming out today when all we wanted was Fable 5.5
Replying to @ArtificialAnlys
Large gains in agentic performance: Kimi K3.1 Pro scores 3rd on GDPval-AA with 1,846 Elo, up 312 Elo from Kimi K3, behind only Claude Opus 5.5 (1,866 Elo) and Muse Wonder Full (1,857 Elo). Kimi K3.1 Pro also achieves the highest score of any model on Terminal-Bench 4.0. K3.1 Pro scores 67%, up from 13% for Kimi K3 and ahead of Claude Sonnet 5.5 (64%). AutomationBench-AA, which tests agentic SaaS workflows, rises from 58% to 73%.
Human slop
past ~48 hours in security:
> PHP: unauthenticated remote code execution
> GitLab: CVSS 10.0 remote code execution
> Node.js: Brotli decoder RCE + 9 other flaws
> Chrome: 353 security fixes
> Signal: a bug that can prevent calls fully ending
> eMudhra: compromised CA keys
> Oracle: 1,295 security patches
> Fortinet: critical/high fixes
> Atlassian: 176 vulnerability entries
and that’s after the Linux VM escape disclosed last week.
ask Codex / your agent to check your code, dependencies and infrastructure for exposure to these.
set up automations to regularly check for newly disclosed vulnerabilities and flag what needs patching.
same opus 5.5, newer claude code cli version, less thinking on my tickets test.
testing versions 2.1.308 against 2.1.284, every tickets run thought less, about 19% at the median.
Opus 5.5's thinking on my bug hunt spiked on Sept 30, then went back.
from my testing it used:
~ 1,737 thinking tokens per run on Sept 29
~ 5,314 on Sept 30
~ 1,736 on Oct 21
same Claude Code version.
different cli version tests running today.
keep seeing reports of reduction in ability of Opus 5.5 on x in the past few days.
going to run some tests in the lab and will post on workloadbench.brenhq.com
also starting a new testing theory today with different cli versions to see their potential impact on test results
Coding is solved
October has been an absolute nightmare for software
> Railway - customer secrets exposure
> LiteLLM - malicious PyPI + Docker releases
> Zeabur - infrastructure breached
> OpenCode - AI API keys exposed
> OVHcloud CLI - official binaries backdoored
> Rank Math PRO - malware in official plugin
> TeamUpdraft - customer data compromised
> ConnectWise - cloud backup identities compromised
> Tensorlake - npm package infected
> SubQuery - backdoored npm package
> GhostAction - GitHub repos with malicious workflows
> PHP - critical RCE vulnerabilities
> Node.js - security vulnerabilities patched
> eMudhra - certificate authority keys compromised
> Tailscale - eMudhra incident
At this point, the biggest security risk might not be the code we write, but the code we trust...
keep seeing reports of reduction in ability of Opus 5.5 on x in the past few days.
going to run some tests in the lab and will post on workloadbench.brenhq.com
also starting a new testing theory today with different cli versions to see their potential impact on test results
From my testing of Opus 5.5 and GPT-6.1 SOL, 89 to 96% of a typical long task's prompt came from cache.
Replying to @BrenBuilds
Haven’t read the thread, and I’m still sleepy, just woke up.
But the premise that the “agent has to learn everything in the project from scratch” is false.
It doesn’t need to understand the entire project to implement a new feature or fix a bug. This is obvious true, because much can be inferred from a single file from the interfaces it interacts with.
And the more self-contained your files and modules are, the less context it needs.
Of course, some tasks require a broader understanding of the codebase. But that can be rediscovered as needed.
This is something I realized pretty early on. And it’s still one of the things I actively control as the human in the loop: I decide what context the agents get, and what they need to know. A well specified prompt will make the agent state whether more context is needed, but the agent can figure that out by itself anyways.
I’d go further and argue that “knowing everything about the project” is detrimental. The underlying architecture is still the same, and everything in the context competes for attention. Things unrelated to the task injected into the context can only be detrimental.
all the comments here are plugin/tools/mem caches which all *still burn tokens, either by more model calls or just file bloat.
And yes, all the solutions to this problem become "stale". a conversation you just had adding a feature has made prior chats stale.
just ported pstack to a codex plugin
updated models.json to only models i prefer
opened codex
$pstack:create-verification-skill
its creating verification and tests for areas ive never thought to check in my app...
simply magnificent























