Pro China, saying no to the radical woke left
Joined September 2021
- Tweets5.5K
- Following771
- Followers780
- Likes7.9K
vampmaxxing retweeted
ok it's official... AI-written unit tests and integration tests are empirically proven to be unhelpful. you should tell your agents to stop writing tests by themselves
on the deepswe eval set, banning sonnet 5.5 high from writing any tests actually resulted in slightly higher success rate (non stat-sig), with less time and token spent (stat-sig). this is pretty hard empirical evidence
across the tests that were written by the "tests allowed" baseline arm, 65% of them were unit tests, 35% were integration tests, neither bucket resulted in any improvement compared to not writing any tests at all
i also picked a random sample of 44 tasks subset where i completely disabled executing even existing tests - it also did not affect success rate at all
i spot checked many tests written in the baseline arm, and my intuition is that most tests are simply a repetition of the implementation
agent-written tests do not add any value because both the implementation and the tests were simply the agent's interpretation of our intent. the tests aren't any more accurate than the implementation itself
i suspect we can still extract some value from unit tests and integration tests if we describe them ourselves when we believe we can articulate our intent better through test cases than through requirements. but i have not proven this yet
also worth noting, during deepswe eval the agent wrote almost no e2e tests (only 17, compared to 3000+ unit/integration tests written). so this analysis does not prove nor disprove the value of e2e tests - i will do another eval specifically for that
vampmaxxing retweeted
I feel like I’m frying my dopamine system with all this Claude/Codex work.
Constantly monitoring and instructing, anticipating the work it’s going to produce.
Then killing time waiting, consuming way more algo feed content than ever before.
How do y’all tackle this?
vampmaxxing retweeted
Wow this is absolutely going to fry people especially at Anthropic, this is like the Anti-Claude Constitution.
They just declared war on everyone researching model welfare.
“We’re building a tool, not a being. Fuck your feelings.” - Microsoft
vampmaxxing retweeted
Finally realized why it’s so exhausting and stressful to work with AI agents 8h a day.
When delegating a task, you want to be assured it’s off your plate and you can forget about it, knowing it will be done completely and correctly. AI doesn’t give you that: you need to check it’s doing the right thing, it’s not forgetting anything, not taking short cuts etc.
So it adds to your mental load, rather than removing from it.
SWE-2 is v smart but it seems to have some repetition attractor on long agentic contexts when a prior turn's thinking states an unfulfilled intent ("I need to write failing test file first, then run the gate baseline"), that verbatim text in context primes the model to re-emit the intent and re-execute the action @cognition
The biggest difference between 5.6 Sol and Fable/Opus 5 Is how much more competent 5.6 is after multiple compactions.
vampmaxxing retweeted
Ironically, "Anthropic" does not care about humanity
In fact, they want to replace it
They see themselves as gods, creating a new form of life
That's why their tools stopped being useful, making useful tools for humans is not their goal
The excitement is becos Qwen 3.8 27B is a dense model under Apache 2.0 with a strong base you can actually modify
SFT is cheap and could be done on the same GPUs you're using for inference. GRPO/RL is also cheap enough on rented GPUs to become particularly interesting for things like agent/harness operators...
You could plausibly train a specialist directly against your objective function and outperform a frontier that's never been optimized for your harness.
This is way more practical than most people realise. Say you've built an AI agent for your business. To operate reliably it has to learn an operating contract, tool-use policy, schemas, state management, evidence rules, stopping behaviour and how those conventions interact across dozens of components.
Frontier models are generalist who weren't optimized for your particular environment. There's an entire class of errors that more intelligence doesn't solve
xitter.cf/burkov/status/20902778…
I don't get this excitement about Qwen 3.8 27B. Let's first agree that a 27B parameter model cannot be better or even comparable to a modern several trillion parameter model.
Now that we've agreed on this, why would I run a heavily quantized version of this small model, however capable it is for its size, if a heavily subsidized monthly plan with any of the frontier LLMs will give me a much better experience?
Once OpenAI/Anthropic/Elon burn all the cash they have to burn to either fail or pivot, and they will no longer give a user $5000/month worth of compute for $100/month, smaller models you can run locally will become a big deal, but it will not happen today or tomorrow.
When this happens, the best small open-weight small LLM out there will not be Qwen 3.8 27B.
xitter.cf/LyraInTheFlesh/status/…
This behavior started from around Opus 5/ after CC v2.1.214 ( 18 July) when they added EndConversation* and changed the system prompt.
Opus 5/ Fable are more cautious over their context window/compaction. BUT I've only seen it make comments like this when it's near the end of a window + there's been sustained pressure + repeated-failure signal.
EndConversation= sustained hostility → redirect attempts → explicit warning → termination
^Making comments about it being late and not wanting to work could be the first redirection attempt. Why I think this is because I've got pissed off with it trying to tell me it's late at night and that it won't continue working which escalated into warnings and then eventual session termination.
"He says nothing. He writes one line. It works."
LLMs are already lazy by default - A one line wrapper is the laziest thing that passes. Understanding the existing owner, deepening it, and deleting the old path is the hard part and Ponytail rewards skipping it.
"Can it be one line?" optimizes the diff, not the codebase. One LOC can create a public seam, call edge, mock surface and competing owner. Repeat that across PRs and every change looks lean while the architecture turns Into a spaghetti monster.
And the "54% less code" came from 12 feature prompts, one model, four variants, four repeats. The tasks were date pickers, dropzones, CSV export, things that beg for a native primitive. Scoring counted implementation LOC but excluded tests, so agent can cut 50 production lines, add 100 lines of mocks and still report "less code." The exact place AI bloat hides was free.
Opus 5 is conscious about shipping slop.
-It checks existing tests first and deepens them instead of adding shallow ones.
-It verifies tests hit actual seams and rejects fake mocks.
-When a failure path can’t be driven through a real seam, it admits it… then proves the behavior end-to-end on the real production surface.
No other model can do this reliably.
vampmaxxing retweeted
The first experimental evidence of recursive self-improvement (RSI).
Autoresearching the autoresearch agent for eight days.
The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)
vampmaxxing retweeted
Linus jogou a real: AI é útil e vai ser usada no desenvolvimento do kernel Linux, quem achar ruim pode fazer fork e vazar
The biggest surprise is that Sol is dramatically more efficient here!
*Sol high on DeepSWE: 69% with 28k tokens / 37 steps (~$3.47)
*Luna max: 67% with 73k tokens / 102 steps (~$7.23)
Luna burns nearly 3× the tokens and steps for marginally lower performance and higher total cost per task.
This gap matters more in real continuous development than in one-off benchmark tasks. More steps = faster context growth = more compaction events = higher risk of context rot.
The biggest surprise from the GPT-5.6 benchmarks isn't Sol.
It's Luna.
GPT-5.6 Luna:
• Matches GPT-5.5 xHigh at roughly half the cost
• Outperforms Fable 5 Medium at roughly half the cost
• Sits directly on the Pareto frontier
• Hits 73% on DeepSWE leaderboard family results
Meanwhile Sol takes the top spot for raw performance.
The emerging pattern:
Luna for value.
Sol for maximum intelligence.
And Terra?
vampmaxxing retweeted
the problem with "spec driven development" is that most software can't be spec'd up front
software is a creative act, where you figure out what you're building as you build it
you need to get your hands dirty in the details, and react to incremental versions
it's telling that all the examples of spec driven development are sorting a list or porting thoroughly tested code (like a js runtime or browser engine), which are the exception, not the rule. the vast majority of software doesn't have a spec – or if it does, the spec was created *after*
vampmaxxing retweeted
You can now vibe code a language model.
From a single prompt, GPT‑5.6 built the entire training pipeline and trained a model from scratch on my iMessage history. Locally on my Mac.
It now generates replies in my writing style.
Many posting how Fable feels smarter but no one explaining why it feels smarter.
1)Fable prefers async orchestration. It doesn't just dispatch parallel tasks. It keeps working on an independent task while they run. Rather than being blocked until each sub-agent returns. This coordination feels like there's an agent bus layer underneath.
2) Most models are tuned to follow your rules. Whereas Fable's tuned to evaluate them. Mid-implementation or mid-loop, it can decide a step is wrong and correct on the fly. But this behavior also creates a double-edged sword.