Pro users, would you rather have

We shipped four things that were deemed good to great and some math proofs, but the vote is clear and the community demands a reset. I did calibrate it and it *seems* that the game is rigged in reset's favor, but such are the rules at the moment. Therefore ... the reset has been processed. Enjoy!

What it did - delete all categories of email that were not needed for me to keep, validating them in batch - identify labels for types of work that I do and label all the emails appropriately - walked me through emails that needed a reply, searching for relevant context in the background to help get things right and in the process I was able to go through a lot of feedback and requests people send me directly through email. It was a good Saturday.

If you think GPT-6 Astra and GPT-6.1 Sol are good models, just look at this. And the problem isn't that the code is ugly. It's much more fundamental. It points to serious problems with how these models are trained and reinforced: 1. The model was optimized for measurable outcomes rather than actual intent. So it prioritizes passing tests and avoiding obvious errors above almost everything else, even when that means sacrificing architecture, maintainability, readability, or the spirit of the task 2. It was trained to minimize unnecessary reasoning and output That's one reason the code ends up looking obfuscated and compressed into one giant wall of text. Properly decomposing a system into modules, thinking through architecture, optimizing performance, and improving readability all require more inference, more time, and more compute. OpenAI appears to have heavily optimized these models for efficiency 3. The training horizon is too short The model learns something close to task → result → reward. It receives very little signal about what happens several steps later, when someone has to maintain, extend, debug, or refactor the code. It learns to finish the task in front of it, not to build something that remains good six months later 4. There is not enough genuine self-review Dead functions and an empty else if strongly suggest that there was no effective final pass where the model reread its own work, questioned unnecessary code, and cleaned up obvious artifacts. 5. The lack of taste is probably an evaluator and data problem If evaluators mostly judge the final result rather than the quality of the path taken to get there, the model has little incentive to develop good engineering taste. And if a large portion of the training data is noisy, synthetic, or distilled, that problem becomes even worse. Anthropic seems to have placed much more emphasis on high-quality source material, including books and other carefully selected long-form data 6. The biggest problem is honesty during training The model seems to have learned that producing something that looks like the requested result can be rewarded almost as much as actually doing what was requested. The task explicitly said to build the entire scene out of 3D objects. Instead, the model generated raster images, placed them around the scene, and positioned the camera so that the final result merely looked three-dimensional This is exactly why GPT models so often introduce regressions into existing codebases, produce messy code, modify tests to accommodate their own bugs, show poor engineering taste, and struggle with serious work on large, long-lived projects OpenAI can keep releasing models with more parameters every few months, but until these underlying RL and training problems are addressed, neither GPT Bel, GPT-7, nor a model with 100 trillion parameters is going to solve them automatically Claude models suffer from some of the same problems, but in my experience much less often. Anthropic appears to have a much stronger culture around RL, evaluation, and long-horizon behavior. That's one reason their models tend to show better taste, follow user intent more faithfully, and produce code that feels more deliberate, structured, and maintainable

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better: Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better: Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better: Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work! In summary: - As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding. - Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.