Founder @CohesiveSystems, Prev: Co-founder/CTO @AlvysTeam, First engineer → VPE @JetTechnology acq. by @Walmart
Del Mar, CA
Joined May 2008
- Tweets4.4K
- Following1.3K
- Followers1.6K
- Likes25K
Leo Gorodinski retweeted
Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
Leo Gorodinski retweeted
The industrial-benchmark complex is on thin ice
Nice DeepSWE score! Does it survive a lowercase filename?
Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.
Models clustered around 74% on DeepSWE end up 23 pp apart.
Good at coding, or good at DeepSWE? 🧵
Leo Gorodinski retweeted
Meet TLAPS-Bench: A new benchmark of 265 real-world #TLAPlus proof targets to push AI agents to crack machine-checked correctness proofs using #TLAPS
44.3% rate at pass@3
46 bugs found
Paper: arxiv.org/abs/2611.02849
Code: github.com/specula-org/TLAPS…
lnkd.in/p/g3ET9E8H
Leo Gorodinski retweeted
I'm building a new programming language, Zed (once it's released I'll be posting about it at @ZedLang).
I have been working on this language for over 20 years, but never got it built, because the scope of it is so ambitious. It's an entirely new type of language based on category theory, with core design objectives of precise automatic parallelization up to the Amdahl limit; automatic memory management, with no GC and no need for manual malloc/free; and pervasive use of partial evaluation to increase the power of the language, among other very unique features, such as units (`accel = 5 with units("m s^-2")`) as part of the type system.
I gave Opus 200 pages of design notes that I have been compiling over the last 20 years, and the code I had already written, and asked Opus to compile everything I have created so far into a single cohesive and detailed plan. Then I went through many rounds of cross-review between Claude, ChatGPT and Gemini, having them all do critical reviews, then meta-critique each other's critiques, then collaborate on combining their critiques into an improvement plan, until convergence. I also had the agents cross-check every part of the spec with every other part of the spec (all-pairs comparison) to check for inconsistencies, gaps in the spec, etc. Then I had the agents turn the spec into a long series of tiny incremental steps (a topological ordering of what has to be built before what), as well as the DAG-structuring of dependencies underlying the topological ordering.
I then had Opus build a visualizer that shows me all the tasks in the DAG as my coding agents are working on each task, with realtime update as tasks change status (in progress / completed), which I highly recommend when you ask Opus to do some mammoth complex task, because you can see exactly what it's doing and when.
It is absolutely magic to see this coming together before my eyes, and to see how Opus is handling this massive DAG of complex tasks and their interdependencies.
It is crazy to me that Opus is now simply building what I have dreamed of and wished would exist for over 20 years, but never had enough waking hours to implement! I now have an army working for me, completing what I couldn't due to the bandwidth limitations of having just one brain, and 24 hours in a day.
What a day of miracles we live in.
Leo Gorodinski retweeted
AI may well be able to write massive software projects “from scratch”, but that almost certainly isn’t the fastest / highest performance / highest security / most token efficient way to do things. Good building blocks matter.
In addition to RL’ing the agents on final task loss, we should try to get them to regularize the building blocks as well. What would the standard library look like that makes the agents most effective?
It is likely at the limit that it turns out to be some alien organization of code, but given the existing human-bias in training, I bet that in the near term it would be lovely code that old programmers would appreciate.
I suspect the same could be said for mathematics, but that is out of my lane.
Leo Gorodinski retweeted
When I was writing Unreal Engine 1, my time was divided as follows:
- Around 30% went to seriously deep coding on hard problems: assembly language texture mappers, C++ BSP trees, data persistence, object oriented systems design, and networking.
- Around 70% was just typing and refactoring large bodies of code needed for a complete engine feature set. This was not hard, and while I was doing it my mind was mostly on the deeper coding topics, thinking and planning for what's next. It didn't feel much different from vibe coding today, except that it was just me typing code on an IBM Model M keyboard rather than prompting it and reviewing the results.
So, it has always seemed perfectly logical that we would rely on AI for that 70%, once the AI could do it with a similar quality standard as we could, which has never been perfection. It came as no surprise that, after a few years of AI model evolution, it was able to take the reigns on that 70% and free up programmers' time for more valuable tasks.
What's more of a surprise is the emergence of AI as a sort of insta-Carmack able to solve hard technical problems quite cleverly and often cleanly, and that it's able to reason about and work in complex 3D scenes both by operating the tools and by directly understanding the raw data files.
I abandoned assembly language in 1998 when out-of-order CPUs made much simpler C/C++ code faster, and never I missed it. If prompting through architectural documents, concept art, and design plans can transcend typing code and tweaking vertices, that will enable developers to spend their time on higher-level, higher value technical and creative tasks.
What matters is the end product, whether it's a game or an engine. Anything developers can do to try out more ideas and make better software and games faster is a win for everyone.
Leo Gorodinski retweeted
Agents make code cheap. Understanding the system is still the bottleneck.
Cohesive is a language for describing systems: their rules, workflows, and guarantees.
People and agents work from the same model. Machines check and compile it.
Here’s a brief introduction.
Leo Gorodinski retweeted
Kevin Buzzard on why he is excited about math in the age of AI:
xenaproject.wordpress.com/20…
Leo Gorodinski retweeted
Code reviews today are mostly about gut checking if the complexity is worth the benefits.
Leo Gorodinski retweeted
"openai engineer explains /why/ he didn't need to understand the kernel line by line" 🙃
we're doing compilers 2.0
Leo Gorodinski retweeted
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
Leo Gorodinski retweeted
Have been updating my "physics of AI" blogs every day. Requiring only 2 hours every day, I learn surprising facts about neural networks via toy models. Many insights might be trivial or irrelevant in the end, but some will be huge and transform the field.
kindxiaoming.github.io/blog/
Leo Gorodinski retweeted
In some important ways, a user’s LLM chat history is an extended interview. The social media algorithms learn what you like, but chats can learn how you think.
You should be able to provide an LLM as a job reference, just like you would a coworker, manager, or professor. It can form an opinion and represent you without revealing any private data.
Most resumes are culled by crude filters in HR long before they get to the checking-references stage, but this could greatly increase the fidelity. Our LLM will have an in-depth conversation with your LLM. For everyone.
Most people probably shudder at the idea of an LLM rendering a judgement on them, but it is already happening in many interview processes today based on the tiny data in resumes. Better data helps everyone except the people trying to con their way into a position, and is it really worse than being judged by random HR people?
Candidates with extensive public works, whether open source code, academic papers, long form writing, or even social media presence, already give a strong signal, but most talent is not publicly visible, and even the most rigorous (and resource consuming!) Big Tech interview track isn’t as predictive as you would like. A multi-year chat history is an excellent signal.
Taken to the next level, you could imagine asking “What are the best candidates in the entire world that we should try to recruit for this task?” There is enormous economic value on the table in optimizing the fit between people and jobs, and it is completely two-sided, benefitting both employers and employees.
Leo Gorodinski retweeted
Can Being Really Emerge from Nothing? Hegel’s Science of Logic Explained
Full episode with @UrsSchreiber link: youtu.be/GO2UnQUHfu8
#philosophy
Leo Gorodinski retweeted
Evaluating the Infinite
🧵
My latest paper tries to solve a longstanding problem afflicting fields such as decision theory, economics, and ethics — the problem of infinities.
Let me explain a bit about what causes the problem and how my solution avoids it.
1/20
