Jerome Dahdah @ 🇳🇱

Seeing this makes the end-state clearer to me: code that can't be read by humans, run on machines that humans can't access, via inaccessible APIs, trained on data that humans didn't create. LLMs are starting to get really good at manipulating mostly-opaque-to-humans code (see: their improved ability to reverse engineer machine code/assembly). Additionally, folks are accessing more-and-more of their LLMs in cloud-based environments without ever actually interacting with an LLM API (myself included). I think it's going to become increasingly likely that the end state is one in which you ask an agentic company to produce some result and it does it via [some system completely opaque to humans] and produces [some output in a format that humans are unable to interpret]. It makes me wonder if our current state of competing LLMs cross-training on each other will eventually become literally impossible (not just logistically so) when they become fully self-recursive and are producing output that even other LLMs can't understand, let alone humans. I think we're in something of a golden period here where humans are still capable of understanding how things work and how things are run (shell commands! python! node.js!) but it's going to quickly become irrelevant. We're already past the previous high-point where we were able to see reasoning traces for the top-level LLMs. I'm skeptical that open models will ever get to this point as the economics are working against them (I suspect that they will always need human-generated training materials in one form or another). But it does make me wonder if the upper-echelon companies will end up converging into this high-level "black box that produces outputs" singularity of sorts. This has been hitting home for me lately as I've been using Claude Projects in a cloud-based environment where I can't see transcripts, look at the machine it's running on, nor even know what commands it's running. I don't even know what languages its writing its code in. We're just a couple steps away from code being completely irrelevant.