Mallchad 🏴󠁧󠁢󠁥󠁮󠁧󠁿

This argument is getting old. I’ve participated in it far more than I ever thought I would, but here we go again because I’m increasingly convinced that many of the people arguing that LLMs are conscious, or even open to the possibility, just don’t understand how LLMs actually work, so here is context. This is not a debate about understanding what consciousness is or comparing observations to human experience. It’s about understanding the capabilities and mechanics of the LLM, nothing further. First off, how LLMs work is understood. There isn’t magic happening inside the model. The calculations an LLM performs are known mathematical operations. Matrix multiplication, attention, normalization, nonlinear functions, probability calculations, etc. Every operation is defined, implemented, and understood. In theory, you could work through an inference by hand if you were sufficiently masochistic. :-) But what makes it difficult isn’t some unknown computation happening inside the model. It’s the absolutely enormous scale of computations operating across billions of learned parameters. But scale does not magically transform computer software into a self-aware consciousness, but it does get more convincing at emulating it. LLMs learn what they know during a training process. After training their “brain” is effectively locked. During normal inference (using the AI), the model does not continue learning, grow, or modify its weights based on its experiences or conversations. When you start a new interaction with a model, the model itself has no inherent knowledge of your previous conversations with it. None. It doesn’t remember talking to you yesterday because that conversation wasn’t learned into its weights. The model itself is static. The apparent continuity was engineered in the surrounding system. This includes previous conversations, stored memories, accessed summaries, preferences, tool results, and other information can be assembled into context and supplied to the model along with your current input. In other words, the model isn’t remembering the previous conversation. The system is telling the model about the previous conversation again, every time, like it’s the first time. That’s what might sorta resemble short-term memory. The continuity exists in the context supplied during inference, not as new knowledge learned by the model. Long-term memory is also emulated via tool calls and vector databases outside of the model, and then injected into the context. It has to be done this way because the model (its “brain”) is completely static, it can’t change. And there’s a limit. When there’s too much information, some of that context has to be discarded, summarized, compressed, or selectively retrieved to fit within the available context window. Note: A later version of the model can be trained or fine-tuned using data that includes previous interactions, but that’s a separate training process that produces a new version of the model’s weights. The deployed model did not learn that interaction during inference. And between inference calls, the model stops computing entirely. There is no thinking, no ongoing inference, contemplation, consideration, or processing of the passage of time. Its weights simply sit there unchanged until the model is invoked again. Yet somehow, some people decided this is analogous to human consciousness and self-awareness? Maybe their own weights are frozen and they ran out of context window! ;-) And if you think anything I’ve said here is technically incorrect, go research it yourself. The tooling necessary to build the model architecture, train it, and run inference is all available as open source on GitHub. If you want to debate anything I said, go review the source code and point me to the lines where you think the consciousness is.