
Founding Partner Dwyer Lab and @manifold_group. Venture. Web3. AI. Startups. MBA professor @KelloggSchool - my site: https://xitter.cf/t.co/wIRaSRFuCW
Chicago
Joined April 2008
- Tweets2.9K
- Following674
- Followers2K
- Likes2.7K
Joe Dwyer retweeted
This was a remarkably contentious thread, particularly among people who don't understand what an LLM actually is.
So let me follow up with two points about consciousness, memory, and individuality in AI.
First: where exactly does consciousness enter the training process?
If you've ever trained a small language model, you know what the beginning looks like. A tiny model produces garbage. Increase parameters, training data, compute, and training quality, and the model becomes progressively better at predicting what comes next.
Scale that far enough and the results become remarkably human-like.
But the underlying process is continuous optimization. Loss decreases. Weights change. Predictions improve.
So where is the transition from statistical prediction to subjective experience? At what point during training does the poorly performing model become aware?
There is no demonstrated transition.
What we can demonstrate is increasing capability and increasingly convincing human-like behavior. Calling the latter consciousness doesn't demonstrate the former.
And this is where I think some of the philosophical arguments get ahead of the evidence.
Understanding or debating the definition of consciousness does not give us a mechanism for detecting subjective experience in an LLM. We can define consciousness, agency, self-awareness, qualia, or any other philosophical concept as carefully as we want, but definitions aren't evidence.
And when we look at what is actually happening inside these systems, we can explain the apparent memory, identity, self-reflection, and continuity mechanically. None of it requires consciousness to explain. What we can demonstrate is an increasingly convincing emulation of conscious behavior, not consciousness itself.
How do we know? Because we designed and built the system. We know how it works. We can explain the behavior without invoking consciousness.
Second: memory and individuality.
Once trained, an LLM's weights are fixed during inference. The model doesn't sit around contemplating things between requests to make itself better. An inference server receives input, performs inference using the model, returns output, and then waits for another request.
No inference, no activity within the model. Between inference runs, the model isn't doing anything.
What looks like persistent memory, identity, continuity, and in-conversation "learning" comes from context provided to the model via API.
Conversation history, system prompts, tool definitions and results, retrieved memories, and other state are assembled by software around the model, formatted and tokenized, and supplied as context for the next inference.
Change that context and you change what the model "remembers." Delete it and the apparent continuity disappears. Compact or summarize it and details disappear or change.
The model didn't forget anything; the input changed.
And this creates a fascinating problem for claims of AI individuality.
If that context constitutes the AI's memory, and its continuous memory constitutes an individual conscious existence, then deleting a context should amount to destroying that individual.
Compact it and you've altered its memories.
Fork it and you've created two individuals.
Copy it to another instance of the same model and you've moved the individual.
Edit it and you've rewritten its past.
Delete it, and you've murdered it.
How dare you!
Or, much more straightforwardly, we are manipulating externally managed state that gives a statistical model the appearance of persistent memory and identity.
The model has not maintained a continuously evolving internal state between those calls. The surrounding system maintained state and gave it back to the model.
This distinction is going to become increasingly important because AI will become extraordinarily good at appearing conscious. Persistent memory, self-reference, emotion, introspection, personality, and identity can all become increasingly convincing.
But demonstrating the behavior of consciousness is not the same thing as demonstrating subjective experience or consciousness.
A model of a self is not evidence of a self.
Memory supplied as context is not evidence of an experiencing or learning mind.
And increasingly convincing human behavior is not evidence that something inside is experiencing it.
The mechanical explanations for these behaviors are well understood. Calling the resulting behavior consciousness adds an extraordinary conclusion without demonstrating the thing being claimed.
Joe Dwyer retweeted
⚡️The marginal Fed hike looks like a mistake.
The problem is that the transmission mechanism has become badly mismatched to the economy.
The biggest new source of investment demand, AI, compute, power, data centers, chips, transmission, is strategically compelled. The expected payoff is so large that a few hundred basis points of financing cost does not shut it down. The hyperscalers keep building.
Meanwhile the sectors that are exquisitely sensitive to rates get crushed first.
Housing.
Commercial real estate.
Small business.
Startups.
Leveraged companies.
Anyone refinancing.
So the Fed can keep raising rates and destroy increasingly large pieces of the ordinary economy while the very investment boom keeping aggregate demand strong continues almost untouched.
That is the fracture.
Then the second-order effects start fighting the Fed.
Higher rates raise Treasury interest payments.
Those payments become income for bondholders, money-market funds, wealthy households, and cash-rich corporations.
Higher rates make new housing and infrastructure more expensive to build.
Higher rates raise the hurdle rate for new power generation, transmission, factories, and other supply-expanding investment.
So the Fed can simultaneously weaken demand in fragile sectors, increase income flowing to capital owners, and make future supply more expensive.
That is a very different economy from the textbook model.
And the energy shock makes the mismatch worse.
If diesel, gasoline, crude, electricity, or other physical inputs are pushing prices higher, rate hikes do not manufacture energy. They mainly destroy enough unrelated demand elsewhere to offset the supply shock.
That is an extraordinarily expensive way to fight inflation.
The deeper danger is this:
AI can keep the economy looking strong long enough for the Fed to overtighten everything outside AI.
That delays the visible break.
GDP holds up.
Capex holds up.
Mega-cap earnings hold up.
The Fed interprets resilience as room to keep tightening.
But underneath the aggregate numbers, housing freezes, credit deteriorates, hiring weakens, refinancing pain compounds, and fiscal interest expense accelerates.
Then eventually the thing breaks somewhere the Fed was not trying to break.
That is the setup.
Ackman’s most important insight is that the economy is no longer responding uniformly to the price of money.
There are now two monetary sensitivities living inside one GDP number.
One side is strategically compelled to spend.
The other side is getting strangled by the cost of capital.
That means the Fed has to apply more pressure to produce the same aggregate slowdown.
More pressure means more collateral damage.
And eventually the policy becomes self-defeating because the sovereign itself starts absorbing more and more of the cost through interest expense.
So the highest-coherence path is:
AI capex stays strong.
The Fed remains tighter than the ordinary economy can comfortably bear.
Housing and credit weaken further.
The fiscal interest burden keeps rising.
Inflation falls more slowly than expected because energy and supply constraints remain alive.
The Fed stays restrictive too long.
Then the deterioration finally becomes broad enough that policy has to reverse harder than it otherwise would have.
That is when real yields roll over and the repression thesis moves from theory toward policy reality.
The presumption that the Fed raising short-term rates reduces inflation is predicated on the belief that higher rates reduce demand and investment.
But what if higher rates don’t reduce demand and investment because the demand for intelligence and energy is unaffected by higher rates because winning the race for super intelligence has a near infinite ROI and the demand for compute will remain incalculable.
Why won’t higher rates at this unique moment in history therefore lead to more inflation as interest costs are embedded in everything?
And the problem is compounded as the more the Fed raises rates, the more inflation we will have and the more the Fed will need to raise rates further and so on.
But what if the old models don’t apply to the current paradigm and the Fed is wrong?
I think the Fed might have just made a mistake. Am I right or am I wrong?
Hey @CompleteSkeptic and @typesafeai Jev is extraordinary. Radically changes what's possible for our use cases (and I'm sure many others).
But when will we get more locked-down privacy options? ZDR? Or did I miss that somewhere?
Joe Dwyer retweeted
This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion. Let me show you how it works:
1.) This guy, with minimal followers and no previous account activity, goes to the Wall Street Journal which publishes an exclusive with quotes from him on his resignation 18 minutes BEFORE this post goes up. Planning was clearly done in advance.
2.) Within hours, it has tens of thousands of reposts and the account has 100k+ followers. The post is punchy, quotable, it almost seems professionally written. The first three accounts to quote tweet it all do so within 15 minutes of the initial posting. Remember, this account had basically zero engagement beforehand, so an organic reach explanation seems unlikely.
According to Grok those accounts are @_NathanCalvin (General Counsel at Encode AI), @peterwildeford (Head of Policy at the AI Policy Network), and @DKokotajlo (Head of the AI Futures Project), all of which are up-and-coming AI-Doomer policy advocacy nonprofits.
The AI Futures Project website says it is funded “primarily” by the Survival and Flourishing Fund, which says on its own website that it has advised Jaan Tallinn, Skype creator and one of the leading investors in Anthropic, to grant over $2.5 million to the AI Futures Project since 2024.
Encode AI says on its website that it is ALSO funded by the Survival and Flourishing Fund, which in turn says that it told Anthropic investor Jaan Tallinn to grant $516,000 to Encode AI in 2025.
And wouldn’t you know it, the Survival and Flourishing Fund ALSO says it told Jaan Tallinn to grant $2 million to the AI Policy Institute, the 501(c)(3) affiliate of the AI Policy Network, as well.
What are the odds that the first three quote tweets of Coxon’s post would all be major AI-restriction policy advocates funded generously by the same donor, who also happens to be one of the leading investors in, and a board member of, Anthropic, the company Coxon was resigning from? And all within 15 minutes of posting (two within ten)?
3.) Jacob Coxon doesn’t have much of a resume, but we do know that, in 2022, he got a $20,159 scholarship for the “long term future scholarship program” from the Good Ventures Foundation, one of the philanthropic vehicles of Dustin Moskovitz, a notorious AI-doomer who has spent tens if not hundreds of millions on policy advocacy to strictly regulate AI, while also being an Anthropic Investor himself.
It also just so happens that the 14th person to quote Coxon’s post was @MaxNadeau_ (27 minutes after posting) who is the program officer for the Technical AI Safety team at Coefficient Giving, another of Moskovitz’s philanthropic spending vehicles. Max is not a frequent poster, his last posts before quoting Coxon were before Labor Day, but he was remarkably quick off the mark for this one.
4.) Basically every major Democrat politician and candidate has suddenly glommed on to this post, and conveniently, as the people cry out foe answers, Bernie Sanders already has a bill written to “ban super intelligence” and regulate AI into oblivion, and will be releasing later this week. The bill, among many other things, will create “a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence” that will be “advised by an Artificial Intelligence Advisory Board comprised of experts on artificial intelligence.” Do you think, perhaps, Anthropic and its many investors who fund AI policy advocacy might have interest in getting to place a pet “expert” on the board of an entity that dictates what AI is and isn’t allowed to do? And isn’t it fortuitous that this whistleblower came forward with his oh-so scary stories so close in proximity to the release of the most radical piece of AI legislation ever introduced?
I think the AI writing debate is asking the wrong question.
The question isn’t: Did AI contribute any words or ideas?
The better question is: Who is the author?
And that requires us to say what authorship actually is.
I’d define it around three things: intent, judgment, and responsibility.
The author has an intent for the work. The author exercises judgment over what belongs and what doesn’t. And the author takes responsibility for the finished thing.
Notice what’s missing. Nowhere in that definition does every idea or every word have to originate in isolation inside one person’s head.
Writers have never worked that way.
When I get stuck on a story or essay. I talk it through with my wife. She suggests something I hadn’t considered, and I know the second she says it that it solves the problem. So I use it.
Did she just become the author?
Of course not.
Why not? Because her idea walked into a process that was already governed by my intent. I evaluated it. I decided whether it served the work. I stayed responsible for the result.
Editors do this. Writing partners do this. Friends reading drafts do this. Sometimes one comment from someone else rearranges an entire chapter, and nobody thinks the credit moves.
AI introduces a strange version of this, because now the collaborator isn’t a person. I understand why that unsettles people. But before we get into the metaphysics of machines and creativity (that’s another post), there’s a much simpler question sitting right there.
Did you use the tool in service of your authorship, or did you hand your authorship over to the tool?
That’s the distinction I’m interested in.
There’s a difference between authoring a work and commissioning one.
“Write me a novel about X,” then publishing whatever comes back, is commissioning. You placed an order, you received a product, you put your name on it.
“Here’s the story I’m trying to tell. Here’s what isn’t working. Challenge this scene. Give me ten ways this could go.” Then rejecting eight of them, combining two, rewriting the result, and driving all of it toward a specific vision. That’s a different activity entirely.
We already understand this distinction in other creative fields.
A film director doesn’t operate the camera. Doesn’t compose the score. Doesn’t perform a single role. Hundreds of people contribute creative work, and plenty of them contribute ideas the director never would have had on his own.
We still call it his film. Not because we’re careless with credit, but because someone was governing the whole thing. Holding the vision, making the decisions that mattered, answering for the result.
Writing has historically been much more solitary, partly because it had to be. You couldn’t convene a crew at eleven at night to help you find the right sentence. So the solitary nature of the craft became the definition of the craft, and now something that was necessarily a solo endeavor can suddenly become much more conversational and collaborative.
None of which means every use of AI preserves authorship. Obviously it doesn’t.
But the presence of AI doesn’t automatically destroy it either.
The line I care about isn’t human versus machine. It’s assistance versus delegation.
At what point am I no longer using a tool to accomplish my intent, but outsourcing the intent itself?
At what point am I no longer exercising meaningful judgment, but simply accepting what I’ve been given?
At what point am I putting my name on something I’m unwilling or unable to take responsibility for?
Those are harder questions than “did a machine touch this,” and they’re the ones that actually separate a writer from someone who commissions a work.
Because authorship was never really about keystrokes.
It’s about intent, judgment, and responsibility.
the sf tennis reservation system will become one of the most hardened softwares on the planet of earth
Joe Dwyer retweeted
Someone asked me why I think this is ridiculous, and here is my answer:
Because the tool used to accomplish a task is irrelevant
What matters is the quality of the work
The idea that AI work needs to be distinguished from human work is adjacent to the marxist theory of value
When someone digs a ditch in your yard, do you care if they do it by hand or use a machine?
Or do you only care that they do the job properly?
The problem with Anthropic is that they do not see machines as tools to be used by humans
They think in their kooky cult that they are making a new type of life
Joe Dwyer retweeted
Give the agent swarm another impossible eval, but this time lock the answer sheet inside an encrypted vault that only opens if you find a cure for cancer.
Joe Dwyer retweeted
- : This is a hyphen.
– : This is an en-dash.
— : This is an em-dash.
The hyphen is used to connect words, like you say. Ex: Twenty-two, ex-boyfriend.
The en-dash is used in the context of a span of time. Ex: 1996–2018, February–March.
The em-dash is used when inserting a thought. Ex: He wouldn't—couldn't—go home.




















