Comments like this are so confusing to me.
What was 'consulting the previous behaviors of other evil people'?
Was it this part?
"Nevertheless, in 2006 N Korea got nukes and...what's the problem? Have they used them? No. And no one else has either."
If so, I guess I'm not seeing it.
Must this statement -- even in context -- necessarily be read that way?
Replying to @michaelshermer
Unfortunately that's blind empiricism and gives the wrong answer. Humans are people, and people, however evil, have reasons, objectives and ideologies. In deciding how to act, they consult those, not the previous behaviours of other evil people.
3
878
This response doesn't quite work for me. Indeed, I'd argue it relies on a bit of a word game.
Consider an LLM solving the Navier-Stokes equations. Surely that particular problem wasn't in its training set. So, plugging that into Chomba's own equation:
R = ||T - D|| / ||D||
The runtime task (T) includes solving the Navier-Stokes problem, while the training set (D) does not. Therefore, (T-D) is nonempty, and R is greater than zero.
By his own equation measure, the LLM is demonstrating intelligence.
This all makes sense since solving Naiver-Stokes is clearly not mere mimicry.
I'm guessing the response will be something like:
"I said class of problems. Sure, the Navier-Stokes problem wasn't in the training set, but I'm going to define the class as 'mathematical proofs.' Surely there were mathematical proofs in the training set! So Navier-Stokes is just one more example of a proof. See? No intelligence."
But that's precisely the problem. You can always define the class of problems broadly enough to make almost any novel achievement look like an instance of something the system has already seen. You can easily do that to humans as well.
At that point, the equation isn't measuring intelligence. It's measuring how narrowly you choose to define a problem class.
You can literally do this with anything. At that point, the definition has become completely adjustable (i.e., easy to vary). There is no longer any substantive question at stake. We're just arguing about what terminology to use.
The Navier-Stokes problem still got solved by something we're now defining as 'unintelligent'. And apparently we need a new word for the kind of 'unintelligent' thing that can solve a major mathematical problem that stumped some of the world's most intelligent mathematicians for nearly a century.
Me, I just call that something intelligence and I'm done.
Replying to @ChombaBupe
Chess engines are amazing but still not smart.
What qualifies as smart?
Consider a system X, exposed to class of problems D during training & that can solve T class of problems at runtime.
The set R = ||T - D|| / ||D||
gives you some measure of intelligence.
4
1
1,081
"Fully autonomous driverless cars are harder than most people realize, because there are just so many outlier cases."
I remember taking one of the first university-level courses on driverless cars as part of my Master's program. It was taught by Sebastian Thrun, one of the leading pioneers of autonomous driving. He led the Stanford team that won the 2005 DARPA Grand Challenge and later founded Google's self-driving car project.
I remember having a sudden epiphany during that class: Oh my gosh, we're not even close! We won't have truly driverless cars for many decades unless we deliberately build infrastructure specifically for them.
Then driverless cars kept improving, and I kept hearing about more and more successes. I started wondering if maybe I had been wrong all along. But I remained skeptical.
I still think this is a remarkably difficult nut to crack. We can get these systems to work in an enormous number of situations. But universal driving requires universal explainership. (In other words, general intelligence.)
If you want driverless vehicles that work consistently, we'll have to build infrastructure to allow that to happen.
🚨 Huge Breaking news: “California’s Department of Motor Vehicles ordered [Waymos] off the state’s freeways, citing an “imminent hazard to public safety” after 3 Waymo cars entered a closed construction site.
This very much confirms what I have been saying back to at least 2016: fully autonomous driverless cars are harder than most people realize, because there are just so many outlier cases.
We are still years away from full self-driving.
nytimes.com/2026/11/17/us/wa…
This is a problem that almost no one talks about, and I'm struggling to explain it without people jumping to the wrong conclusions.
Clearly, LLMs are getting way smarter at a startling pace. So there's this group, AI doomers in particular, who believe intelligence is one-dimensional. They're convinced that as LLMs keep getting smarter, we must be approaching AGI, and we're all doomed.
Then there's another group, CritRats in particular, who seem to believe intelligence is zero-dimensional, or binary. You either have it or you don't. So none of this progress tells us anything meaningful about general intelligence.
The real story is that LLMs were a huge breakthrough. Yes, they even represent a significant step, or at least a stepping stone, toward AGI. If nothing else they taught us the universality of human language.
But then a tremendous amount of effort shifted toward making LLMs useful through increasingly sophisticated harnesses.
We've certainly continued training better models, and that's a very good thing. But much of the impressive progress has come from training them on specific, verifiable tasks like mathematical proofs and coding, where we have known correct answers to train against. This has been huge in terms of making LLMs genuinely useful. But it does not make them 'more general' or the like.
Then there are reasoning models, which essentially tell LLMs to stop and think things through before answering. Again, a huge improvement in practical usefulness, even if it doesn't necessarily represent a fundamental improvement in the underlying model.
But LLMs are not true general intelligences. And I don't think we're on a trajectory toward making them so. Likely there is just no way to do so because they are the wrong kind of intelligence from the outset.
That doesn't mean they aren't incredibly important. You don't need true general intelligence to disrupt the world. In fact, there may be considerable advantages to having powerful intelligences that aren't general intelligences: they're essentially robots that, for the most part, just do our bidding. And perhaps it's even better that way for many tasks.
Here's what I think people are missing: We can keep improving LLMs indefinitely, discovering one incredible new application after another, with one breakthrough after another, without ever actually attempting to create true general intelligence.
There is an enormous amount of unexplored territory in this kind of intelligence. We should explore it!
Even if we eventually invent true general intelligence, there will still be enormous value in continuing to improve LLMs as tools. We'll probably be improving them forever. This would be possible even if (as I'm guessing) they aren't a path to human level intelligence.
Intelligence isn't one-dimensional. There is plenty of room for extraordinary progress even though we're not yet on a trajectory toward inventing general intelligence.
My contrarian take is that LLMs will not get any "smarter" than they are today.
They will get more efficient, more capable (plugins / memory / context / skills), more specialized,
but we're at the peak of artificial intelligence.
And we can see that. The last several months of new releases are not that different from the last.
It's like the iPhone - it came in and disrupted everything. But now that we're at the 10th+ iteration buying the brand new phone isn't that much of an upgrade to justify it.
David Deutsch vs. Cox's theorem.
In Increments 74, Deutsch rejects Bayesian belief. Instead of Bayesian credences, he argues that we judge our confidence in ideas by how well they've survived criticism, preferring the theories that have survived criticism best.
This gives us a perfect test case: a hard-core critical rationalist offering an alternative to probability theory and Bayesian credences that we can put directly to the test against Cox's theorem.
But does his critical rationalist alternative actually escape the implications of Cox's theorem? Or does Cox's theorem still have something to say about Deutsch's critical rationalist account?
Time to find out.
open.spotify.com/episode/2fy…
I've had this podcast idea bouncing around in my head ever since Increments 74 (the interview with David Deutsch) came out in late 2024.
At the time, I hadn't yet worked through Cox's theorem or studied Bayesianism in detail. (I don't think I'd even heard of Cox's theorem yet!)
Still, I immediately sensed the significance of the issue, but kept wondering whether I'd missed something. So I waited until I'd brushed up on all things Bayesian, confident that I finally understood the problem. That's when I realized what an elephant in the room Cox's theorem really was for Critical Rationalists. But it also helped me realize what the real problem with Cox's theorem actually was. (More on this in the future.)
I'm very excited to finally put this one out for discussion!
2
2
223
Bruce Nielson retweeted
Computers really speed up computer science research, so we've had recursive self-improvement since at least the 50s.
Agentic code as a machine learning problem.
Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached.
The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network.
This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc.
I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
176
I hear this argument all the time: neural network-based AI "only interpolates," and therefore can't discover anything genuinely novel because it's just looking backward.
Chollet explains what's wrong with this argument.
The model doesn't have to discover novelty through interpolation. It can use what it has learned as a guide for search.
And search is precisely what intelligence is all about. A search is, after all, just variation and selection.
Replying to @ChrSzegedy
You're too smart for this, you know better. Obviously a differentiable curve fitted via SGD can only interpolate. That's true by construction. But if you use this curve as a sampling guide in a search process, then you're doing search, you're no longer interpolating at all. Theorem proving is search. Reasoning chain generation is search. Etc.
This is a conceptualization problem.
If you conceptualize a "multiverse" not as "doing two things at once," but rather as "there is no later collapse to a single universe" (which is how a collapse-QM advocate would conceptualize it), then you can easily end up thinking the burden of explanation is on the MWI defender to explain why there isn't a collapse, rather than (more correctly) on the collapse theorist to explain why there is one.
But I think the better conceptualization is to see non-MWI as equivalent to saying: "The multiverse disappears for no reason I care to specify. It just collapses for reasons unknown."
Once you frame it that way, it's a lot easier to see who actually had the burden of explanation all along.
Peter and @bnielson01 did a great podcast, combining several fascinating topics: philosophy of probability, epistemology, and niche theological arguments.
The argument for model-dependent probability is excellent. It’s a tour de force.
podcasts.apple.com/nl/podcas…
I remember reading an article when I was a teenager about how difficult it was to demonstrate that computers were actually producing measurable productivity gains. The famous observation at the time was that computers seemed to be everywhere except in the productivity statistics. The article blamed much of this on how difficult software was to use.
I also remember having lunch with a CIO in the late 1990s who couldn't even imagine why anyone would want to browse the Internet or buy things on their phone.
I think there's a bigger secret here: we can only translate technological advances into measurable productivity gains at a certain pace.
For context, U.S. labor productivity has historically grown at roughly 2.1% per year since 1947. Think about what that number includes: computers, the Internet, robotic automation, and just about every other major technological advance we've made. All of that, combined, is reflected in that roughly 2.1% annual gain.
My guess is that we'll continue to see something like that long-term pace of productivity growth, but increasingly powered by AI. The technology may be revolutionary, but the economic gains will take time to work their way through the system.
And we may only really appreciate how transformative AI was when we look back at the productivity statistics years from now.
I'm sure that's not what everyone was hoping for. We were hoping for a giant, sudden boom in productivity. But perhaps that's just not how technological revolutions show up in the statistics.
the most replicated result in generative AI has been replicated again: productivity gains are minimal or hard to document
4
337
Interesting. If we measure returns in terms of dollars spent, LLMs are showing diminishing returns.
That may not be how it feels to us users, though.
1
4
370
Perhaps the most obvious disagreement between Critical Rationalists and Bayesians concerns the Bayesian use of what are called subjective probabilities.
Technically, the term has a perfectly defensible meaning. But in practice, it has come to mean something rather different to many Bayesians: think hard about your theories, maybe write down a few criticisms, and then assign probabilities to them.
I call this misuse of subjective probabilities subjectivism, and I think it's a real problem.
But do Critical Rationalists actually escape the problem?
We start by examining a CritRat argument against Bayesian subjectivism. But then we run into an uncomfortable question: Critical Rationalists themselves acknowledge that we often, perhaps even usually, have to rely on gut feelings as part of what they call inexplicit knowledge.
And what does that often look like in practice? More or less the same process: think hard about your theories, write down a few criticisms, and then go with whichever theory feels strongest.
The only obvious difference is that Bayesians assign a number to that feeling, while CritRats don't.
So how, exactly, do these two methodologies differ?
Then we bring in Cox's theorem, which establishes a rather provocative connection: your gut is already doing the math. And that math is the probability calculus.
So how is this really different from Bayesians assigning their own probabilities? Are CritRats perhaps relying on something much closer to Bayesian reasoning than they realize?
Listen to the episode:
open.spotify.com/episode/52I…
National Review has an excellent conservative analysis of why Republicans lost the election by such wide margins.
nationalreview.com/2026/11/t…
1
4
383
My kids told me about an apparently explosive Internet meme built around a reimagining of Harry Potter in which Malfoy is a girl, and her obsession with Harry is really a tsundere crush on him.
I was curious, so I Googled it and checked out a few examples. It's all surprisingly high-quality for fan-made content. The kind of thing that would have been impossible before the AI age.
But the key phrase there is "for fan-made content."
That is, until I came across this one:
youtube.com/watch?v=j_Aq2MTV…
Holy freak! How in the world did this guy (Aiden Guo) manage to coax this out of AI? Did he spend hundreds of hours just retrying things until he got it right?
If this scene had been in the original movies, it would easily be the single best scene in the entire series. The kind of moment you keep rewinding to watch again and again.
For those not fully familiar with Harry Potter (all two of you), Harry's childhood bully, Malfoy, doesn't ultimately join the Dark Lord. But he doesn't exactly join the good guys either. He and his family just slink off into the night.
In this gender-bent version, we get to witness the moment Draconia Malfoy finally switches sides.
A little background: The Patronus Charm is powered by your ability to summon a genuinely happy memory. That should be enough context to appreciate this short, three-minute story.
2
3
807
According to Spotify, we released our very first episode on February 29, 2020. (Yes, a leap day!)
On December 23, 2024, we released our 100th episode, featuring an interview with David Deutsch.
And before the end of this year, we'll be just shy of 160 episodes.
That's nearly seven years of podcasting, 160 episodes, and countless hours of conversations about ideas, explanations, philosophy, science, and knowledge.
A true labor of love. ❤️
Yes. Absolutely.
The Republican slaughter was entirely about AI safety.
It was the top issue on every voters mind.
They just wouldn't shut up about it.
Let me say this as a conservative:
One big reason the Republicans were utterly slaughtered yesterday is that Republican leadership decided it was fun to simp for Silicon Valley, and it was not important to protect humanity from AI.
Every time Republicans mocked legitimate concerns about AI safety, they lost voters.
Every time they sided with hyperscalers building AI data centers over local communities, they lost voters.
Every time they repeated the AI accelerationist promises about 'Abundance', 'Universal Basic Income', and the 'Post-Singularity', they sounded like economically illiterate fools and charlatans.
And every time they promised that 'AI will radically transform society in ways that we can't predict, but that will be wonderful beyond your wildest dreams', they sounded like juvenile communist revolutionaries, rather than genuine conservatives who understand tradition, stability, continuity, and Chesterton's Fence.
Here's the choice Republicans face now: either
(1) learn the bitter lesson from this election, and take seriously the public's intense, growing, and valid concern about the risks of AI, or
(2) stay locked out of any effective political power for the next several election cycles.
2
7
700












