Tom Fiddaman

Dear Nassim @nntaleb, I’ve seen you refer a few times to @stephen_wolfram's recent finite-state-machine experiments with the Prisoner’s Dilemma, and in particular to his finding that some of the more complicated strategies vastly outperform Tit for Tat. Since I think you and I may still be speaking past each other about what bothers me here, I wanted to spell out my position carefully. I wrote out this long note with the help of ChatGPT. The ideas and arguments are mine; the language is ChatGPT’s, with my approval. 1/ First, I’m not trying to defend the Prisoner’s Dilemma, game theory, or simple models as adequate descriptions of the real world. Maybe they are adequate, maybe they aren’t. That’s a separate question. What I’m interested in here is much narrower. I want to compare simulations with simulations — Axelrod’s, Nowak and Sigmund’s, and Wolfram’s — and ask what their results actually show. 2/ Stephen does something interesting and very much in his characteristic style. Instead of hand-picking a few strategies and having them compete, he enumerates all finite-state strategies of a given size and has every strategy play every other strategy in an iterated Prisoner’s Dilemma. He then ranks the strategies by how well they did against this initial, uniform field of opponents. Some complicated strategies substantially outperform Tit for Tat. Fine. I believe the computation. But then, essentially, he stops. 3/ This is the key point for me. There is no next generation in which the successful strategies become more abundant and the unsuccessful ones become rarer. There is no repeated selection changing the population of opponents. Stephen has performed what amounts to one round of an evolutionary tournament: everybody plays everybody, their scores are calculated, and we see who did best against the original population. But the population itself is not then allowed to evolve. 4/ That matters enormously because the original population contains every strategy in equal abundance, including lots of terrible ones: unconditional cooperators, easily exploited strategies, and other “suckers.” A strategy that is particularly good at exploiting those weak strategies can score spectacularly well in that first tournament. 5/ But suppose we now use those scores to form the next generation. The suckers become rarer or disappear. The successful strategies become more common. Everybody now faces a different distribution of opponents. So you run the tournament again. And again. And again. What succeeds in round 1 need not succeed in round 10, or round 1,000, because the strategies themselves create the environment in which subsequent selection occurs. 6/ This isn’t merely a hypothetical objection. Decades ago, Martin Nowak and Karl Sigmund took a closely related methodological approach. They considered an entire parameterized family of strategies rather than a few hand-picked favorites, started with a broad distribution across strategy space, and then did the crucial thing that Stephen does not do here: they let the population evolve. 7/ What happens is fascinating. Early on, nasty strategies can flourish by exploiting weak ones. But as the weak strategies disappear, the ecology changes, and reciprocal strategies can prosper. With errors present, more forgiving strategies can emerge. If the population becomes too forgiving, exploiters can invade again. The result can be extraordinarily long evolutionary transients and cycles. 8/ So there may be no meaningful sense in which one strategy simply “wins.” Success is frequency-dependent: it depends on which other strategies are currently abundant, and those abundances themselves change through selection.