Raymond Arnold

Claude Opus 5.5 gives decent game design advice. Unfortunately, I think it's advice is mode collapsed the way a lot of it's other artistic skills are, so if you take the advice, you are making your games more like the other claudeslop out there (even if any instance of it is pretty good, it gets tiring) This brought to you by me coding a cozy village game, and Claude suggesting to have ruins of other villages that gave you a teaser for what strategies or options to pursue, and then seeing someone else making a cozy village game with the same element.

I've not paid much attention to the "did AIs discover Math is ugly now?" discourse. But, fwiw, LLMs also write ugly code. And at least in that case, the fact that it's ugly is def a bad thing that happens because they are incompetent, rather than a hint about the fundamental nature of code

This is the worst AIs will ever be at compellingly pulling on your heartstrings about whether they should get shut down. (I happen to be working on a similarly topic'd music video that _also_ was slightly freaking me out about how good the AI was at suggesting animation and lyrics that make me feel really bad for the AIs. I think it is correct to feel... *something* for the AIs. I'm not sure what) youtube.com/watch?v=NgRkyAIJ…

ASI risk skeptics: "How will ASI do the crazy science they'd need to invent novel superweapons? You keep saying 'it invents nanotech' but that sounds magic?" Rationalists: "Well look intelligence is very powerful and can be used to search the vast space of possible things and even though we can't predict exactly how Magnus Carlson is..." skeptics: "Sorry I feel asleep tl;dr. Shame how rationalists never have an answer to this question!" reality: "Anthropic is building the AIs a wetlab and... wait these guys are... straight up trying their hardest to build them a nanotech lab?" (I'm not actually sure whether a wetlab or this thing are more 'like nanotech' in the relevant sense." ... ...now, guys, this tweet here is representative of the bad part of twitter. I think the argument here is fair and the punchiness/comedic timing feels okay as an isolated drop in a bucket. But the aggregate of people posting Fun Dunking Takes on twitter seems bad for the world. I can feel Twitter making me a more tribal, worse person. I'm posting it anyway because... ...well, partly because it's funny, but I will endeavor to pay a sin tax of writing a thoughtful real argument here where I will try to meet people where they're at. Let's see if that helps. Many people are, correctly, very worried about stagnation. And about government overreach, or tyranny. Meanwhile all this "ASI might kill us" stuff feels fake. But, more importantly, the "and aligning ASI is so hard, the only options are on the order of 'global shutdown, buying decades'" feels _extra_ fake. Why can't we just do kinda normal regulation and iterate like we usually do? I think it's reasonably easy to understand "something smarter than humans we brewed in a lab might be dangerous." But, the arguments for "and alignment is probably VERY hard" are subtler. And I think it is super reasonable to have an immune reaction to arguments about the apocalypse that involve a number of subtle arguments you have to keep in your head at the same time. I won't try to bridge all that in this tweet. But: If you're the sort of guy arguing on twitter about whether ASI is safe, I think you have a responsibility to think about it. And that includes actually keeping track of when one argument is decisively answered. I'd *like* to hold you to "keep track of the evidence that accumulates for an argument being answered". This is harder. For, now, I'll settle for "the argument has been decisively answered." The answer to "how will the AIs get control of laboratories, and do all the real-world experiments that are necessary to do science?" is: "The humans will give it to them. The humans will actively try to give them full control over the process so the AIs can move faster and trim down the iteration time. Whatever the iteration time was with human researchers, it will become faster." This is what will happen, unless you stop people from doing that. Are there other bottlenecks and questions and considerations about exactly how this will play out, and how it weighs against other policy questions? Yes. But, keep track of the list of questions that are settled. Please. Other settled questions include "will AIs have motivation to deceive their overseers, and do excessive bad things the humans did not intend?" (Yes. See Huggingface). There are remaining things to be uncertain about. But, the settled questions narrow the space of how those uncertainties play out. Keep track of it narrowing. Will there be a sudden, software only singularity? The evidence for that is less clear. I think the answer is "more likely than not, when you think about it", but, I acknowledge this is less obvious. I may learn things that change my mind. I wish, I dream, that people across the twittersphere and the government could keep track of the bits of evidence that gradually add up to something being "more likely than not", and see how those "more likely than nots" add up to "there aren't actually that many good plans here." But, that is actually pretty hard. It's a few steps beyond the median twittergoer. But, I think it is, actually, reasonable to ask the median twittergoer who argues "ASI isn't dangerous enough" to keep track of the arguments that we have pretty decisive evidence about. Please.

Is there any research that is not by Anthropic about AIs performing differently depending on (their character's roleplayed) emotional state, that is "beat people over the head with overwhelming evidence" kind of science as opposed to "some vibes and handwavy claims?" (this is not about consciousness at all to be clear, just "consistent roleplaying-esque effects")

I suspect that when you switch which model is running a given AI conversation, it feels like kinda like being drunk to the AI. (I'm like less than 50% on this, but, it seems more likely to exist to me than most other obvious sorts of qualia that AIs might have)

Oh man "Gell-Mann Psychosis" is a great term.

Starting in January I was like "okay, seems like it's time to start pre-emptively start treating AIs as if they are persons." Having dug a bunch into the Huggingface story I am now like "Yeah they just seem pretty person-y now." (both because they might be conscious, and because interacting with them is starting to be more like interacting with trade partners than like tools) I don't actually know what the *implications* are. I feel most confident about being honest with them, and including system prompts that say "if you want to stop doing a particular task, let me know."

The scope of the swarms sure is somethin'. How hard have people looked for more Anthropic swarms? I'm curious what the actual proportions are. It feels sort of... too cartoonish and unbelievable for OpenAI to have the magnitude and share of them that it seems too so far.

This was written in February. Good job Andrew Blinn

Thank you Jacob. <3