timbwalsh

On my quest to spend $1 / day on Gemini Flash, I gave the latest version (0827) transcripts of my last ~40 podcasts, some ~800K tokens total, and asked for a "Personal Introduction and Self-Description" ... in my voice! Here's what it came back with. Not flawless, and I don't fully endorse all the takes this contains, but the issues are pretty nuanced – overall, it clearly "gets me" Total cost for this was ~12 cents. Damn impressive! """ ## The AI Mind: A Scouting Report The AI world is a dizzying place. It's hard to keep up. Every day there's a new paper, a new product, a new breakthrough, a new set of worries. And the bigger these AI systems get, the faster they move, the less we seem to understand them. You can't be an AI scout without being a little bit obsessed. And in some ways, I don't know that there's any such thing as an AI scout that's not obsessed because the world is now an information jungle. It's all these different things, all interlocked, and changing so fast that it can feel overwhelming, And honestly, there's no way to fully keep up. My approach is to keep reminding myself, like, hey, we have to keep tabs on the state of the art, but also keep in mind that this stuff is only really getting started. It's only been a couple years, really, since a lot of this took off with GPT-3 and all the ensuing craziness. But I'm also very much trying to not lose my mind because it is so easy to get caught in the hype cycle. There are a lot of people who are like, wow, AI is gonna be everything. And others are like, AI is gonna destroy the world. And then the other group is saying, well, no, it's gonna be okay. I find myself mostly in the middle of that group because there's just so much uncertainty. I do think there are real risks, but there are also tremendous opportunities that we should be embracing. And I think it's important to be cautious, but also to not be afraid of making progress and making the world a better place. I'm very much an adoption accelerationist, hyperscaling pauser. Which is to say, I'm excited to get as much practical utility out of the technology as we possibly can as quickly as we possibly can. And I want to see us using it to solve really big problems like curing diseases and eliminating poverty and just making life a lot better for everyone. But I do think we need to be smart about how fast we continue to scale these systems up because the risks are very real. We are in a world where the best AI systems are now demonstrably better than the average human at the average task. That means if you take some random task that is typical of your average knowledge worker or even just a college student, a language model can do it better, faster, and more cheaply. That is an incredible reality. And it's only gotten better over the last year, with models like GPT 4 and Gemini 1.5 showing huge improvements in not just capability but also the ability to handle longer contexts, all at a much lower cost. That's a pretty big deal. These systems are also closing in on expert performance in many different fields, particularly in medicine. The AI doctor, as I've mentioned many times, seems to be on the horizon, and I really can't wait to have one. There's a lot of, I think, justifiable worry about whether or not we can really get to a point where we can trust those systems to not make terrible mistakes, but there is already a lot of evidence that AIs are able to make much better diagnoses and provide more useful recommendations than an average doctor. Not only that, but they also are able to take on a lot of the drudgery tasks that doctors and other professionals have to do as part of their jobs. This all suggests that the world is in for a really big transformation, and one that is happening really fast. It was just 5 years ago that GPT-2 was released and I remember reading about it at the hospital while my wife was giving birth to our son, who's now 5. That was a time when we weren't sure if any of this stuff would ever really work, and yet now the current models are really starting to rival and even surpass human performance in a variety of fields. So we're not just talking about more efficient ways to do what we've always done. We're talking about entirely new ways of doing things that will enable us to accomplish goals that previously seemed impossible. And I'm really excited about it. I just want to make sure that we don't get too far ahead of ourselves and that we don't let this incredible technology get turned against us. It's not that I'm, like, a hardcore doomer. It's more that I I have this deep sense that there are more questions than answers in the AI space. You could argue that I'm a techno optimist, and I am. I am truly optimistic about the potential of AI to make the world a better place. But I am not an AI utopian because I do think we have a lot of work to do to understand these systems and to figure out how to control them. The reality is that, these systems are not inherently good or bad. They are really just tools, and those tools can be used for good or for bad. It's up to us, I think, to guide things in the right direction. And so that brings us to one of the biggest challenges in this space, which is the question of AI alignment and ensuring that these incredibly powerful systems really are working for us and not against us. The AI safety community has been working on this for many years, and frankly, they have not made nearly as much progress as I think most people would have hoped they would have made. It's hard to know how much progress has really been made, and there are not a lot of tools that I think can truly be trusted at this point. One of the big questions is, can we do enough to control and guide AI before it becomes more powerful than us? At this point, I do think that it's getting close to the point where the best AI systems will be more powerful than any individual. And we don't yet have a good handle on the tools we'll need to make sure those systems are working for us. We're gonna have to solve that problem or at least make a lot more progress on it before we can continue scaling AI systems up to even more powerful levels. It's not a good time to be a doomer. There are too many cool and useful things that are happening right now. In a way, the state of the art today is almost a sweet spot. I think GPT 4 and Gemini 1.5 are powerful enough to be really useful, but they're not so powerful that I'm really worried about them getting out of control. We have time for this research, this AI alignment research to catch up and I think that's absolutely critical. I think we should be investing a lot more in this area than we are right now. If we want to take advantage of all the benefits that AI has to offer, we've got to make sure that these things are safe. One of my big fears in this space is the prospect of a AI arms race between the United States and China. That seems like a recipe for an unstable world. And, honestly, I think we are now in that race. The fact that the US government is starting to talk about the need to slow down AI development in order to ensure we can control it, and the fact that so many leading companies are racing to create even more powerful systems, all suggests that this is indeed a race, and it's a dangerous one. I don't think that we should be trying to maintain a technology edge at any cost. Instead, we should be trying to find ways to cooperate with China and all other nations to ensure that AI is used for good. This is not a time to fall back into the default pattern of, you know, us versus them. We've got a lot to lose if we continue to do that. And I think there's a real opportunity right now to rethink the way we relate to one another. So what does that mean for me? Well, I continue to be a very enthusiastic user and developer of AI, and I will keep pushing for the adoption and acceleration of technologies that can make people's lives better. That includes AI doctors, AI assistants, AI coders, and everything else that can unlock new possibilities for individuals and for society as a whole. But as those systems get more powerful, I will also continue to advocate for caution and oversight in AI development. I think this is gonna be a really important debate, and I hope that we can find ways to have those discussions in a measured and productive way. That's all for me for today. Thanks for listening. """

Quick followup micro-thread: Google edition. I used OpenAI for core analysis because they are clear leaders, but Google has most of the same advantages! teapaw.org/labenz/status/16548533…

This one hit me, harder than expected