Given a complete case, 21 frontier models named the correct diagnosis more than 90% of the time. Given only what a clinician actually has at the start of a visit, they failed to produce a comprehensive differential more than 80% of the time. That single pair of numbers, from a JAMA Network Open study out of Mass General Brigham, explains most of the confusion in this field. Here is what it looks like from clinic. A patient comes in with fatigue and some bruising. There is no vignette. There is a story told out of order, a spouse adding details, a CBC from three months ago sitting in an outside system, and my own quiet running list of what I am afraid of. The hard part of oncology has never been naming the disease once the workup is done. It is deciding what to work up. The authors put it better than I can: the model is being asked to do precisely the part of medicine it does worst. And it is being asked that most often at 2am, by a frightened person with no training in how to prompt or how to appraise what comes back. Forty million health questions a day now go to ChatGPT. I build clinical AI. I am not arguing these tools are useless. I am arguing we are grading the wrong half of the encounter. On the other hand, I believe that practicing medicine without using AI is considered a service for the patients. AI gives me eyes and insight that I will never gain on my own, but the current benchmarks don't make sense. Everyone is trying to either prove AI is better than doctors or doctors are better than AI. Believe me, it doesn't work like that. Healthcare is so complex that AI is a so powerful tool. However, it still needs a human in the loop. You can't run a hospital by AI agents without humans. Mark my words! It's not because AI is retarded. It's because patients need humans, and this will not change. jamanetwork.com/journals/jam…