Introducing Cosign. cosign.co

The future of security in the AI age is "the constant pressure of models attacking you," says Kevin Mandia of @ArmadinSecurity. That means seeing your network the way a swarm of AI attackers would: "We do a thing called a hyperattack. We throw a drone swarm of agents at you, and we map your network. Every service, every route, every system, all assets. We may end up with terabytes of metadata on your network." "This is the attacker's view of your network. But with that metadata, we now just poll you almost like a heartbeat." "What's changed?... Did a route change? Did a service get updated? Did a new machine get presented into the target area? We can poll cheaply for change and then attack the change." "You can't [attack yourself] all the time because it's cost prohibitive, and it's unnecessary. You do it when either the threat changes, new models come out, new intelligence is available, or your network changes." "Over the weekend, there was a zero-day in a popular product, and immediately we've already got the heartbeat. We just polled who's got the problem." "Our goal at Armadin is to go from a known vulnerability to knowing whether it's actually exploitable before the bad guys can."

👀

In the '90s, general-purpose chips wiped out the specialists. The original Thinking Machines, Cray Computer, and a wave of others shut down, and their best ideas got absorbed into other chips. Zain Asgar of @gimletlabs argues the conditions that made generalists win have changed. Dennard scaling (shrinking transistors made chips faster at the same power) ended around 2005, transistor costs stopped falling around 2012, and memory bandwidth never kept up with compute. Meanwhile, token demand is projected to grow 24x by 2030 while available power grows about 2x. When power is the bottleneck, performance per watt wins, and that's where specialized chips shine. Full piece from Zain on why "the time of specialization is upon us": a16z.news/p/why-hardware-spe…

Adit Abraham of @reductoai on why everything in AI is "state-of-the-art all the time," and how to find what actually works for you: "One of the unfortunate things about the AI industry... is benchmarks have become marketing over time." "There's a public benchmark out there that we tested where if you take the same documents and you add a little bit of noise, like you tilt it one degree, you add a little Gaussian blur, the provider that published the benchmark gets way, way worse than everyone else, but everyone else is kind of consistent." "Our recommendation is actually LLM as a judge is quite strong today if you're thoughtful about how you think about a rubric and how to approach it." "We want to see the industry move closer to people testing on their own data, with tools maybe to make that evaluation easier." "Instead of trusting a generic benchmark, you really should be at a point where you know what works best for you." @aditabrm

Introducing the Cosign Awards. cosign.co/awards

AI agents can reason over enormous amounts of information, but much of the world's actual work is still trapped in PDFs, spreadsheets, scans, and other formats that models struggle to reliably understand. Reducto co-founder and CEO Adit Abraham argues that solving this document layer is essential if agents are going to take on consequential work in healthcare, legal, finance, and insurance. Adit joins a16z General Partner Jennifer Li to explain why traditional OCR breaks down on messy real-world documents, how Reducto combines vision-language models with specialized infrastructure to reach high accuracy without sacrificing cost or latency, and why frontier models can silently drop information on long extraction tasks. Reducto now processes billions of pages each month, and crossed its 7 billionth page the morning of this recording. 0:00 Why AI agents need to read PDFs 2:12 From 1990s OCR to vision models 5:00 Training R1 on the messiest docs 7:46 When agents beat humans at extraction 9:45 A penny a page for 7 billion pages 12:47 Why legal and finance lag coding AI 15:01 The stakes of AI errors in healthcare 16:25 Agents that can edit Word and Excel 17:58 Why the labs won't go deep on PDFs 20:14 Frontier models silently drop data 22:21 Benchmarks have become marketing 25:45 Why charts and Excel stump models 28:46 Infra for 10M documents at once 31:03 How "chet pain" gets claims denied 34:32 The paperwork AI should kill first YouTube: youtu.be/xuczlbVVLsQ @aditabrm @reductoai @JenniferHli

Eugenia Kuyda, founder of @wabi, on why chatbots are the MS-DOS era of AI, and why a new interface is necessary to make AI's full potential obvious to everyone: "It always struck me as strange that people were using products like ChatGPT, Gemini, or Claude, but mostly for very simple use cases: for search, or to help them write something, help with homework." "No one was really mentioning any of these exciting capabilities that we just kept seeing the models get, and so we felt like there must be an interface problem." "There should be something simple for everyone, me included, to actually discover these amazing use cases that all these models already have." "We got obsessed with this metaphor that the current chatbots are really the Microsoft DOS era for AI interfaces. And that there will be some sort of a Windows/MacOS moment." "There is so much traction with chatbots... In order to unlock something else, you need a more exciting interface." @ekuyda

Ekin Dogus Cubuk of @periodiclabs on why AI has to learn from the physical world, and building their own lab to close that loop: "One of the more fundamental ways of making things around us is powder synthesis... You can actually, using that method, discover new superconductors, magnets, all kinds of materials that are very important for technologies around us." "But at the core of it is just quantum mechanics. And we feel like teaching these LLMs quantum mechanics will be the next frontier for LLMs." "Science is by its nature iterative, right? Like even the smartest humans tried many times before they discovered the things they discovered. LLMs can be very smart, but if they're not iterating on science, they won't discover science." "The important thing to teach these LLMs is the method of scientific inquiry. So you do simulations, you do theoretical calculations, you do experiments, you get results, and the results are probably incorrect or not what you wanted at first, but you iterate on it." "You have to do it with the real physics, not just the simulation." @ekindogus @AnjneyMidha a16z's Christian Keil makes the case for why building, testing, and iterating in the physical world is the bottleneck for AI in hardware: a16z.news/p/superhuman-hardw…

AI models are getting close to designing hardware better than humans, and that shifts where the hard problems are: - Better simulation, so physical builds become confirmation instead of experiments - Demand shifts toward parts, machining and raw materials - AI labs need to capture the "process knowledge" that lives in machinists and fabs The one step even superhuman AI will struggle to speed up: "A superhuman hardware engineer could design one million new iterations of the Apple Watch and get them manufactured for cheap, but then how can it know what customers actually want?" But once AI crosses over, "the impact should prove immense: humanity will be able to directly use digital AI tools to bring about real, physical, material abundance." Full piece from a16z's Christian Keil on what hardware engineering looks like once AI masters design: a16z.news/p/superhuman-hardw…

"It took me three months to design it, but it's nine months until I can actually start to use it." Former Intel CEO Pat Gelsinger on the manufacturing bottleneck between designing hardware with AI and actually getting it built: "Let's say we're off to do a great new chip together, the one that's gonna leap ahead because we have understanding of the AI workloads... And let's say we turn our AI tools loose. Within three months we have an awesome design." "[But] we still have like a couple of really major bottlenecks today. One is we still have nine months of silicon processing time... So I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months." "We have to really see that bottleneck improve if we're gonna have this innovation." "If it takes me a year and a half until I actually get scale and software on it, my understanding of the AI workloads is no longer applicable to the chip that I designed." "All of those aspects somewhat obviate the fact that AI has made the design piece easy. All of those manufacturing scale issues to me become a huge bottleneck." @PGelsinger @appenz @RaghuRaghuram a16z's Christian Keil makes the case for why manufacturing, testing and iteration are the next frontier for AI in hardware: a16z.news/p/superhuman-hardw…

For AI to become a superhuman hardware engineer, it has to learn from the physical world, and real-world data is hard to get: - It's non-uniform and constantly changing - It's sparse: you can't measure everything all the time - It's expensive to collect - Physical processes take time, and can only be sped up so far - Some of it is qualitative: do users actually like the product? "Data limitations like these are massive drivers in the spikiness of AI capability development today. The more quantifiable and machine-readable the results of a test, the faster the models can climb the hill." Full piece from a16z's Christian Keil on why the physical world is so hard for AI to learn from: a16z.news/p/superhuman-hardw…

Karan Singhal of @OpenAI on the collapsing cost of intelligence, as seen on their health benchmark: "GPT-6 Luna is our current cheapest frontier model on the API... It performed better than 5.6 Sol, which was literally our best model a month ago. And it performs better despite being 34x cheaper." Most tiers of model intelligence became more than 100x cheaper within roughly a year One tier went from ~$32 to ~$0.02 per million tokens in two years

AI is on track to become a superhuman hardware designer, but design is only the beginning. The interface between bits and atoms is lossy, and turning designs into real hardware gets harder at every level: 1. Sourcing off-the-shelf parts that work together (a 2006 Falcon 1 engine test was delayed by one capacitor, in stock 6,000 miles away) 2. Custom parts, where a master machinist translates CAD into machine instructions 3. Assemblies, where every part can meet its drawing and still not fit 4. Outside vendors (even SpaceX has 591 employees with supply chain titles) 5. Doing all of it at a scale of 1,000,000 "Superhuman AI might be great at pure design, meaning at defining the platonically ideal part for a particular job, but struggle with 'design for manufacturing,' or finding a point design that meets the requirements while remaining easy to build/procure and inexpensive." Full piece from a16z's Christian Keil on what it takes to turn AI-designed hardware into the real thing: a16z.news/p/superhuman-hardw…

Five bangers from OpenAI's Karan Singhal: 1. Raising the floor and the ceiling of human health: "There's more than 300 million people weekly using ChatGPT for health and wellness types of questions, and we take that as an incredibly important responsibility to get that right." "We're really invested in making sure basically two things. One is that we can raise the floor of human health and give people access to trustworthy, abundant, reliable health intelligence for free or nearly free, and really raise the ceiling as well and make things possible that weren't possible before." 2. Five physicians, eight answers: "It turns out if you ask five physicians any question, you'll get, like, eight different answers. One of the first things that we did when we first started focusing on health at OpenAI was really build this kind of global physician network of hundreds of experts." "[The goal was] to kind of aggregate the wisdom of experts so that we weren't relying, especially in areas where there could be disagreement, on one or two or three physicians to kind of adjudicate." 3. Why health runs on a general model rather than a specialized model: "What we're talking about here is actually specialized training in a general model, which is, I think, actually the best of both worlds." "If you forked and said, 'Hey, we're gonna train a smaller specialized model that is just for healthcare'... what you'd find is that model would fall behind the frontier relatively quickly, because the pace of the frontier and the general model improvements is so fast." 4. Compressing medicine's 5 to 17-year lag: "How long does it take for clinical guidelines to penetrate the practice of everyday doctors across the world? People have found that this takes, depending on the study, 5 to 17 years... You don't wanna be practicing medicine 5 to 17 years behind." "One of the things that we think about is how do you compress that kind of period of practicing down to weeks or days or even hours." 5. Healthcare's legacy software is ripe for agents: "Healthcare is also one of those industries where you shouldn't expect everything to have an MCP connector tomorrow. It is a place where there's legacy software... This is absolutely ripe for agents and computer use."  "This is a place where a lot of the barriers that have traditionally existed between patients and getting the care they need and clinicians and delivering the care they need and healthcare data in general... I think a lot of those will dissolve." @thekaransinghal

If you ask five physicians a question, you might get eight different answers. Karan Singhal of @OpenAI on how OpenAI decides what its models should say about health: "One of the first things that we did when we first started focusing on health at OpenAI was really build this kind of global physician network of hundreds of experts that we work with." "Across the globe, across dozens of specialties and languages and countries of practice, to think about the right ways to aggregate the wisdom of experts so that we weren't relying, especially in areas where there could be disagreement, on one or two or three physicians to kind of adjudicate." "Let's say the aggregated wisdom of the physicians is pointing in one direction, and then maybe the user actually wants you to say something comforting even when the situation's not comforting. There it's basically a prioritization of safety." "It's very important to us that we take our obligation very seriously when there's hundreds of millions of health conversations coming in." "We have evaluations that track these kind of difficult situations... Over time, we've not only improved how we've done that evaluation working with physicians, but also seen pretty dramatic model improvements across the board." @thekaransinghal @JayRughani