TRISHALA (@hitrishala) header
TRISHALA (@hitrishala) profile picture

tinkering around at https://teapaw.org/t.co/TF0oTzBTtV (yc f26) prev @googlelabs @mongodb eecs @ucberkeleymet

San Francisco, CA
Joined July 2021
Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)Photo by TRISHALA (@hitrishala)
Welcome to the real world. Today we’re introducing FrontierEE Bench, the first benchmark to test frontier models on electrical engineering tasks at a physical lab bench. Real-world engineering benchmarks should involve real hardware. At our lab, models control instrumentation, run experiments on physical hardware, and decide what to test next. The best models scored 36.7% across a range of EE domains including mixed-signal circuits, power systems, & electromechanics.
90
54
13
513
37,985
Models passed 100% of these tasks in simulation. On real hardware, the average pass rate was 29.3%. Across 10 real-world tasks, models built open circuits, ignored stored charge, bypassed current limits, and calculated component properties from incomplete models.
2
1
18
1,171
The next frontier for AI is autonomous hardware R&D: models that can design, test, diagnose failures, and revise until a physical system works. That capability is learned through real experiments. We’re turning the failure modes we find into training data and RL environments grounded in physical lab work. We’re making physical experimentation part of how AI learns, starting with the electrical engineering that powers the economy – and AI itself. Read more about our benchmark results at evidentmachines.com/frontier…
1
2
21
871
Models scored 100% on these tasks in simulation. On real hardware, the average pass rate was 29.3%. Across 10 real-world tasks, models built open circuits, ignored stored charge, bypassed current limits, and calculated component properties from incomplete models. Read more about our benchmark results at evidentmachines.com/frontier…
1
41
The next frontier for AI is autonomous hardware R&D: models that can design, test, diagnose failures, and revise until a physical system works. That capability is learned through real experiments. We’re turning the failure modes exposed by FrontierEE Bench into training data and RL environments grounded in physical lab work. We’re making physical experimentation part of how AI learns, starting with the electrical engineering that powers the economy – and AI itself. Read more about our thesis at evidentmachines.com
1
26
i heard that a really awesome benchmarking startup is launching a super novel first-of-its-kind benchmark tomorrow. it’s just a rumor. idk!
2
26
1,098
hardware has always been expensive enough to force some discipline about what gets built. AI loosens that constraint for design - but it doesnt change the cost of board bring up/fixtures/bench time EEs who benefit from AI the most will be ruthless at cherrypicking designs and deciding which of them are actually worth taking into manufacturing
14
759
it’s crazy how papercut-y the physical world is. we use the most advanced simulators we can get our hands on to pre-screen our EE benchmark tasks & models that pass flawlessly in simulation get bogged down by the randomest tiny papercuts in the real world. there are so many of these papercuts to keep track of across different domains of EE that it’s hard to imagine simulators ever capturing all of them accurately enough to replace building + testing the actual thing
10
25
2,010
teaching ai about the physical world. unfortunately this is the physical world
2
23
1,018
the faster we can design hardware, the more the bottleneck to fast production shifts to manufacturing & iterating at a physical lab bench. the physical world is very messy and the difference between simulating electronics work & building it physically at our lab is huge. winners in this space will take design + test + ship + iterate + iterate again to speeds we've never seen before
1
14
693
i’m telling you guys spy-as-a-service is a billion dollar market and this is just the start
4
1
22
998
btw we rebranded 💸 something epic coming soon
3
29
1,135
our logo has been doing this for 4 hours and nobody’s offered it water 😵‍💫@OpenAIDevs
4
1
18
494
so is everyone making a benchmark now
10
24
1,695
the ai for hardware curve is gonna be fun to watch
12 years ago I founded Boom, as a software engineer doing hardware for the first time. I’m probably the least handy founder in hard tech. My 12 y/o son does my soldering and 3D printing for home projects, and I’ve struggled with imposter syndrome the whole way through. What business does a software guy have building supersonic jets and turbines? More than a decade in, the most insane realization is that software is the key to reaccelerating hardware engineering. But in a way that I don’t think is widely understood or implemented, even in the burgeoning hard tech community. Most hardware engineering is actually software engineering in disguise. It’s spreadsheet “code,” engineering calculations done as one-offs. Eg wing structural loads in, spar sizing design out. Thermal loads in, air conditioning system sizing out. Everything is a calculation, and calculation is software. But these calculations are done essentially by hand, as if we are doing arithmetic but have not yet discovered algebra. Hardware engineers have baby software, built in spreadsheets, the kind of crappy software I wrote as a kid before I understood architecture, unit tests, APIs, version control, and everything else that makes modern software good. Our airplanes and engines are built with the software sophistication of a script kiddie. So what we tell our engineers: don’t design the hardware, design the *algorithm* for designing the hardware. And take the algorithm seriously—as an algorithm and as a first-class engineered software system. Never design a piece of hardware, such that you couldn’t redesign it tomorrow with a different set of assumptions. This all sounds obvious, yet our experience at Boom is that it changes everything. Ten years ago we couldn’t afford to build much software. We had a small team—and there were few software engineers with the curiosity, brilliance, and vision required to learn hardware engineering. But now AI changes the equation. Now everyone can code. The software engineers define the architecture. The hardware people bring the physics and engineering know-how. We have a turbomachinery design tool that lets one engineer do in realtime what previously took 100 engineers six months. Another tool auto-generates 65% of entire aircraft structure. I’m sure we’ll figure out the other 35% too. Eventually these should not be multiple tools, but one integrated physics model of a complete vehicle, airframe and powertrain. The results speak for themselves. A boomless supersonic airplane built by 50. A full scale supersonic turbine made by 150. And with this approach, I don’t think we’ll need that many more people to build and certify a supersonic airliner. You don’t need a big team to build big things—not when everyone on the team is a software engineer.
1
7
496
holy conductor is down
2
7
720
oh and btw bring your wallet bc all the instrumentation will drain your entire bank account and cost you your first born child
if you’re in software, do yourself a favour and pivot to hardware.
4
19
919