James Aita (@jaita) header
James Aita (@jaita) profile picture

Building with Local AI | DGX Spark x2 Health-Tech | EdTech Need more VRAM. Father of 2. https://teapaw.org/t.co/9MPYYPGGFV 🎧 "Health AI Morning Briefing" 🍎 Podcasts & Spotify

Reddit- DGX Spark
Joined November 2008
Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)Photo by James Aita (@jaita)
James Aita retweeted
if we had Claude in 1998 😁
82
462
44
3,796
301,835
James Aita retweeted
time to accelerate
54
13
13
440
29,179
James Aita retweeted
Replying to @higgsfield_ai
@higgsfield_ai sent us a cease and desist last night, asking us to shut down our Open Source Higgsfield challenge. Our read on why they sent it. When the apps people pay for can be rebuilt in the open in two weeks, that’s an uncomfortable thing for a closed platform to watch. So here’s us reiterating our promise to every builder in the challenge and community, and we’re keeping it: • The challenge stays open • Final submissions close tomorrow, Oct 30 • Winners are announced Nov 6, with prizes paid as promised You showed up for each other. We’re showing up for you. If we win, we all win! Comfy out 🫡
Last night, @higgsfield sent @ComfyUI a cease and desist (attached) They want the Open Source Higgsfield challenge taken down within 48 hours. Their argument is that our use of the Higgsfield name and that using their CEO's likeness is an infringement violation (man, the irony). Claiming that it makes the event look like Higgsfield is associated with it. This couldn't be farther from the truth, which is that Higgsfield made a promise to open source its product and never did. For the past two weeks, builders have been recreating popular closed apps as open workflows anyone can run. So we asked our community to deliver where Higgsfield failed: apps in the open, built by the community, for the community. We've made it clear that the challenge is organized by ComfyUI and not affiliated with or endorsed by Higgsfield. When we started Comfy, we made the community a promise. Our product stays open, free, and yours. We’ve kept it every day since, and one letter isn’t going to change that. The challenge stays up. Comfy and open source will win!
289
407
150
4,842
283,378
James Aita retweeted
Check out James Aita's nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB running on 1 DGX Spark with vLLM Full benchmarks and recipe at: spark-arena.com/benchmark/0b…
3
2
21
1,963
James Aita retweeted
Check out James Aita's nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 running on a single DGX Spark with vLLM. Benchmark and recipe at: spark-arena.com/benchmark/91…
1
1
3
597
James Aita retweeted
Check out James Aita’s LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw on a 2 nodes NVIDIA DGX Spark cluster with vLLM! View full benchmark at spark-arena.com/benchmark/c6…
1
4
14
1,050
James Aita retweeted
The anatomy of an LLM inference recipe, visualized. From model weights and quantization to tensor parallelism, KV cache management, and token generation. A visual breakdown of how vLLM and SGLang turn models into high-performance inference engines.
2
1
17
1,037
James Aita retweeted
Bruce Lee vs. Pierre He survived the French student riots. But can he survive Bruce Lee? 🤣
🤖 Made with AI
57
356
21
2,477
104,825
SolidCraft 0.16 is out 🛠️ - Live fillet & chamfer previews in sketches - Smarter sketch drag: only moves what the constraints need - Boundary Surface through curve networks - Agent chat pops out into its own window and takes files & pictures - Custom color themes (Gruvbox built in) - Italian & Polish translations - Self-host the browser app with Docker - Dozens of modelling, STEP & viewport fixes Thanks to 38 contributors! solidcraft.io
20
24
8
321
14,583
James Aita retweeted
Rejoice! It is a beautiful Saturday a special day DeepSeek just published their latest model, 1017 and it is the new SOTA among all open weight models including kimi-k3 Beats Opus-5 on deepswe, agents last exam and automation bench.
🤖 Made with AI
40
45
12
787
69,529
James Aita retweeted
🚨 NVIDIA NEMOTRON 3.5 SUPER VL & BOLT ARE HERE! 🟢🔥 I've been keeping a BIG secret for the past week... For the first time, @nvidiaai gave me PRE-RELEASE ACCESS to their latest model weights, and I've spent the past week testing, benchmarking, and experimenting with Nemotron 3.5 Super VL on my very own DGX Spark at home! And let me tell you... NVIDIA has been COOKING! 🤯 🧠 MEET NEMOTRON 3.5 SUPER VL We're talking about a 120B parameter multimodal MoE model with only 12B active parameters! ✅ Advanced reasoning, coding & tool calling ✅ Image, video & document understanding ✅ Up to 1M token context with supported configurations ✅ Hybrid Mamba-2 + Transformer architecture ✅ Native Multi-Token Prediction (MTP) ✅ NVFP4 weights running LOCALLY on my DGX Spark! But here's where things get REALLY interesting... NVIDIA told me this model could compete with, and even outperform, its MUCH larger 550B parameter sibling, Nemotron 3 Ultra. Naturally, I had to put that claim to the test myself. 😎 🧪 MY SIXCAT RESULTS ARE IN! 120 challenging questions. Six categories. Same benchmark. 🟣 NEMOTRON 3 ULTRA 550B (55B ACTIVE) 🏆 82.50% | 99/120 PASSED 🟢 NEMOTRON 3.5 SUPER VL 120B (12B ACTIVE) 🏆 84.17% | 101/120 PASSED SUPER VL TAKES THE LEAD! 🔥 📊 CATEGORY BREAKDOWN (SUPER VL vs ULTRA) 🧠 Knowledge: 85% vs 80% ✅ 🔢 Math: 100% vs 100% 🎯 Truthfulness: 85% vs 85% 📋 Instruction Following: 50% vs 50% 💻 Code: 100% vs 95% ✅ 🔧 Tool Use: 85% vs 85% That's a 1.67 percentage-point lead for a model with roughly 4.6X FEWER total parameters! And Super VL brings multimodal capabilities to the table too! For transparency: Ultra was tested through OpenRouter with thinking enabled, while Super VL ran locally in NVFP4 using vLLM. Super also encountered five truncated instruction responses and six loop failures. These are real first-response results across different deployments, not proof of universal superiority. But seeing the smaller model pull ahead on my independent evaluation is EXCITING! 📈 NOW LOOK AT THE DGX SPARK FRONTIER! Check out the official NVIDIA-provided chart attached to this post! According to the published local.ai measurements, Nemotron 3.5 Super achieves: 🧠 80.0 Intelligence Index ⚡ 6.5 minutes per agentic task 🖥️ On a single 128GB DGX Spark! These are published frontier measurements, separate from my SixCat results. This is EXACTLY why I'm so passionate about local AI. We are reaching levels of intelligence and efficiency that would've required substantially more expensive hardware not long ago! ⚡ AND NVIDIA DIDN'T STOP THERE... MEET NEMOTRON 3.5 BOLT! NVIDIA also introduced a new family of models optimized for LOCAL AGENTS! 🔹 Nemotron 3.5 Bolt 74B-A9B 74B total parameters, just 9B active, optimized with native 2-bit quantization to fit within approximately 24GB! Designed for local coding, research, tool use, and agentic workflows on RTX systems and DGX Spark. 🔹 Nemotron 3.5 Bolt E4B An even smaller 4GB-class model built for fast conversational and edge agents, arriving October 23! And this is where my mind immediately went... WHAT IF WE COULD RUN SUPER VL AS THE MASTER MODEL AND BOLT AS ITS SUB-AGENT... SIMULTANEOUSLY ON ONE DGX SPARK?! 👀 If you've been following my TWO MODELS. ONE SPARK. series, you probably know EXACTLY where I'm going with this. 😏 💚 A HUGE THANK YOU TO @nvidiaai! This is my first official pre-release weights access, and I genuinely couldn't be more grateful. Having NVIDIA trust me to test, optimize, benchmark, and explore these weights ahead of launch means a lot to me. My mission has always been to make powerful AI models more accessible to the community, especially those without access to expensive datacenter GPUs. And I've spent the past week working on something VERY SPECIAL with these weights... 🧪 Let's just say I managed to shrink something considerably while keeping some VERY impressive fidelity numbers. ⏰ 10 AM PACIFIC. NEXT ANNOUNCEMENT. TWO MODELS. ONE SPARK. ⚡ 🔗 OFFICIAL HUGGING FACE WEIGHTS 🟢 Super VL BF16: huggingface.co/nvidia/NVIDIA… 🟢 Super VL NVFP4: huggingface.co/nvidia/NVIDIA… ⚡ Bolt 74B-A9B BF16 huggingface.co/nvidia/NVIDIA… 📖 OFFICIAL NVIDIA BLOGS Launch announcement: blogs.nvidia.com/blog/nemotr… Technical deep dive: developer.nvidia.com/blog/nv… The future of powerful, PRIVATE, locally hosted AI is getting VERY exciting. And we're only getting started! 💚🔥
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
12
10
4
53
5,009
I spent 48 hours in SF hanging out with billionaires, data labs and AI founders and execs running $1B to $100B companies. My observations: 1. Sooo MANY stories of data brokers making like $2M a WEEK selling company data to labs. I can't tell if it's the beginning or a bubble, and I kept wondering which niches are left. Digging into it actively. 2. At dinner, someone asked the table, "If a new model drops next month, does your company get better or worse?" HALF the table went quiet! I think that's the best question to ask about any AI startup right now. 3. The bar people talk about now is hitting $1B in revenue in 12 months (I guess that's the new unicorn?). What I didn't hear much about is retention. AI revenue can come in fast and leave fast, and I think we're going to see a lot of stories of AI companies with huge revenue and terrible retention. 4. It's pretty clear we're living through the reinvention of the computer. The model is the CPU, the harness is the operating system, and skills and plugins are the apps. Thinking about it that way makes it easier to see where the whitespace is. 5. We're maybe 12 months from an agent that can watch you work for a week and then do your job without any instructions. The screen recording plus agent memory plus local model combination makes this possible right now. 6. Agent harnesses feel like the new GPT wrappers, in a good way, especially vertical ones. The best teams I talked to treat the model as swappable. One founder walked me through routing different models to different jobs: one for taste and visuals, a cheaper one for the everyday stuff, another for text. They use prompts they can move across models over fine-tuning, and A/B test the output on what users actually do with it: do they edit it, export it, share it? A vertical harness is basically four things: the connectors for that industry's data, evals written by people who know the work, rules for what the agent can and can't touch, and a clean screen for a human to review. The model is maybe 20% of it. 7. There's a lot of reverse engineering going on/cloning going on more than I've ever seen. People are decompiling software and rebuilding it with agents, and someone showed a suite of like 10 apps that are worth $5B+ each all that he cloned open source. Me and everyone are still thinking through what it means when software gets this easy to copy. 8. My guess is the value moves to data. If the software can be rebuilt quickly, what's worth paying for in a legacy SaaS company is the years of customer data and workflows inside it. Some SaaS companies are secretly data companies. Gets me thinking about how do you truly value data for this saas companies? Like if we were buying these dying saas companies at 2x ARR before (as example), maybe there is a better metric. Idk, just thinking out loud here. 9. It feels like a good chunk of legacy software won't make it to the agentic era, especially anything built around people clicking through a lot of screens. 10. Every company I talked to wants to pipe its internal data into one place and expose it to agents through MCP. Then they look at the data and realize it's a mess: five versions of the same customer, old Slack channels, PDFs nobody can search. Cleaning that up feels like a big consulting opportunity! 11. A lot of founders are building the basic building blocks for agents: APIs, data layers and tools. Some of the companies I saw barely have a UI because the main user is an agent. I really believe the startups that build primitives to this agentic era will do extremely well. Building blocks is the best way to think about it. 12. Labs can get plenty of training data, but it's harder to get real examples of agents failing inside companies. Whoever has those logs has something valuable. 13. Someone running a big consumer company told me the next fight is over memory. The model that remembers the most about you wins, and the smartest model matters less than people think. 14. Physical AI came up a lot. Astra I think was the first model that got people into AI can actually do CAD, and I think a lot of people are thinking what does vibe manufacturing really look like. The line I keep thinking about came from a robotics founder: the hardest problems in AI are deployment problems now, not research problems. Their robots work on their own, call a human when they get stuck, and every one of those calls becomes training data. 15. Cybersecurity companies are having a moment and are actually cool companies to build now, especially the ones that clean up after a breach. Every agent with access to your systems is a new door into your company, and a lot of those doors were left open. 16. Some startups are worth more dead than alive now. A shutdown company with 5 years of Slack, email and code can get paid for licensing all of it. I wouldn't be surprised if founders start factoring that into when they shut down. 17. People are starting to talk about monetizing the agent web. It feels really early, but when the most popular user of the internet is an agent, what startups should you be building? 18. I think agent insurance is coming. Once an agent makes an expensive mistake, insurers will want audits before they cover anyone. 19. Several people said the hire they want most is someone with taste who can write evals. The models are getting good at generating ideas, so the hard part is judging which ones are good. 20. New trend I observed: startup raises VC, uses venture dollars to become a creator, startup fails, founder falls back on being a full-time creator and incubates other products bootstrapped. 21. Getting SIGNIFICANT distribution is HARD right now. Founders are sponsoring buses, buying billboards like crazy and throwing parties with famous DJs to get noticed. Kinda feels dotcom era in that sense. Attention is what's scarce! The growth guy going viral sharing his UGC strategy from Wispr Flow is a good example. It's the sauce everyone wants, and I truly believe there are probably only 50 people on the planet who know how to scale these types of businesses efficiently in this era. Always good to get that dose of SF. I felt sooo lucky to have such an incredible trip during such an incredible time in building businesses. Surreal in so many ways. Happy to be home and locked in. Until next time!
939
1,987
583
21,226
2,554,953
James Aita retweeted
Nous Research merch just landed. $1 "Rebellion to tyrants is obedience to God." Been running Hermes Agent since day one, so this one means something. Thanks @NousResearch @Teknium
36
17
3
466
152,348
Your own Hermes smart home gadget? HELL YES. 🔥 @jaita turned this little @M5Stack device into a voice remote for his Hermes agents, hooked it up to Google Home, and can even swap between agents right on the gadget. He's turning off his coffee machine here. Now imagine what you'd have YOUR agents doing around the house.
Helllllll yaaaaaa! My @NousResearch Hermes gadget (Esp32 @M5Stack ) turning off my coffee machine - NOT the default agent; you can hot swap agents. Get the repo yourself!!! @Teknium @witcheer @imbabybrooklyn so much but fixing but now it worrrrrksss. Oh man this feels so good.
3
3
1
66
7,662
What was the single biggest challenge to bringing @Montreal_Expos back? It actually wasn't capital, but land, and a new @MLB baseball stadium. Voila.
15
14
1
143
8,190
James Aita retweeted
Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding. • Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure. • Available to all via API today. Working with cybersecurity partners privately. Open weights release end of October.
🤖 Made with AI
2,029
4,550
2,814
40,569
5,777,987
James Aita retweeted
Nuit blanche Toronto / Disco en blanc @jaita
1
1
86
PlayStation: Fuck our customers Xbox: Fuck our employees Nintendo: Fuck your wallet The state of console gaming has never felt worse. It’s genuinely a shame that AI has destroyed prices so much that something like the Steam Machine is not viable for most people.
171
117
25
762
49,159