Vision Transformers (@vitransformer) header
Vision Transformers (@vitransformer) profile picture

life is just a single agent rollout

India
Joined April 2024
Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)Photo by Vision Transformers (@vitransformer)
Looks so cool
Introducing SID-2, a new search model built on compute-scaling swarms. Our swarms find better results than Opus and Astra, while being 8x faster and 20x lower cost. By 2030, AI will write an internet's worth of text every 24 hours. Current search pipelines were built on hand-designed rules, adding more won't scale to meet this incoming volume of data. SID-2 is available on our API today. Try it on your most demanding searches.
85
Vision Transformers retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen
56,054
115,595
23,594
1,089,977
48,830,647
Compiler one day might just make all the mega kernels we want
Last Year We Built Megakernels. This Year We Deleted Them. One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target. Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
7
307
Now share the 27B
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery! Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window. Highlights: - Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation. - Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables. - Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery. - $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%. The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. Start building today! 🚀 - Blog: qwen.ai/blog?id=qwen4-max - API: qwencloud.com/models/qwen4-m… - Token Plan: qwencloud.com/pricing/token-… More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
1
160
Vision Transformers retweeted
💛 Inspiring to see your work on the big screen.
2
36
864
Vision Transformers retweeted
whats being posed as a groundbreaking forensic investigation is literally public information available on GitHub.
37
81
9
1,580
129,799
Yes
I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
105
Vision Transformers retweeted
I died. This has to be the best one yet
611
9,885
898
74,683
2,891,477
Personal video models when?
Asked Astra to clone my voice by editing the weights. My father and a few friends couldn't tell it was cloned. Can you tell me which one is original?
1
2
164
Reminds me of mafia
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game). The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies. I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump. Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game. You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
1
109
Vision Transformers retweeted
20
3
10
187
70,538
Dockerize it and built a super fast backup infra?
We rented the same H100 for 80% less. The catch: the cloud can take it back at any moment, and your job dies with it. So we built Portal. It saves your running Python job and resumes it on any other machine, any cloud, mid-run. No code changes. One-prompt install and forget.
1
1
154
Vision Transformers retweeted
Swarm of Agents has been running our AI perfume influencer for 18 days with minimal human input. 31k views, zero ads Introducing VibeGrow, agent swarms that run companies fully autonomously. How far can a swarm take your idea? Find out yourself 🧵
3
17
2
23
1,131
Delhi is a live version of this meme
How has delhi air gone from best ive seen in a decade to worst ive seen in a decade in 48 hours I am convinced we can nail the cause. Its literally something thats happened in the last 48 hours.
1
127
Vision Transformers retweeted
Introducing the new LiteFold. Now generally available. Hill Climb Biology.
25
34
12
231
43,910
Would be crazy if claude can vibe code it with airpods
I recreated subvocal in 9 minutes with one prompt for AirPods Pro 3. Source: github.com/joeblau/subvox
169
yes 50 it is!
A rare winter day in Delhi with an AQI of 50
2
1
1
22
1,863
Vision Transformers retweeted
A potentially controversial thought: We should make AI PhD coursework only optional. So many of our PhDs are tied in numerous classes, but: - Self-learning is easier than ever. - Many incoming PhDs are already strong researchers. - We need to free Profs' time for more research mentorship. - Urgent AI problems need free-thinking minds (like our academic students!), independent of industry for-profit directions. Courses are valuable. But they don't need to be mandatory, especially when so many problems with AI are threatening our society, and urgently need to be solved.
34
23
4
319
34,436
they are just gonna drop a hydrogen bomb soon!
Announcing Gemini 4 Carbon, our strongest model with state-of-the-art cybersecurity and long horizon capabilities. Carbon represents a step up across most benchmarks, delivering strong frontier performance in autonomous coding and cyber defense. We’re first making Carbon available to a subset of trusted cyber defenders in our Fairwind program. We’ll expand access as soon as possible.
6
452