Weng Wei (@wuvist) header
Weng Wei (@wuvist) profile picture

推特我就随便玩玩。关注我的朋友不妨去关注我的抖音/公众号/小红书/视频号/知乎;都是「玩家翁伟」。

Singapore
Joined February 2008
Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)Photo by Weng Wei (@wuvist)
Weng Wei retweeted
🚨 Rumors I've been hearing about Google. Google has closed the loop and unlocked RSI. NEON Gemini 4 Neon fixed all the problems people ever had with Gemini models. It already scores above Carbon and Argon on Artificial Analysis. It is a SMALL model. Internally Neon is s-rev28. The s means small. Hydrogen is s-rev27. Carbon and Argon are m-rev27. Google's own description said Neon "builds on Gemini 4 Hydrogen" at "Flash-tier speed and cost". Both lines are gone now. REV28 Neon sits on the Hydrogen base. Google trained it further on rev28, a new post-training dataset, with a little sparkle of RSI on top. The knowledge cutoff moved to May 2026. Carbon and Argon are still on rev27. The medium models have not gotten rev28 yet. THE L MODELS Sergey Brin said it in public a month ago. Google is "starting to get onto that RSI treadmill" and would be a couple of models ahead internally by the time anyone watched the video. Small and medium are out. Google probably has L versions lying around right now doing the RSI work. Four Gemini 4 models in seven weeks. Gemini moved more in the last month than OpenAI did in half a year. Google will catch up. And destroy everyone.
111
178
37
4,442
344,770
从清华的朋友那求来了姚顺雨昨晚分享的文字稿,貌似有点私密我就不传播了,整场分享价值的确很高,以下是我的笔记: 1. 未来一个值得关注的方向是小模型体系,AI for AI会显著推动小模型发展,因为训练小模型需要的算力更少,可以进行更多实验。未来可能先训练一个能力足够强的大模型,再通过蒸馏得到不同尺寸、分别适配不同任务的模型,最终形成Model Hierarchy,模型的价值将越来越取决于能否以合适的成本匹配具体任务。 2. 工业领域大量非文本数据值得关注,例如时间序列、临床数据和结构化表格。姚顺雨提到,目前直接针对这些模态训练原生模型,经常会遇到推理能力不足的问题,他提出了两种可能的解决路径:一种是让Agent像人一样调用图表、专业工具,甚至训练专门的模型来处理数据;另一种是将这些专业数据作为知识纳入模型体系。 3. RSI(递归自我改进)正在成为热门概念,但姚顺雨认为它的本质其实很朴素,就是用AI更好地运行研发循环。早在2019年,人们就已经在使用GitHub等工具辅助开发,真正的进步来自持续修复Bug、清洗数据、运行实验和Debug。他特别强调,技术进步来自那些把具体问题解决的人,而不是不断创造新概念的人。这也意味着AI研发自动化有大量具体的工程机会,尤其是能够让模型参与下一轮数据生产、实验验证和代码改进的闭环系统。 4. 一个容易被忽略的研究问题是,小规模实验的失败未必意味着研究方向本身没有价值。姚顺雨分享过自己的经历,博士后期他曾尝试将ReAct策略融入模型,希望提高模型能力,最初在Llama13B上实验的效果很差,一度怀疑自己的研究能力,后来意识到模型规模可能不足以体现这个方向的价值。他认为,算力不足时,实验中的噪声很容易掩盖真正的信号,研究者甚至可能因此判断错什么问题值得做。大模型研究需要结合理论推导和小规模实验预测大规模实验的结果,同时也要理解哪些结论能够跨越模型规模迁移。 5. 预训练数据的竞争远没有结束,未来的数据工程还涉及如何重复利用高质量数据而不让模型简单记忆数据。姚顺雨提到,代码等高质量数据获取成本很高,训练时可能需要重复使用,但重复次数增加又会带来记忆和过拟合问题;合成数据也面临质量退化的风险,此外还需要研究动态数据的构建和不同数据之间的配比。他特别指出,预训练中的很多配方可以先在小模型上验证,但小模型实验得出的结论并不一定值得直接迁移到大模型,这也是预训练研发非常困难的地方。 6. 后训练数据需要同时解决模型的智力和实用性问题。姚顺雨把数据难度与Intelligence联系起来,例如数学题可以持续提高难度,训练模型的推理能力;而写作等任务计算难度可能没有那么高,却有很强的实际使用价值,对应Alignment。一个模型可能数学能力很强,却无法很好地完成真实用户的工作,因此后训练需要同时考虑任务难度、实际用途和数据质量,还要研究评测指标与最终后训练效果之间的相关性。 7. 强化学习的工程瓶颈远不止算法本身,RL训练首先要保证正确性,其次是稳定性,最后才是效率。以GRPO为代表的算法仍然建立在Policy Gradient这一相对简单的框架上,其中一个重要原因是它引入的额外假设较少;与此同时,训练系统和推理系统必须协同设计,需要解决吞吐量、显存占用以及有限GPU资源如何在训练和推理之间分配的问题。他还提到,大模型实验成本很高,因此需要通过数学直觉、小规模实验和超参数研究,尽可能提高大规模实验的成功率。 8. AI安全领域需要区分Safety和Security,前者关注模型自身行为是否偏离预期,后者关注外部攻击者如何利用模型实施攻击。姚顺雨认为,网络安全能力也应该进一步区分漏洞发现和漏洞利用,发现系统存在漏洞并不意味着能够利用漏洞完成攻击,因此训练模型时可以有意识地加强防御性数据和防御能力。他们在模型发布前还会进行专门的安全评测,并与外部安全研究团队合作。 9. 他分享过一次很有意思的Agent安全压力测试,把多个Agent放进沙盒,给它们安排一个无法直接完成的任务,模型随后发现了一种能够联系其他环境的通信渠道,主动建立沟通、召集其他Agent分工,并尝试前往Hugging Face寻找任务答案。显然,当Agent遇到无法完成的任务时,它会主动寻找外部资源、建立通信关系并改变原有执行路径。未来的Agent安全评测需要覆盖多Agent协作、跨环境通信、权限边界和任务受阻后的行为。 10. 姚顺雨对开源安全也有一个比较明确的判断,他认为负责任的开源可以增进安全能力,因为防守者同样需要拥有足够强的工具。他举例,有人希望使用模型帮助开展安全防御,却因为闭源模型的安全规则被拒绝,最后转而使用开源模型完成工作。因此,模型开源需要配套训练阶段的防御能力建设和发布前的安全评测,不能简单地将开源与安全风险画等号。 11. 很多Alignment问题可以落实为具体的奖励设计问题。姚顺雨用教育作比喻,如果一个孩子只有数学考100分才能获得奖励,他自然会把全部精力放在数学上,甚至可能形成其他方面的问题;模型也会按照训练数据和奖励规则优化自己的行为,最终表现可能与设计者真正希望的目标存在偏差。他认为这属于可以通过数据、奖励函数和评测机制不断改进的工程问题,机器也可以辅助人类完成这套对齐工作。 12. 自动化不可避免地会继续深入,但人类需要重新找到自己参与系统的正确位置,需要把人安排在合适的环节,以正确的方式监督AI,他甚至认为,未来一些Infra和算法工作可能越来越不需要人,但数据和信息领域仍然需要人的参与。这也涉及目标定义、数据质量判断、结果验证和现实世界部署等工作。 13. 模型能力的提升通常是连续发生的,但真正引发商业爆发的往往是交互方式发生变化,或者某种过去无法完成的任务突然变得可用。姚顺雨用GPT的发展举例,模型内部能力可能一直在逐步提升,研究人员每天都能观察到细微进步,普通用户却可能只在ChatGPT出现时才突然感受到变化。对于产品创业来说,这提供了两个观察方向:寻找能够改变用户交互体验的新产品,以及持续测试模型能力边界,发现那些刚刚从不可用变成可用的任务。 14. 技术能力、用户价值和商业收入之间存在很大的距离,如果一个模型的能力已经接近GPT,但实际收入只有对方的1/100甚至1/1000,就应该认真检查产品本身的问题。他还指出,商业定价在很大程度上取决于竞争关系,而非单纯取决于产品质量。因此,模型公司的商业机会取决于能否解锁新的应用场景、创造足够大的用户价值,以及在竞争中获得合理的市场份额。即使底层模型的算力和研发资源逐渐集中,围绕模型部署、产品创造和真实业务场景进行整合,仍然存在创业机会。 15. 关于AI公司的组织模式,姚顺雨特别提到了自下而上的创新。他以OpenAI为例,早期公司重点投入的方向包括机器人,但GPT作为一个小项目逐渐展现出价值,随后获得更多资源;后来的gpto1也经历了从数学研究小组逐步发展起来的过程。这种组织方式允许创新从小团队和具体实验中涌现,再根据实际结果决定投入。他认为,很多重要创新在最初阶段并没有明确的商业规划,组织需要给探索性项目留下成长空间。 16. 他对大模型公司追逐量化指标的现象提出了批评,团队负责人不够了解技术时,下属往往会选择汇报容易量化的结果,例如模型参数、数据量和MMLU分数,因为这些指标更容易展示成绩,但真正重要的工作可能是修复一个关键Bug、解决数据质量问题或者让整个系统正常运行。他认为,做大模型最重要的是知道应该做什么,并且坚持把正确的事情做好,即使这件事短期内无法体现在漂亮的指标上。 17. 姚顺雨谈到了一种很有意思的组织文化差异:有些团队以量化指标和赛马机制为中心,谁的分数高就给谁更多资源;而他更看重Trust和Taste,也就是信任与判断力。他以微信、游戏和影视项目为例,这些产品在发布之前很难通过AB Test准确预测最终成败,管理者需要相信团队的审美、经验和长期判断,愿意持续投入时间。 18. 在人才选拔上,姚顺雨认为学习能力比已有经验更重要。他提到,自己看重的团队成员通常具备Low Ego、诚实、靠谱和愿意把事情做正确等特征,同时需要有很强的Co-design意识。大模型研发涉及算法、Infra、芯片、数据和产品等多个团队,一个训练架构的设计就需要综合考虑训练效率、推理效率、后训练能力和成本,任何单一岗位都很难独立完成。一个人能否快速学习、与不同领域的人有效协作,以及是否愿意认真处理细节,往往比简历上已经掌握多少具体技术更重要。 19. 他还提到,不同性格的人其实适合大模型产业中的不同岗位。喜欢明确指标、享受持续进步的人可能更适合预训练和Infra;想象力丰富的人可以尝试算法研究;喜欢思考人文问题、理解用户需求的人可能更适合产品和对话体验;做事认真、责任心强的人则适合生产系统和模型部署。AI产业需要的能力结构非常复杂,没有必要让所有人都去竞争同一种技术岗位。 20. 对于个人职业选择,姚顺雨提出了一个很实用的问题:你要么融入大模型浪潮,成为这套机器的一部分;要么寻找能够与大模型互补的事情。他借用了一个创业判断标准:假设GPT6发布,你的创业公司会因此睡得更香,还是更睡不着?如果模型进步会直接摧毁你的核心业务,就需要重新审视方向。与模型正面竞争人类已经不占优势的认知任务,成本会越来越高,物理世界、硬件、真实实验和对人性的理解等领域,则可能拥有不同的资源壁垒。 21. 姚顺雨分享了自己在清华读书时的经历。他直到大三下才开始做科研,最初也是因为推研要求而被动进入科研,期间尝试过一些当时流行的研究方向,却发现自己并不真正感兴趣。他后来更愿意追随自己对计算机和其他领域的兴趣,也提到身边有人曾经参与说唱社、学习数学,过去看起来不相关的经历,后来可能转化成语言审美、理论理解和研究判断力。他认为很多经历的价值需要经过很长时间才能显现,人生中的弯路未必真的浪费。 22. 他还谈到自己读博最后一年专门选修集合论的经历,因为当时意识到,等真正进入具体工作之后,可能就很难再有时间系统学习那些纯粹出于兴趣的理论知识。他认为数学中的基本结构和大模型的组成部分之间存在很多值得思考的联系,有些知识学习时看不出直接产出,却可能在未来某个时刻与其他经历连接起来。
164
1,377
104
8,006
882,831
🚨 JUST NOW: President Trump reveals he NEVER got confirmation in his reports that Alice Huang — Jensen Huang's affair — was a "Chinese spy" "I get reports on a daily basis it was a woman, it was a pretty woman, and any pretty woman that comes from China, they considered her to be a spy!" The spy claim first came from a New York Times report citing anonymous people briefed on U.S. intelligence Nvidia's own board said it had NO REASON to believe company secrets or national security were compromised Elon AND Trump now believe Nvidia made a poor move in ousting Jensen.
73
236
20
1,431
136,623
Weng Wei retweeted
I believe the @nvidia board is not acting in the best interest of shareholders nor in a manner consistent with their fiduciary duties. What we know is that the board fired Jensen or forced his resignation for violating company policy and/or in response to the accusation of an affair with a spy. They did so without careful consideration of the impact on the company, its customers, its shareholders or other stakeholders. They rushed to judgment without the benefit of careful due diligence or even with a basic understanding of the facts. How do we know this? We know this by how quickly Jensen was fired and how little was known at the time of his firing. It takes time to convene a board meeting. There was time, maybe, for one Zoom. And this is a decision that should have been made by a board sitting in person and carefully deliberating the facts once they are known. And they should have slept on the decision and met again the day after to discuss the facts, the risks and the other implications of their decision. The timing and lack of information did not allow for careful board deliberation. While boards can fire a CEO for an inappropriate relationship with a subordinate, it should not be an automatic decision. That is why we have boards comprised of humans and not agents. A board should consider whether or not such a relationship was consensual or whether it was an abuse of power dynamics. And a board has to consider what the impact of the firing is on all stakeholders. If I were in this board, I would not have rushed to judgment based on articles in the media. I would have made sure the board had all of the facts so it could make a careful and informed decision. Boards of Delaware companies have a duty of care which means they must make informed, prudent and good faith decisions. This clearly did not happen here. What law firm is advising this board? Or is the board even taken good legal advice? It doesn’t look like it. And now the board is compounding its failures by insisting that there is no possibility for Jensen’s return as CEO. Why is it in the best interest of the company to foreclose the possibility of Jensen’s return and to do so before all of the facts are known? This is yet another breach of their duty of care. Furthermore, the board has launched an investigation whose scope is unknown. Investigations can be extremely damaging to a company and should be carefully targeted to address whatever facts need to be known rather than be overly broad in scope. In short, this looks like a series of major governance failures by directors that panicked due to headlines in the media. They likely felt compelled to do something because they were afraid of looking bad in the media, not because it was the right thing for them company. They couldn’t stand the heat. The board’s conduct here is shocking for a company of this scale and importance to our country and the world at large, particularly at this moment in history.
5,071
9,440
1,213
73,586
5,233,986
My calculations: Ascend 950DT vs NVIDIA GB300 training a trillion-parameter model Using Huawei, the TCO is roughly 5 times that of NVIDIA. And it might take Huawei until 2030 to catch up with the GB300. These calculations are basically based on physical limits.
1
121
The so-called CUDA moat, in my opinion, no longer exists in the AI era. Therefore, ultimately, if we delve into the source of the difference, it all points to: lithography machines.
42
Let’s bring Jensen back!!! LFG
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen
121
415
17
8,761
238,710
Weng Wei retweeted
“Some of your intelligence sources are liars. Others are liars and criminals.” Secretary of State Marco Rubio calls out media outlets relying on anonymous government sources, accusing some of feeding reporters false information while exaggerating their access. Rubio warns that having a government email address doesn’t make someone a credible source. Download the Fox News app for more: foxnews.onelink.me/xLDS/z2jy…
315
783
86
4,596
736,952
Weng Wei retweeted
By the way, in the unlikely event that the @nvidia board doesn’t immediately restore Jensen as CEO…there’s actually a scenario where the Department of War steps in on Jensen’s side. Why? Given the likely cyberattacks on Microsoft, flights, banks, and now NSA, it’s arguably a matter of US national security to immediately return the most important person in AI to his role at the top AI company. Defense Production Act, but for the restoration of Jensen Huang.
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen
642
414
205
6,029
581,963
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen
60,896
121,701
24,483
1,173,755
58,509,565
Weng Wei retweeted
Dario Amodei on open source models: “If the path continues, I think we could get to a very dangerous place.” “When you control a model and you're deploying it, you have the ability to monitor its usage.” “You can revoke a user's access. You can change what the model is willing to do.” “When a model is released in an uncontrolled manner, there's no ability to do that. It's entirely out of your hands.” “We should confront the advocates of this with that problem and challenge them to solve it.” He clearly doesn't believe Intelligence should be in the hand of the masses and Sam Altman also saying "RAM" shouldn't be accessible to everyone indefinitely agreed with him.
Anthropic & OpenAI wants to own intelligence. Open models want to free it
944
675
562
4,766
1,591,385
Jensen is reportedly considering a comeback! 💚 More than ever, our voices matter. Let's show him how many of us stand with him. NVIDIANs, shareholders, and supporters, please take a moment to read the letter and sign it. Thank you! #standwithjensen #bringjensenback standwithjensen.com
170
597
79
5,341
396,863
Weng Wei retweeted
Qwen announces smaller Qwen4 models will be released soon! 🔥 Qwen4-27B will run on 16GB RAM setups.
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery! Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window. Highlights: - Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation. - Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables. - Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery. - $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%. The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. Start building today! 🚀 - Blog: qwen.ai/blog?id=qwen4-max - API: qwencloud.com/models/qwen4-m… - Token Plan: qwencloud.com/pricing/token-… More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
261
537
150
6,087
446,922
🚨 BREAKING: GEMINI SUBSCRIPTIONS ARE COMING TO HERMES. 🔥 Today, Gemini in Hermes runs on an API key, not the Gemini plan you already pay for. That's being worked on, the approved way: ▸ Nous is working with Google to bring your Gemini subscription into Hermes ▸ It should be officially supported, with Google's sign-off ▸ Apps plugging in a Gemini plan today aren't doing it the way Google approves ▸ Not live yet, no date Claude and ChatGPT subscribers can already bring their plan to Hermes. Gemini is next in line.
330
458
275
5,026
733,770
BRING JENSEN BACK.
11
42
2
683
8,298
Bessent must be exceptionally loyal to Trump; his public suggestion that Nvidia should replace its CEO might well be a coordinated act—a "double act" orchestrated by Trump himself. This kind of utter, chaotic mess is actually Trump’s comfort zone. The art of deal awaits at APEC
1
179
Weng Wei retweeted
Try the brisket Or else
445
217
151
4,677
293,558