ito66 retweeted
Don't make the mistake of thinking this is about his infidelity. These articles will not stop until he agrees to follow Anthropic's proposal to block inference for unsigned and registered open-models in the firmware of modern GPUs.
This is a pretty transparent concerted effort at centralizing control over compute ressources and the ability to execute ever more powerful local models. Jensen is the tip of the spear pushing back at this (in part for his own economic reasons).
I'm team Jensen, his affairs are none of my business, not even the alleged support of circumvention of export controls are my business, his position on open-weight models is.
Jensen Huang must go
We argue in @TheEconomist that his position is untenable
"Like many powerful leaders, he came to believe that he was too important for the normal rules to apply to him."
But he is not the only one. Howard Lutnick has failed to stem the flow of chips to China. More heads should roll
ito66 retweeted
Yann LeCun was right & nobody in this industry wants to say it out loud.
109 million parameters beat 72 billion.
He has spent years saying scale is not the thing that makes a model understand the world. Meta bet the other way. He left in November and started his own lab in Paris.
Last Thursday he published the proof.
The theory everyone has been running on since 2024 is that if you train models big enough, on text, on images, on sound, they all end up with the same picture of reality inside them. Size does it. Nobody ever proved that part.
Randall Balestriero and LeCun proved it. And the proof says size was never doing the work.
What does the work is which two things you show a model together.
A photo and the caption of that photo are two views of one thing. Models trained on pairs like that get there fast.
Two paragraphs pulled from the same document are not two views of one thing. Models trained on those get there slowly and expensively.
Next word prediction is the second kind. That is what every chatbot in the world runs on.
In their tests the first method reached the same understanding after 21.6 million examples. Next word prediction needed 900 million.
And a 109 million parameter text model ended up closer to an image model's view of the world than a 72 billion parameter language model was.
660 times smaller.
ito66 retweeted
set your phone to airplane mode before takeoff or we will all die. yes you can bring your deadly device onto the aircraft, just select the option to not kill us all after takeoff. we're not gonna check, it's not that important, but you definitely have to do it or we will perish.
