Pavel Suchmann retweeted
The hardest problems are rarely solved by adding more complexity to the solution -- they are solved by reframing the question until a simpler, clearer answer reveals itself.
Pavel Suchmann retweeted
Replying to @fchollet
The hype cycle around AI has always been fascinating. We tend to simultaneously overestimate its short-term impact while underestimating its long-term potential - it's been this way since the early days of computing.
AI winter again?
I don't wanna say "I told you so", but I told you so.
Quote: "Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued." ...
threads.net/@yannlecun/post/…
66
Pavel Suchmann retweeted
Meta will *not* release the multimodal versions of its AI products and models in the EU because of an unpredictable regulatory environment.
This means that EU users of Ray-Ban Meta won't be able to use the image understanding features.
It also means that the EU industry will not have access to future multimodal versions of Llama-3.
theverge.com/2024/7/18/24201…
Pavel Suchmann retweeted
- Regulators should regulate applications, not technology.
- Regulating basic technology will put an end to innovation.
- Making technology developers liable for bad uses of products built from their technology will simply stop technology development.
- It will certainly stop the distribution of open source AI platforms, which will kill the entire AI ecosystem, not just startups, but also academic research.
- the strangest aspect of all this is that all of these regulations are based on completely hypothetical science fiction scenario that very, very few people in the field are plausible.
The effort to protect innovation and open source continues. I believe we’re all better off if anyone can carry out basic AI research and share their innovations. Right now, I’m deeply concerned about California's proposed law SB-1047. It’s a long, complex bill with many parts that require safety assessments, shutdown capability for models, and so on.
There are many things wrong with this bill, but I’d like to focus here on just one: It defines an unreasonable “hazardous capability” designation that may make builders of large AI models liable if someone uses their models to do something that exceeds the bill’s definition of harm (such as causing $500 million in damage). That is practically impossible for any AI builder to ensure. If the bill is passed in its present form, it will stifle AI model builders, especially open source developers.
Some AI applications, for example in healthcare, are risky. But as I wrote previously, regulators should regulate applications rather than technology.
- Technology refers to tools that can be applied in many ways to solve various problems.
- Applications are specific implementations of technologies designed to meet particular customer needs.
For example, an electric motor is a technology. When we put it in a blender, an electric vehicle, dialysis machine, or guided bomb, it becomes an application. Imagine if we passed laws saying, if anyone uses a motor in a harmful way, the motor manufacturer is liable. Motor makers would either shut down or make motors so tiny as to be useless for most applications. If we pass such a law, sure, we might stop people from building guided bombs, but we’d also lose blenders, electric vehicles, and dialysis machines. In contrast, if we look at specific applications, like blenders, we can more rationally assess risks and figure out how to make sure they’re safe, and even ban classes of applications, like certain types of munitions.
Safety is a property of the application, not a property of the technology (or model), as @random_walker and @sayashk have pointed out. Whether a blender is a safe one can’t be determined by examining the electric motor. A similar argument holds for AI.
SB-1047 doesn’t account for this distinction. It ignores the reality that the number of beneficial uses of AI models is, like electric motors, vastly greater than the number of harmful ones. But, just as no one knows how to build a motor that can’t be used to cause harm, no one has figured out how to make sure an AI model can’t be adapted to harmful uses. In the case of open source models, there’s no known defense to fine-tuning to remove RLHF alignment. And jailbreaking work has shown that even closed-source, proprietary models that have been properly aligned can be attacked in ways that make them give harmful responses. Indeed, the sharp-witted @elder_plinius regularly tweets about jailbreaks for closed models. Kudos also to Anthropic’s @cem__anil and collaborators for publishing their work on many-shot jailbreaking, an attack that can get leading large language models to give inappropriate responses and is hard to defend against.
California has been home to a lot of innovation in AI. I’m worried that this anti-competitive, anti-innovation proposal has gotten so much traction in the legislature. Worse, other jurisdictions often follow California, and it would be awful if they were to do so in this instance.
SB-1047 passed in a key vote in the State Senate in May, but it still has additional steps before it becomes law. I hope you will speak out against it if you get a chance to do so.
[Original text (with links): deeplearning.ai/the-batch/is… ]
Pavel Suchmann retweeted
Umberto Eco, who owned 50,000 books, had this to say about home libraries:
"It is foolish to think that you have to read all the books you buy, as it is foolish to criticize those who buy more books than they will ever be able to read. It would be like saying that you should use all the cutlery or glasses or screwdrivers or drill bits you bought before buying new ones.
"There are things in life that we need to always have plenty of supplies, even if we will only use a small portion.
"If, for example, we consider books as medicine, we understand that it is good to have many at home rather than a few: when you want to feel better, then you go to the 'medicine closet' and choose a book. Not a random one, but the right book for that moment. That's why you should always have a nutrition choice!
"Those who buy only one book, read only that one and then get rid of it. They simply apply the consumer mentality to books, that is, they consider them a consumer product, a good. Those who love books know that a book is anything but a commodity."
Pavel Suchmann retweeted
Replying to @elonmusk
Join xAI if you can stand a boss who:
- claims that what you are working on will be solved next year (no pressure).
- claims that what you are working on will kill everyone and must be stopped or paused (yay, vacation for 6 months!).
- claims to want a "maximally rigorous pursuit of the truth" but spews crazy-ass conspiracy theories on his own social platform.
Pavel Suchmann retweeted
Polostan, my new novel, is coming out October 15. It’s the first book in a new series I’m calling Bomb Light. More info to come as we get closer, but for now: harpercollins.com/products/p… @wmmorrowbooks
Interesting discussion about LLMs and reasoning.
This Post is from an account that no longer exists. Learn more
2
42
Pavel Suchmann retweeted
That memorization (which ML has solely focused on) is not intelligence. And because any task that does not involve significant novelty and uncertainty can be solved via memorization, *skill* is never a sign of intelligence, no matter the task.
Pavel Suchmann retweeted
Not your model, not your GPTs.
All those folks rushing to add stuff to the GPT store are writing free functions for another OpenAI llm that they will brand as “AGI”. Almost free labor extraction. I will bet, for most folks, the revenue share will be pennies. Folks who think this is an iOS App Store moment are deluded.
Be more transparent and support open source or change the name from OpenAI to something less hypocritical.
Pavel Suchmann retweeted
Extremely Cool! Mamba, announced today, is a structured state space model that challenges the dominant Transformer architecture.
Transformers are computationally inefficient, esp. when it comes to extended contexts.
With some clever improvements, Mamba enjoys fast inference (5× higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences.
On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size in pretraining and downstream evaluation.
It's great to see alternate architectures that improve upon Transformers. 😍
We will need more research and bigger models before we can apply these models in real-world use cases. Given the pace of open-source development, that day will come soon!























