calvin

?

so i have experiments that show pretty convincingly, tensor-train style decomposition works for modeling exact AR joints over high dimensional data tractably. i did lit search and only found recent papers which apply it very narrowly, without realizing the larger implications

Weight-Space Programming is fascinating, but the hype is getting ridiculous. We spent decades moving away from manually engineering neural networks toward learning representations through gradient descent. Now an LLM tweaks a few weights, fixes a pathological reasoning loop, and suddenly we’re witnessing a revolutionary new programming paradigm. Let’s be clear: fixing a broken model is not the same as teaching it something new. There’s something deeply ironic about using a model trained by gradient descent to declare that gradient descent is becoming obsolete. Old ideas, new vocabulary, a frontier model in the loop, and suddenly it’s a paradigm shift. Sometimes the biggest innovation in AI research is the marketing.

读博之后才慢慢意识到，科研最容易让人陷入的一种错觉，就是觉得每一分钟都应该有产出。 没有读论文，没有写代码，没有推进实验，就会莫名产生一种负罪感。 但人的大脑不是 GPU，不能靠持续满载来获得最好的结果。 很多真正重要的想法，反而是在散步、看电影、喝咖啡，甚至无所事事的时候突然出现的。 休息不是对科研时间的浪费，它本身就是创造性思考的一部分。 我们应该允许自己拥有一些不需要被证明有价值的时间。 毕竟，人生也不应该只是为了下一篇 paper 而活着。

This is an old hat. That's exactly what the neural fast weight programmers did in the 1990s (including the 1991 unnormalised linear Transformer). Overview with links to the original papers: people.idsia.ch/~juergen/fas…

Introducing NotebookLM for arXiv Papers 🚀 Transform dense AI research into engaging convos you can hear on the go With context from thousands of related papers, it captures motivations, draws connections to SOTA, and explains key ideas like a prof who's read the entire field