Dmitry Kislyuk (@dkislyuk) header
Dmitry Kislyuk (@dkislyuk) profile picture

somewhere at the intersection of evolving networks and backpropagation.... ML things @pinterest @stanford @ucberkeley

San Francisco
Joined February 2010
Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)Photo by Dmitry Kislyuk (@dkislyuk)
Writing notes has been as important as ever to me in the era of infinite learning via LLMs dkislyuk.com/writing-textboo…
1
3
70
Prof. Stoica's full statement against SB 1047 should be taken seriously — mandatory audits for matrix multiplication are a bad idea.
Local politician @Scott_Wiener: I will make AI safe! Scientists: You are literally doing the opposite "This bill...will hurt the innovation in California, and it will result in a more dangerous rather than a safer world." - Prof. Ion Stoica (UC Berkeley) on SB 1047
4
372
Dmitry Kislyuk retweeted
The James Webb Space Telescope has started capturing images of galaxies so far away that they are causally disconnected from the Earth — nothing done here or there could ever interact. 🧵 1/
770
3,333
794
27,660
5,735,350
I've thoroughly enjoyed going back through the @TheRestHistory podcast catalog — the dry British humor from @holland_tom and @dcsandbrook makes for a wonderful companion and complement to the other giants of the space (Dan Carlin, Mike Duncan, etc.)
1
1
1,565
good luck with the detection occlusion labeling guidelines on that first example...
What’s the most difficult problem in computer vision? Cows. It’s always cows.
1
201
Andrew is the strongest ML engineer I've had the privilege of working with. Worth a follow!
appreciate it @cosminnegruseri 🙏. one of the best coworker and friend someone can have :). I'm new to X but will keep sharing insights here Pinterest was really fun and I'm lucky to have had a career that compounded for 9 years. The key was a _really_ amazing team and supportive manager. while making some calls that in hindsight was right: 1⃣[2013] building vector dbs @ berkeley. and becoming one of the very few that knew how to build billion document scale embedding search @ sub 100ms latency. 2⃣[2014] all in on deep learning @ pinterest. after seeing how much better alexnet embeddings were compared with SIFT features for CV. luckily deep learning was a bigger trend than just embeddings. 3⃣[2015] studying deep learning foundations @ stanford (@karpathy 's cs231n was key). handwriting backward passes was fun 4⃣[2016] adopt pytorch in prod early. learned what worked and didn't from a team-wide training framework setup perspective. and later being _the_ expert at pinterest. 5⃣[2016-] start building embeddings for everything. Later being integrated in everything at pinterest (150+ use-cases across 10+ embedding entities) 6⃣[2018-] viewing the ideal rec system as e2e ML learned and working towards replacing heuristics to learned. (e.g. user as sequence of clustered content embeddings => e2e learned user representations in embeddings and ranking) 7⃣[2021-] commit to moving the company towards one shared pytorch-first training framework. And learning how to sell that to execs / engineers. Worked and made it much easier for research => deployment to occur as well as deep investments in ML efficiency
3
255
Thanksgiving dinner topic with my parents: why is von Neumann absent from the Oppenheimer film, while plenty of other "cameos" make it: Gödel, Fermi, Szilard, etc. Our conclusion is that he is a future Nolan film subject.
199
Regulatory capture around matrix multiplications seems like a bad idea.
1
141
It makes me immensely happy to see this kind of research being done in AI. Growing neural networks seems like a satisfying alternative to designing model architectures by hand.
Introducing Neural Developmental Programs (NDPs)🧬🧠Instead of neural networks with fixed architectures, we allow neural networks to grow through a dynamic self-organizing process, inspired by how biological nervous systems develop👇 PDF: arxiv.org/abs/2307.08197
250
Dmitry Kislyuk retweeted
the most important animation of how papyrus scrolls are made, carbonized, and resurrected you will watch today it's time to read the scrolls! scrollprize.org
11
68
12
451
174,201
An AI license? As in, a license to do big matrix multiplications and gradient descent? Pretty disappointing that the discourse is moving there. If there's regulation, it should be addressing specific harms at the application layer, not at the modeling stack.
1
137
Modeling "agents" in A-life / neuroevolution experiments is too restrictive and feels like a dead end. My mental model is that the modeled "entity" has to be an entire environment or biome (open-ended environments) for interesting intelligent behavior to emerge.
A ridiculous open question that confounds artificial life (and trad biology): what exactly constitutes “something”, and how is it separate from the environment? A classic example is the “agents” that emerge in critical cellular automata: their agency is kind of just our opinion
1
1
331
Congrats @jeffreyhuber and @atroyn on making embeddings easy to work with!
I'm thrilled to announce Chroma's $18M seed round led by Quiet Capital We are excited to continue to invest deeply in the project and make it a ubiquitous open source standard trychroma.com/blog/seed
2
297
I think the most interesting neural networks in the near future will have dynamic computation graphs with highly variable latency: it shouldn't surprise us that certain queries need 1000x more computation!
Most LLM prompt hacks — eg “let’s think step by step” — are basically introducing looping to a feed forward network. Every new token generated is a loop, and the more you give the LLM space to “talk” the better it can compute because looping is essential for that.
3
139
Interestingly, seems like this is already an obsolete paradigm. Turns out it's much easier to just teach the model about system calls, plugin APIs, and other tools :)
This work, along with the Tracr compiler (github.com/deepmind/tracr), opens up some fascinating research directions. Compiling code into weights = "the potential to fuse hardcoded models with larger pretrained transformers, in order to harness the strengths of both".
164
An adaptive, personal tutor could be really interesting if coupled with a personal knowledge base (obsidian, notion, etc.) example: a ControlNet-esque model which learns on your existing knowledge, then guides the bigger global model on the most pedagogical explanation for you
Turn ChatGPT into a (pretty good) adaptive tutor with a paragraph: I want to do some interactive instruction. I want you to start explaining the concept of BATNA to me at a 10th grade level. Then stop, give me a multiple choice quiz, grade the quiz, and resume the explanation. If I get the quiz wrong, reduce the grade level by 3 for the explanation and language you use, making the language simpler. Otherwise increase it by three and make the language harder. Then quiz me again and repeat the process. Its not 100% consistent, but shows the potential power of LLMs as educational tools.
1
1
197
This work, along with the Tracr compiler (github.com/deepmind/tracr), opens up some fascinating research directions. Compiling code into weights = "the potential to fuse hardcoded models with larger pretrained transformers, in order to harness the strengths of both".
"Looped Transformers as Programmable Computers" Shipped today, long thread with arxiv link tomorrow. This paper marks a significant shift in my group's research, that I'm very excited about.
1
351
Dmitry Kislyuk retweeted
The life of a neuron is anything but simple — a live image movie from our lab!
17
188
16
1,060
102,691
Fun to think about. Maybe in a decade or two, CS will be taught as a "foundation" for some yet-undefined <prompt architecture / AI interaction> major, akin to what EE is to CS majors today. It'll have its place, but it'll be one rung below the apex in the abstraction ladder.
What is time horizon until 50%, 90%, 99% of “important” code written solely or predominantly by AI? At what point should one not recommend CS as a great major for jobs? If not CS then what**? (**Note: liberal arts as suggestion not realistic IMO)
141
I am convinced that the Garmin race predictor is programmed to goad me into thinking “Oh that’s ridiculous, going to go out there and prove this thing wrong!” (and record the activity with a Garmin, of course)
Me: splits 15:32 for 5k in the middle of a 5 mile workout Garmin Race Predictor: After seeing that, you’re in shape to run 16:18 for 5k
1
177