Ferdinand Kapl (@KaplFer) profile picture
Munich, Germany
Joined July 2023
Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)Photo by Ferdinand Kapl (@KaplFer)
Looped LLMs are everywhere right now! But why are they interesting? Do they model language differently than regular LLMs? Grown and looped transformers are better sequence models! With @kaitlinmaile & @oswaldjoh (@Google, Paradigms of Intelligence Team) and @stefanAbauer. 🧵
8
27
4
143
8,444
Paper: arxiv.org/abs/2610.31421 More results (sizes, tasks, additional head analysis) in the paper! Last in a series of works with @Google Paradigms of Intelligence Team that I am very excited about arxiv.org/abs/2512.08819 arxiv.org/abs/2602.16490 Let's see where this grows next!
Excited to share “Growing to See”: growing SSL vision models provide better representations with up to 25% fewer FLOPs! Co-led with @KaplFer. With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer. 1/13
1
3
16
765
This project started with a literature search on "growing" work in modern vision pipelines, and I was surprised that there was not much, although there have been more recent works in language. Since I started my PhD by working on vision (object-centric?!) I was interested.
Excited to share “Growing to See”: growing SSL vision models provide better representations with up to 25% fewer FLOPs! Co-led with @KaplFer. With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer. 1/13
1
4
10
448
Early to middle layers contain more information (better downstream metrics vs. baseline), and early layers are more important than late ones (skipping early layers has a bigger and later smaller influence vs. baseline). For more details check out the thread or the paper!
1
1
19
There could have been no one better than @AMK6610 to work on this together. He really is one of the kindest and most meticulous people I know in research, and I am happy to have started my PhD journey with him! Fyi: He will graduate soon and is looking for positions in Sweden.
1
23
Ferdinand Kapl retweeted
Excited to share “Growing to See”: growing SSL vision models provide better representations with up to 25% fewer FLOPs! Co-led with @KaplFer. With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer. 1/13
1
12
1
38
2,698
Ferdinand Kapl retweeted
Today we release crowd-code 2.0, our second phase of crowd-sourcing the largest long-horizon open software engineering dataset. Install once. Forget about it.
We introduce crowd-code 2.0: An open system for crowd-sourcing long-horizon human software engineering traces as action-observation rollouts Today’s models have short horizons not because of architectural bottlenecks, but because we don’t train them on months-long human behavior
2
1
2
18
1,447
SOTA LLMs are trained as large monolithic models. But looping or depth growth can beat that. How? Co-led with @AngelisManos and @TobiasHppe1. Incredible collaboration with Paradigms of Intelligence Team @Google 🙌 @kaitlinmaile @oswaldjoh @ninoscherrer and @stefanAbauer. 🧵🔽
1
12
1
19
7,668
🔮 Outlook: towards Looped Transformers. With depth growth, we see compact computational blocks repeated along depth with an internal cyclical structure. This is reminiscent of Universal Transformers: loop-like computation emerges from depth growth without explicit weight tying.
1
3
178