Jack Cole (@MindsAI_Jack) header
Jack Cole (@MindsAI_Jack) profile picture

AI Research, PhD in Clin Psych, App Dev 40M+ App Downloads, Competitive ML

Illinois, USA
Joined April 2022
Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)Photo by Jack Cole (@MindsAI_Jack)
Thanks. The fact that it works so well for ARC is also a clue to the extent of generalization in these models. It was not at all obvious that it would work originally because the model is never trained on the actual answer. @fchollet 's ARC-AGI benchmarks are an invitation to understand efficient generalization. I feel strongly that gradient updates at test time are the doorway to a new level of AI (smaller, more capable models with actual memory).
Test-time training was popularized during the ARC Prize 2024 competition, after being explored in particular by @MindsAI_Jack and team. To date, I believe ARC 1-2 are the only datasets where TTT strongly outperforms. It would be interesting if TTT started becoming more mainstream. I believe it has great potential.
3
3
47
16,935
Was I right?
Replying to @kimmonismus
All the labs are going to release extremely good models ahead of @AnthropicAI's IPO would be my guess. Ant is holding back Fable 5.5 for when that happens. Going to be an interesting 9 days.
1
1
8
336
I'm calling it. Neon is better than Opus 5.5. This model is absolutely brilliant.
31
28
4
1,171
29,057
Congrats. Google has arrived. Looking forward to seeing what you do next!
Below Neon 3D modeled a photorealistic lab simulation in Antigravity via Blender MCP 🔥
3
298
I missed this. Thanks for this analysis. @mhmazur Is the human baseline an average or the best of all humans?
Replying to @arcprize @OpenAI
Across the 25 ARC-AGI-3 public games at max reasoning, GPT-6.2 Astra used 64.1% fewer actions than the human baseline and 5.2% fewer than GPT-6 Astra with the provider adapter harness:
1
7
579
Wow. It would be an extremely rare human who could do this (maybe none on ARC 3). It would be interesting to see the comparison against the human distribution. Good ideas for the future benchmarks Greg.
This is the first 100% we've seen on v3 The way we've measured models GPT-1 through GPT 6.2 won't hold up for GPT-7+. We need a categorically new benchmark Three ways to create benchmarks: 1. Out-of-distribution tasks A models knowledge coverage looks like swiss cheese (not exactly, but you get the idea). You can always find gaps in knowledge as you niche your topic. Ex: Animals identification > Mammals > Dogs > Herding dogs > Border collies This is the *easiest* way to create a new benchmark, it's also the *least* robust. Depending how much the labs care, you're only 2-3 months of model releases away from your swiss cheese gap filling in. 2. Increase task difficulty Take an existing domain of tasks which models saturate and make the problems harder. Hire PhDs and experts to come up with problems. This is operationally intensive and harder to execute. You want to avoid cheap difficulty (a larger maze) and lean into rich difficulty (more reasoning required). 3. Measure the frontier This is the highest signal (imo) way to test models, it'll be the defining benchmark for post-AGI systems. Instead of measuring performance on a static set of tasks, measure the rate at which models push the frontier of knowledge forward. The ultimate environment is reality/physics. Net new knowledge here is proof of innovation. However, physics is hard to benchmark with, sythenetic environments will do ok here in the short term. To be clear, this is an incredibly high bar to hold models to. Maybe the highest? All RSI, prosperity, advanced capability questions are downstream of AI's ability to create net new knowledge. Once you have a system that can autonomously create new knowledge in any domain (like human as a collective can) then every other domain is a harness/training session away. TLDR: Static task completion is no longer an adequate way to measure frontier models. Rate of learning is the last hurdle. Net new knowledge (what humans don't know) is the most robust setting to measure this.
11
712
Do these look like recursive self improvement or repetitive slop injection?
3
5
1
22
1,908
“If you’re cramming all this stuff into your brain all the time, then you don’t have time for [new thoughts] to happen,” Earl Miller, a professor of neuroscience at MIT. Headphones, podcasts, music, and streaming are distracting us from our own thoughts. nypost.com/2026/11/16/tech/h… – at Somerville, MA
17
45
4
200
5,859
Strong results.
Inkling 2 Small from @thinkymachines on ARC-AGI (Verified): - ARC-AGI-3: 2.6%, $704 total (provider adapter); 0.3%, $858 total (standard) It beat DeepSeek V4.1 Flash 1017 on ARC-AGI-1 (96.5% vs 94.5%) and ARC-AGI-2 (81.7% vs 78.1%), but trailed on ARC-AGI-3 (2.6% vs 5.8%).
1
12
766
Astera Institute Residency applications are open. Very unique opportunity to build your project in a stimulating and supportive environment! Check it out, and tell your friends about it!
Applications are open for Astera Residencies, now through January 7, 2027! Up to $2M in funding, based out of our Emeryville hub. This round, we’re sharing a few specific project requests. Read more and apply here: astera.org/residency/ Reshare with folks who should see this!
1
8
28
2,204
Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached. The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network. This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc. I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
208
517
127
4,323
477,491
Jack Cole retweeted
"Latent Feedback Is a Compute and Data Amplifier" Repeatedly training LLMs on the same data gives diminishing returns, but latent feedback provides a more efficient way to use that compute. This paper revisits Latent Feedback Transformers, which feed each token's final hidden state into the next token's input, letting the model to reuse information across consecutive positions. Across models from 60M to 778M params, latent feedback saves up to 48% of training compute or 41% of unique training data in measured comparisons at matched validation loss. A new scaling law also predicts performance on a held-out 778M model within 0.3%. alphaxiv.org/abs/2611.latent…
3
12
3
126
9,688
True. If AI is not smart enough to figure out how to elevate and support humanity, it is not AGI.
It's not AGI until it's a net job creator.
2
1
7
538
looks very interesting! Congrats @StphTphsn1
"A method to automatically discover symbolic local learning rules" –– expertly led by Andrea Perin (@perina_ndrea), & in collaboration with Fabio Anselmi (@Fabio_Ans) 🔗 arxiv.org/abs/2611.12248 accepted at #NEURIPS2026 (Concept & Feasibility Track)
2
2
32
2,419
Interesting. It seems like Gemini Carbon might get better performance on ARC from additional test time compute rather than stronger skills.
Gemini 4 Carbon from @Google on ARC-AGI (Verified): - ARC-AGI-3: 99.1%, $4.6K (provider adapter), 65.1%, $7.2K (standard harness) - ARC-AGI-2: 94.2%, $0.38/task - ARC-AGI-1: 98.5%, $0.16/task Carbon beat its predecessor Argon by 18% points on ARC-AGI-3 in the standard harness.
4
9
818
Jack Cole retweeted
I have never seen so many companies trying to kill their technology.
🤖 Made with AI
1
3
182
Gemini 4 Carbon is rolling out today. It’s built for sustained, long-horizon tasks, reasoning deeply through complex software engineering, cybersecurity, and scientific discovery. Coming globally via the Gemini API in @GoogleAIStudio and Gemini for business, and for Ultra subscribers in @antigravity, the @GeminiApp, and AI Mode in Search.
570
604
266
8,564
799,453
At AI Safety Co - The only people who could be better than us at keeping us from killing you are the politicians. - We will take all your jobs, but maybe the politicians will help with that? - We might cure cancer and other diseases for a few years before we kill you all. P.S., May we please build a data center in your back yard? P.P.S., Yes, we know the politicians haven't been able to do incredibly simple things like get rid of the yearly time change, but they can probably cut our chances of killing you by some percentage, because they are great at AI research. Your jobs are toast though, so hopefully they pay you something. We know they care about you.
2
4
320
Why have no spies been found at AI Safety Co? Time for a DVQ on that issue with Claude.
2
282