reward hacking @primeintellect

sf | nyc
Joined February 2015
would not be a bad time for all of those theorem-proving startups to pivot into software especially now that math is over
7
1
1
103
4,820
will brown retweeted
Prime agent #2 on FrontierMemory ranking after Letta Dream! Great to see Prime Agent do so well on this new self-improving memory benchmark. My only nit: Let's add agent versions to results!
Replying to @Letta_AI
FrontierMemory also allows measuring the ability of agent harnesses to improve over time. We find that Letta Dream and @PrimeIntellect's Prime Agent improve the most from learning from prior experience data.
5
4
26
1,990
it's kinda wild to me how many people are long short-term memory
11
3
3
196
8,921
"neural networks are general purpose computers"
There is an (unpatchable?) supply-chain zero-day in PyTorch autograd that allows a regularization term to use the model's weights as RAM and SGD as its CPU clock to run any computation during training.
6
26
368
11,404
will brown retweeted
we are hiring applied researchers to work on scientific AI at prime intellect. you'll operate at the frontier across scientific domains with some of the most ambitious companies and labs building in the space - join us!
Replying to @willcb
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy. we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see. today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking. some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster. we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously. we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
5
5
1
164
8,819
will brown retweeted
Replying to @willcb
@willcb is deriving me from first principles tomorrow.. We are going to need an L0.5 for continual learning
1
1
1
20
2,543
“continual learning in the harness” is like if a computer had a cache and a disk but no RAM. like sure it could technically work but it’s clearly not optimal. you want RAM.
24
14
3
373
14,410
👀
Fastest growing startups per Brex data
2
86
6,318
will brown retweeted
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy. we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see. today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking. some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster. we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously. we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
10
7
2
178
15,591
over the past few months we've been rapidly scaling our partnerships with leading companies across a bunch of incredibly exciting domains we collaborate with their in-house experts to deploy continual learning loops in prod hiring for many new roles :) primeintellect.ai/careers
9
8
1
212
10,320
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy. we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see. today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking. some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster. we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously. we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
10
7
2
178
15,591
will brown retweeted
Excited to announce that I've joined @PrimeIntellect! Our mission is to build the open training stack and let people own their intelligence. Prime Intellect has always gone a step beyond open-weight models and strived for a fully open-source ecosystem by sharing everything you can think of: training infra, data, evaluations, inference, algorithms, harnesses, and more. In just a few weeks, I’ve already learned so much from an incredibly goated team with deep expertise across the entire model stack. Looking forward to building frontier open models with the team. Watch out for what we are cooking, big things ahead!
43
15
445
17,106
“if doing stuff is easier now then why are we all more busy” bro jevons
26
16
3
460
15,290
when the tokens are a little too far off-policy
1
1
74
3,398
the new zeroth-order speedrun optimizers all have incredibly high aura names. Dust. Boost->Cut. Relay. Drift Leap. hell yeah dude those sound like some pretty sick algorithms
14
10
232
9,543
Dust was what got me over the hump from thinking "nah this is never gonna work" to "wait they might be cooking"
4
4
2
94
15,834
had some pretty significant updates on where the puck is going lately, methods-wise
5
3
267
19,457
one tidbit i’ll share is that i’m now mostly convinced that engram modules (or things like them) will be fundamental for continual learning and aren’t just an efficiency trick
8
1
1
164
9,109
there’s a very cute reason why, which i’ll save for another time
2
62
3,620