
VP research ops @EngramLab Previously memory research @LongevityCenter @ZuckermanBrain
San Francisco
Joined April 2017
- Tweets36
- Following72
- Followers327
- Likes80
Pinned Tweet
I've spent years studying how human memory works: how we learn and forget, and how our memories shape what we do.
I'm thrilled to share that I've joined the founding team of Engram, which is coming out of stealth today.
Now is the time to understand how to build machines with neural memories. Excited to do it with the best team for the task.
If you're curious, reach out.
Excited to share our new blog post. We built a synthetic law firm with @harvey, containing 100M tokens of legal work, and had Engram's agent study it. The result: a model that knows where to look, and answers better with far fewer tokens.
Natalie Biderman retweeted
In collaboration with @harvey, we’re excited to build a new kind of agent environment to reflect realistic knowledge work: an entire synthetic law firm, Calderwood & Harkness, with over 100M (!) tokens of documents and 250 client cases. 💼
Today’s AI models know a lot about the law, but don’t understand how the law is practiced, because this information remains proprietary within firms. Unlike how agents are benchmarked today – starting each task from scratch with a new set of context — lawyers accumulate knowledge over time, building on years of experience.
Calderwood & Harkness makes it possible for agents to do the same. Legal agents do many tasks in the same environment, making it possible to leverage memory and experience to do better work over time.
We’ve had a great time co-developing this benchmark with Harvey’s research team, partnering @ItsJulioPereyra @nikogrupen @gabepereyra. We’ll share more results on this soon.
Natalie Biderman retweeted
We are working with @EngramLab to train models that can understand a law firm’s knowledge.
A huge source of differentiation for a law firm comes from all the past work the firm has done. When an associate is working on a new deal they have access to decades of similar deals the firm has done and a big part of the job is knowing how to effectively leverage that knowledge.
However, a large amount of this data is either client data or derived from client data which means you can’t naively train all of it into a single model because this would mean breaking confidentially. In order to separate the research problem from the data privacy and deployment problem we built a synthetic law firm.
@ItsJulioPereyra wrote an awesome article describing the dataset, tasks and approach to creating a synthetic version of the type of data you would expect to find at a large law firm. The firm has 46 clients, 266 client matters and each client matter contains work product, versions, emails, drafts, etc that you might find when searching the DMS of a firm.
This dataset allows us to explore how well agents can reason across large corpuses of knowledge (100M+ tokens) and require complicated multi-hop reasoning. For example, answering a query like “in similar deals, how did we structure the reps and warranties” requires the model to understand what makes deals similar and then search over all past deals to find examples.
We find that generic agent approaches for this type of reasoning are both expensive and not exhaustive and there is a lot of room for improvement. Excited to open source this dataset and also share more results soon
RT @harvey: We're open sourcing a 100M+ token synthetic law firm we built with @EngramLab.
The firm contains work product from 250+ synthe…
This quoted post is unavailable.
yes, this actually happened.
Thank you to @Nasdaq for supporting Engram on our launch day yesterday!
Some have commented that this photo looks AI-generated. It's not. This really happened.
Feel free to send this picture to your moms. We're certainly going to.
It's been a pleasure working with James (@buckhouse) and Andreas (@Aweiland). We're grateful to them for helping us think clearly about our story, discover our brand, and bring it to life so beautifully.
We are immensely grateful to @buckhouse @Aweiland from @sequoia for the branding, storytelling, and website. They were generous with their time and ideas and we learned a lot from our process with them.
Natalie Biderman retweeted
I'm really excited to be working with Engram to bring their models into Custom Agents in Notion. There's something magical that happens when the agent stops needing to search your workspace, and just knows it.
Natalie Biderman retweeted
Most times I use an agent it spends minutes rereading all my stuff – spending dollars and energy just to gather the context it needs to understand my work.
But it won’t always be this way. We can train that understanding into the model – amortizing across tasks.
We started Engram to build the systems required for a future where there are billions of models, each learning from users and deeply understanding their work.
I’m so grateful and excited to be working on this problem with this incredible team
Natalie Biderman retweeted
Engram is making a personal AI that really learns and internalizes everything you care about. It’s a different kind of lab with an amazing team put together by @dan_biderman @EyubogluSabri @realJessyLin and @jxmnop. Very excited for what they are building!
Natalie Biderman retweeted
So thrilled for @EngramLab, working on such an important problem!!! Had the pleasure of working closely with @EyubogluSabri @dan_biderman @MayeeChen @NeelGuha and many of the team members for most of the PhD and they are the absolute best out there. Can't wait for all they'll do! 💫
Natalie Biderman retweeted
I met @dan_biderman Biderman and the Engram team only 6 months ago, and I’m still blown away by what they’re building. It's so obviously needed yet so hard to build true AI memory for enterprises. The quality of the team, the scale of the ambition, and the pace of execution!!!🔥
Natalie Biderman retweeted
Working with @dan_biderman and the @EngramLab team has been great, excited for whats next
Natalie Biderman retweeted
Replying to @dan_biderman
@dan_biderman is an amazing researcher and leader. I'm amazingly proud to call him my former intern.
He has assembled a world-class team, and I truly believe they have the ability and resources to transform the way we build relationships with language models.
The single biggest impediment in AI isn't absolute intelligence or coding ability. It's deeply understanding large repositories of knowledge that every person and every company has.
Our repositories will explode in size, with AI agents writing much of them.
This is a two-way street: we need to understand what our AI generates, and AI needs to better understand us. A breakthrough in memory is needed – gradients need to be taken, and our team is the right one to take the big swing.
Natalie Biderman retweeted
Replying to @dan_biderman
@dan_biderman and @EngramLab are building what every enterprise asks me for:
Specialized AI that activates the context they actually own, not the labs
It’s an honor to be part of their journey and we @rox_ai are excited to partner and arm the “rebel militia,” as Dan calls it. 🫡
Very proud of this man ❤️
Part of my speech at the Columbia Medical Center PhD hooding ceremony:
“The world we’re graduating into now has machines that review literature, write code, analyze data, and even draft manuscripts that pass peer review. Our scientific profession is shifting beneath our feet. Where does this leave the graduating class? I suggest we focus on three things: deep domain knowledge, scientific taste, and character.
First, deep domain knowledge. AI can answer general questions well, but when you go deep, you find that the AI is wrong often, and sometimes in fundamental ways. The world will soon be flooded with AI-generated content. As PhDs, we become world experts in our domain, among the few able to detect subtle AI hallucinations. We can also be a part of the solution, if we so choose. We can help shape AI to better serve our fields.
Second, scientific taste. People talk about the “AI scientist”—a machine that reads papers, spots gaps, and proposes new experiments. This is starting to happen. We shouldn’t compete, but collaborate. As PhDs, our edge lies in intuition: we can guide AI more directly and elegantly toward the discoveries. Such collaboration will also be cheaper and consume less datacenter energy. Remember: we know many things that the internet doesn’t. Unpublished results, failed experiments, and informal conversations over the years. We may better know what to try next and what to avoid.
Third, is our character. In a world where every scientist has access to superintelligence, what sets us apart is not what we know, but how we lead. Memorizing the literature and crunching numbers will not be our competitive edge. The scientists who shape the future will be the ones who can guide teams, make good judgments, and act with clarity.”
How can we use small LLMs to shift more AI workloads onto our laptops and phones?
In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@openai, @together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost while maintaining 97.9% of the accuracy.
See Gru and the Minions in action below, 🔉on please (h/t @cartesia)!
amazing work by an amazing team. and i'm not biased at all :)
⚡️Announcing️Lightning Pose⚡️
A pose estimation method that learns from both labeled images and a lot of unlabeled videos, via inductive biases like temporal smoothness, posture feasibility, and multi-view consistency.
Preprint: tinyurl.com/litpose
Ever noticed when your chosen option turns sour, the unchosen suddenly seems appealing? @DShohamy, @gershbrain and I have shown that this counterfactual inference is a unique signature of an internal policy-gradient model of choice and it is driven by memory.
Replying to @Natalie_Bid
@Natalie_Bid, @DShohamy, and I have some new work suggesting that people apply policy gradient algorithms to experiences retrieved from memory: gershmanlab.com/pubs/Biderma…























