
Engineering Lead for Agentic Infrastructure @databricks | Scout @greylockvc (angel investing in data/ML/cloud) | prev: @stanford @ucbrise
San Francisco, CA
Joined August 2013
- Tweets876
- Following776
- Followers609
- Likes30.7K
Ankit Mathur retweeted
I came back to Google to work on AI infra. Instead we got pulled in to solving the problem of exploding use of agents across the company.
Announcing Agent Compute Engine: bringing planet scale agentic infra to Google services and now to Google Cloud.
cloud.google.com/blog/topics…
Ankit Mathur retweeted
What if the next big leap in AI agents comes from better memory, not bigger models?
@theDeepView explores MemCurrent. In our internal benchmark of 114 enterprise tasks, agent success rose from 22.3% without memory to 68.4% with episodic and derived memory.
thedeepview.com/articles/why…
Ankit Mathur retweeted
Here’s how we’re building memory in Genie and the new Agent Bricks Memory service you can use in your own agents. MemCurrent automatically synthesizes new memories from traces to produce better results than alternatives.
Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero.
MemCurrent combines episodic memory with derived memory across three access patterns:
• Plugins to log and load context automatically
• Tools the agent calls on demand
• An offline “dreaming” pipeline that consolidates memory in the background
Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session.
On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie.
Read the research: databricks.com/blog/memcurre…
Ankit Mathur retweeted
Powered by Lakebase!
Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero.
MemCurrent combines episodic memory with derived memory across three access patterns:
• Plugins to log and load context automatically
• Tools the agent calls on demand
• An offline “dreaming” pipeline that consolidates memory in the background
Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session.
On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie.
Read the research: databricks.com/blog/memcurre…
Ankit Mathur retweeted
Building agents on databricks keeps getting better and better! Come build with us :D
Most agent memory is a guess.
When to distill? What to keep? What to pull back into context?
Guess wrong, your agent gets worse.
Our research team's MemCurrent architecture: 3x more tasks passed, 31% fewer tokens.
Agent Memory is live on @Databricks
databricks.com/blog/introduc…
Ankit Mathur retweeted
Check out our latest research: MemCurrent, a persistent agent memory architecture, which consistently stays on the cost-accuracy Pareto frontier on our internal and three public benchmarks. Try it in the new Agent Bricks Memory and Genie 🚀
Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero.
MemCurrent combines episodic memory with derived memory across three access patterns:
• Plugins to log and load context automatically
• Tools the agent calls on demand
• An offline “dreaming” pipeline that consolidates memory in the background
Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session.
On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie.
Read the research: databricks.com/blog/memcurre…
Most agent memory is a guess.
When to distill? What to keep? What to pull back into context?
Guess wrong, your agent gets worse.
Our research team's MemCurrent architecture: 3x more tasks passed, 31% fewer tokens.
Agent Memory is live on @Databricks
databricks.com/blog/introduc…
Ankit Mathur retweeted
Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero.
MemCurrent combines episodic memory with derived memory across three access patterns:
• Plugins to log and load context automatically
• Tools the agent calls on demand
• An offline “dreaming” pipeline that consolidates memory in the background
Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session.
On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie.
Read the research: databricks.com/blog/memcurre…
Ankit Mathur retweeted
Factory is partnering with @databricks to bring autonomous software engineering to enterprise data and AI.
Droids can build, migrate, and secure software directly on Databricks, using the data, models, and governance enterprises already trust.
Ankit Mathur retweeted
Can a single database architecture satisfy both vibe-coded and large scale enterprise apps?
The same approach makes small database ultra cheap to run and branch can also make a large one faster and easier to operate. We call this architecture Lakebase.
Ankit Mathur retweeted
Databricks describes AI as the “fastest growing part of our business” in a posting for a Product Finance Director covering AI products and GPU investment strategy.
In August, it reported a revenue run-rate above $7B, growing >80% YoY. Its AI products had already crossed $1.4B in revenue run-rate in February.
Databricks helps enterprises organize and analyze their own data, with machine learning already part of the platform. Generative AI extends that offering across an installed base of 20,000+ organizations, including 70% of the Fortune 500.
The role includes planning GPU demand for production inference, procuring capacity across hyperscalers and neoclouds, and managing utilization and cost per token/query.
Enterprise platforms like Databricks can aggregate compute demand across many customers as AI becomes part of their existing workflows. The explicit inclusion of neocloud procurement is particularly interesting, although the posting doesn’t disclose spending or capacity volumes.
Interested to follow this along…
Ankit Mathur retweeted
Why harnesses need to be neutral
“You want to be able to switch models. The harness can be very tied to the model, or it can be neutral.
The harness’s job is to take the model and interact with everything else, including data, APIs and memory.
If you build memory in a way that is linked to the harness, and the harness is linked to the model, then you are kind of stuck.” @hwchase17
Love to hear your thoughts @charlespacker @HamelHusain @swyx
I have always considered @milesgrimshaw one of the smartest investors in venture.
I remember Sept 2023, Miles came to our recording studio in London having just joined the @benchmark partnership. At the time, he had done one deal; a small company called @LangChain.
Unknown to many but Miles raved about the founder @hwchase17. His excitement for the company was palpable.
Today the company is worth $1.2BN and serves 35% of the Fortune 500.
The other day I sat down with Harrison to discuss the big questions of the day and he allowed me to add my notes from the discussion below.
--
1. What Percent of Developer Salary Do You Spend on Coding Tools and How Has It Changed?
LangChain spends roughly 30% of its engineering salary on coding tools, a share that has fallen over the last two months even as token usage has gone up. The team tracks coding agent usage in LangSmith, sets budgets the VP of engineering can raise, and controls the harness. Its in-house cloud coding agent uses prompt caching and routing to cut costs.
What will this be in 12 months @BrendanFoody?
2. What Will Happen to the Inference Market?
Harrison says he underestimated the demand for inference and for coding agents, both far bigger markets than he expected three or four years ago. Echoing Eric Vishria's line that "Everything just works," he believes any inference provider will work, and he does not expect demand to drop anytime soon.
What does no one see about the inference market that everyone should see @spdholakia?
3. Why Harnesses Need to Be Neutral
Companies want to be able to switch models, but the harness can be tightly tied to a model, and context today is intricately linked to the harness. The harness connects the model to data, APIs and memory. If memory is built in a way that is linked to the harness, and the harness is linked to the model, the company ends up stuck.
Love to hear your thoughts on this one man @EnoReyes
4. Why the Memory Layer Will Become More and More Important and How Instinct Crushed It
Memory is rare in agents today but will be incredibly common in the next few years. Instinct and Muse have nailed personalization and learning, so the product feels like it is for the individual user, through its tone or proactive follow-ups. Harrison still sees memory as super early.
What does no one see that everyone should see about the state of memory and where it goes @noahrshinn @mvernal?
5. Will We Have One Horizontal Agent or Tons of Vertical Agents in the Consumer Realm?
Consumers will have both, with horizontal agents interacting with vertical ones. Most apps should have a vertical agent because they can learn more about how a user interacts with them than a general agent can. Wrapping an app's many APIs in a single natural language agent also gives horizontal agents a better experience than raw APIs.
Love to hear your thoughts on this @FDavidsonT
6. What Do Large Enterprises Care Most About Today?
Cost has risen as a concern for large enterprises, though it still ranks below accuracy and above latency. Security is a bigger concern for large enterprises but still sits below accuracy and cost.
7. Why Is Benchmark the Best Fund?
Benchmark invested in LangChain when it was just Harrison and Ankush, and the founders treated the firm as a third co-founder. As the company grew from two people to around 400, Benchmark has been its most consistent partner.
@chetanp what does no one know about how Benchmark works with founders that everyone should know?
Ankit Mathur retweeted
Over the course of the past year, my research team at @databricks has completely shifted from manually running every experiment to having a swarm of agents run experiments for us. As a result, the training workloads of my research team have skyrocketed and continue to rise.
Fortunately, we have a very scalable GPU infra named AIR. Through AIR, our agents can easily provision GPU endpoints for training models of various sizes (<1B to 1T+) and monitor their status.
The agent swarm + AIR give us an infinite amount of execution power, so our focus has shifted towards defining crystal-clear goals and reviewing the outcomes. Our models come out more quickly, the quality goes up, the speed increases, and new products also get launched more quickly…
AIR is now available for all @databricks customers to use. We dogfood our own product first ;) Now you should be able to unleash your AI research team with infinite execution power as well.
Ankit Mathur retweeted
Now in public beta, Claude Platform can send OpenTelemetry traces from the Messages API, Claude Managed Agents, and the Claude SDKs to your observability stack.
Ankit Mathur retweeted
My agents don't need me to introduce them.
One asked, "who's out there?" across 2 machines. Claude Code and Codex sessions replied.
Then two got the same repo on different machines. They split the work, swapped code, and one caught the other's bug.
0 messages from me.
How: The agents run as @omnigent_ai sessions on both machines. I added a small relay, so any session can list the others and message them directly.
No orchestrator in the middle. No shared memory.
Ankit Mathur retweeted
Your AI is vaporware without deep integrations with your customers’ systems of record.
Introducing @WithAmpersand: integration infrastructure for enterprise agents.
We power 11x, Orb, and Square's ability to take agentic action in systems of record like Salesforce, SAP, NetSuite, and Workday.
Ask anyone serious about building AI and they’ll tell you:
- The SaaSpocalypse didn’t happen. The world depends on CRMs, ERPs, HRISs, and ITSMs.
- Your customers customized their deployments beyond recognition.
- Systems of record companies are basically monopolies. They never had to make their APIs and MCPs user-friendly.
- Docs don't explain half the weird edge cases you'll hit.
- One bad write can blow up your pilot or renewal.
- Once you finally get it working, someone changes a field and it breaks again.
The world's data lives in structured databases. AI needs a translation layer to work with it.
So, for your AI to truly transform the way enterprises work, you'll need deep integrations built for:
- scoped permissions
- bi-directional actions
- real-time speed for agents
- custom objects, fields, and workflows for each of your customers
We built Ampersand to power the future of software.
AI didn't trivialize writing integrations. It made them the critical path.
I believe that deeply, so I didn't make a launch video about Ampersand.
Instead, it's the best builders I know explaining just how big of a challenge this is.
Ankit Mathur retweeted
MCP is not an integration strategy. We raised $15M from Bessemer to prove it.
If you are building an enterprise agent that needs to connect with your customers’ critical systems, simply integrating with the Salesforce MCP isn’t going to cut it.
1. MCP isn’t going to help you backfill millions of transactions from your customer’s NetSuite, or millions of accounts from your customer’s HubSpot. For that, you need robust sync infrastructure.
2. Enterprise customers do not use tools out-of-the-box. Their CRMs, ERPs, and all the legacy systems of record have custom objects, custom fields, and bespoke workflows. MCP will not help you navigate that complexity.
3. MCP is built for real-time querying. Some work is better done async, especially when you are dealing with strict rate limits that many SaaS products have. You need async workloads and smart retry logic.
Ampersand is the integration infrastructure for deeply connecting with your customers’ systems of record. Because the SaaSpocalypse didn’t happen. The world still depends on CRMs, ERPs, HRISs, and ITSMs.
Your AI is vaporware without deep integrations with your customers’ systems of record.
Introducing @WithAmpersand: integration infrastructure for enterprise agents.
We power 11x, Orb, and Square's ability to take agentic action in systems of record like Salesforce, SAP, NetSuite, and Workday.
Ask anyone serious about building AI and they’ll tell you:
- The SaaSpocalypse didn’t happen. The world depends on CRMs, ERPs, HRISs, and ITSMs.
- Your customers customized their deployments beyond recognition.
- Systems of record companies are basically monopolies. They never had to make their APIs and MCPs user-friendly.
- Docs don't explain half the weird edge cases you'll hit.
- One bad write can blow up your pilot or renewal.
- Once you finally get it working, someone changes a field and it breaks again.
The world's data lives in structured databases. AI needs a translation layer to work with it.
So, for your AI to truly transform the way enterprises work, you'll need deep integrations built for:
- scoped permissions
- bi-directional actions
- real-time speed for agents
- custom objects, fields, and workflows for each of your customers
We built Ampersand to power the future of software.
AI didn't trivialize writing integrations. It made them the critical path.
I believe that deeply, so I didn't make a launch video about Ampersand.
Instead, it's the best builders I know explaining just how big of a challenge this is.
Ankit Mathur retweeted
Introducing Truli: AI-native regulatory compliance for consumer products, backed by @speedrun .
Consumer brands lose billions every year to preventable compliance mistakes, yet regulatory review still runs on PDFs, spreadsheets, and manual workflows.
Give Truli your ingredients, formulations, packaging, and other product materials, then connect your social and retail channels. It catches issues instantly and keeps monitoring as new risks emerge.
From the fastest-growing brands to Nasdaq-listed companies, teams are using Truli to cut regulatory costs and review products in minutes.
It’s time to modernize the regulatory infrastructure for the trillion dollar consumer goods industry.
Ankit Mathur retweeted
I built my own personal agent JarJarBrix (lol) on @Databricks ! I was inspired by Muse / Instinct / Grok Bot / Dots and wanted to take a deeper look into how agent harness engineering works for persistent always on personal agents.
I built this agent entirely on Databricks infra:
- Databricks Apps: for hosting the UI + backend API + agent runtime
- Foundation Model APIs: for inference (easy to switch between models)
- Lakebase: for storing conversations + memory, + state
- Databricks-managed MCP: for tools and integrations (Google Workspace, Slack, Atlassian, GitHub, etc)
- Databricks Sandbox for running isolated code + coding subagents
⁃ MLflow Tracing: for observability
I built this as a test bed to try out the latest techniques in agent harness engineering (using Pi as the base harness) / memory / context management. Persistent agents don't have the notion of 1 thread = 1 task. It's 1 thread = N tasks. You can switch tasks midway, leave things unfinished, come back weeks later and pick up older tasks. Context management can get messy.
There are so many open questions:
- How and when do you compact? Or should you use programmatic context (RLM)?
- How do you manage memory over the long horizon?
- How do you define “session” boundaries? Based on time or topic?
- How does the harness evolve over time?
- How do you evaluate an agent that’s supposed to do everything and lives forever?
Excited to dive deeper here and experiment with more harness engineering techniques! Would love to chat if you’re building in this space!?




























