
💻ML/AI |🧬🧪| 📈 • Biotech industry • Ex @StanfordMed, PhD @UCSD, big tech &c • 🇸🇬🇺🇸🇮🇳 • Caveat: humor
SF Bay Area
Joined August 2016
- Tweets671
- Following631
- Followers763
- Likes9.8K
Pinned Tweet
ML -> LLM AI -> AGI -> ...?
What this is for and where this is going for AI+science
akshay.bio/variable-expectat…
akshay Balsubramani retweeted
What if a [MASK] was all you needed?
ModernBERT is great, but we couldn't stop wondering if it could be greater than previous encoders in different ways.
Maybe we don't need task-specific heads? Maybe it can do all sort of tasks with only its generative head? Spoilers: Yes
akshay Balsubramani retweeted
“Shotgun” is a way better name than bootstrap, it isn’t even close.
akshay Balsubramani retweeted
Thrilled to announce a preprint describing collaborative work between our group and @fraser_lab, @BShoichet, and @adamrenslo. The paper shows a practical application of the Thompson Sampling method we published earlier this year. biorxiv.org/content/10.1101/…
akshay Balsubramani retweeted
A central challenge of AI-driven biotech is that in drug discovery a fully validated medicine is worth a million times more than a million early-stage ideas that only have little validation.
This is unfortunately at odds with the AI domain where we almost always attempt to "solve" a general class of problems - which even in the optimal case provides us at most with a little bit of validation for a large space of ideas, with most questions remaining unanswered and relegated to further downstream steps. Because of the large number of steps involved in drug discovery, solving even a general class of problems within just one of these steps does not massively impact the overall enterprise.
This dichotomy is at the core of the challenges that first-generation data-generating AI-driven biotechs face - platforms produce a million low-validation ideas that then further clog up the validation bottleneck. Day to day, swimming in an ever-expanding lake of early-stage ideas may feel like progress, but the impact, measured in new medicines available for patients, is limited.
Moreover, this approach to solve the general before the specific neglects the time factor - if you set out to solve a general class of problems before embarking on any specific idea, you will be overtaken by those that elect to instead address the valuable but specific problems in that space immediately. By the time your general capability is ready for action, you may find that the best ideas have already been pursued by others that did wait to first solve the entire class of problems.
In short, we perhaps were too busy building platforms that we forgot to make medicines.
What could we do differently to help address this central challenge?
For one, there may be potential to focus more on the capabilities that enable us to produce fully validated or nearly fully validated ideas (depicted as a full glass of water) more quickly, that produce ideas of greater quality when fully validated, or perhaps even capabilities that open up new avenues that could not have been pursued otherwise.
Secondly, it may help to mindfully balance general capability building with actually developing a medicine *today* to ensure we remain on track for realized and not only envisioned impact.
The future of AI-driven drug discovery is bright but needs to be grounded in actual innovative medicines available for patients.
akshay Balsubramani retweeted
I wrote about some recent studies identifying widespread gene-environment interactions (GxE) acting on common traits, but through an unexpected mechanism. A few highlights 🧵:
akshay Balsubramani retweeted
Short, interesting antibody design paper using available tools (EvoDiff, ColabFold, HADDOCK, OpenMM).
No wet-lab validation, but interesting to see the pipelines people are putting together. Appears to be related to silico.bio
biorxiv.org/content/10.1101/…
akshay Balsubramani retweeted
This last chapter "Horoscopes" is only 3 pages but I keep thinking it should be at the start of the book, because it is about scientific and social context of statistics. Anyway here it is:
akshay Balsubramani retweeted
Then we repeated the experiment, but we dehydrated the tissue first. The architecture was fine. We left it at room temperature for a year. Still fine. Ran over it with a car, had a former Astros starting pitcher throw a fastball at it, fired a shotgun at it. Still fine. (15/18)
akshay Balsubramani retweeted
📈 One standard graph in all bio deep learning papers should be: max similarity to anything in training set vs performance. (Reviewers shouldn't have to guess if there might be overfitting issues).
akshay Balsubramani retweeted
Does more compute equate with greater risk?
What is our track record at predicting what risks emerge with scale?
I don't have much time anymore to write, but this felt important enough to write down some thoughts on.
arxiv.org/pdf/2407.05694v1
akshay Balsubramani retweeted
Replying to @Thom_Wolf
I'm surprised the MuMath-Code paper wasn't more discussed on Twitter. It is awesome! Super dense in information.
Only reference I found was: teapaw.org/gm8xx8/status/17903650… who incidentally tweets about a lot of really interesting papers that don't get much attention.
MuMath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoning
↓
arxiv.org/abs/2405.07551
akshay Balsubramani retweeted
I’m really excited to share our new pre-print “Functional protein mining with conformal guarantees”, with @SeyoneC @ml_angelopoulos @PeterHYoon Michael Jordan @doudna_lab
biorxiv.org/content/10.1101/…
🧵
akshay Balsubramani retweeted
🧵Today is an exciting day! We have just announced ESM3, the design of a new green fluorescent protein, and the launch of our company. Here's a short thread about the scaling and infrastructure work involved in making ESM3 happen! 🧵
We have trained ESM3 and we're excited to introduce EvolutionaryScale.
ESM3 is a generative language model for programming biology. In experiments, we found ESM3 can simulate 500M years of evolution to generate new fluorescent proteins.
Read more: evolutionaryscale.ai/blog/es…
akshay Balsubramani retweeted
read the license before using ESM-3; github.com/evolutionaryscale…. you're welcome, pharmas and biotechs.
akshay Balsubramani retweeted
I think it's a relevant metric, it's just not very hard. If someone could take a GFP and make it 10000x brighter than any known ones with 192 shots I'd be far more impressed, even if it was at 3 mutations.
ESM3 excites me most because it is a relatively high success rate model that lowers the bar for design for a non-ML person. I don't think it would beat an expert designer on its own. I'm willing to bet a high ED GFP can be made with a <1M parameter model and a couple of experts. The controllability is the key value add to me. The steam engine didn't go faster than a horse (at first), but it added a whole world of possibilities.
akshay Balsubramani retweeted
What if we could universally recombine, insert, delete, or invert any two pieces of DNA?
In back-to-back @Nature papers, we report the discovery of bridge RNAs and 3 atomic structures of the first natural RNA-guided recombinase - a new mechanism for programmable genome design
akshay Balsubramani retweeted
Taking a look at the ESM3 GFP, there is a similar but less pronounced pattern to the Profluent OpenCRISPR protein. The "nearest match" is 58% (from the paper) but you can get 74% from mixing and matching two near proteins.
gistpreview.github.io/?81c10…





































