@SchmidtFutures AI in Science fellow with @OPIGlets, lecturer in biochemistry @SPC_Oxford. I tweet about using AI in biotechnology and healthcare.
Oxford, United Kingdom
Joined June 2018
- Tweets397
- Following772
- Followers1.2K
- Likes572
Pinned Tweet
Everyone is talking about @claudeai designing miniproteins.
The question they are missing: can miniproteins (*any* miniproteins) ever make good drugs?
I wrote a long-form essay looking at 30 years of miniproteins in the clinic, and I have an answer for you ⬇️
This is the experiment I have wanted someone to do for half a decade! Amazing work from @KotaroTsuboyama and team
We can now design all kinds of fancy de novo proteins. But can we produce them at scale, at a reasonable cost, and without extensive optimization? That’s what got this project started. Our preprint is now out! Much of the work was led by Selena @sesameandrea and Kaori. 1/8
Carlos Outeiral retweeted
I’m as AI pilled as anyone, but one of the most tedious parts of AI x Bio right now is almost a complete lack of and almost an active avoidance of scholarship.
Carlos Outeiral retweeted
I need all the AI people to stop saying "biology" when they mean "medicine" or "drug development"
Carlos Outeiral retweeted
Let's keep it simple:
- list every molecule in a cell
- list every molecule in blood
Once you have those, please explain step-by-step how having enough examples (like AlphaFold) will answer dynamic questions (injecting X causes Y)
If you had an exponential amount of compute you could solve simulate biology exactly and wouldn't need to run any experiments.
Some of you will think "of course, but that's an unrealistic assumption" and others will think "no, you need experiments even with that much compute. That's just How It's Done".
You would of course need evaluations once you have a solution, but you don't in principle need experiments to find it: Experiments are a way to save compute, dating from a time when it was much more expensive.
And I think the presentation is exponential but real systems aren't irreducibly exponential. AlphaFold worked, after all.
Carlos Outeiral retweeted
This person has blocked me (I have no clue why and for how long)
but since it's being widely discussed I thought I would share my thoughts, having worked in the field of biomedical AI for 7+ years.
"Solving" biology is not limited by compute but BY DATA.
That doesn't mean you can't do impactful bio work with more compute and existing datasets. I actually think applying AI to existing datasets we can make many advances in biology and medicine. (much of what we do by @SophontAI is along these lines)
But to "fully solve" biology, you need to capture the full complexity of biological systems and we're not even close to fully observing it.
With near-infinite compute you could probably solve it. Even then, you need a starting condition, and how would you define it if you aren't able to comprehensively characterize the system in the first place.
What about approximating biology? The challenge here is it's hard to know what parts matter and what can be approximated away.
Consider a disease-associated protein that transiently adopts a rare conformation 0.1% of the time, which enables an interaction that contributes to disease.
A model could capture the dominant conformation very accurately and still miss the interaction that matters to disease biology.
More compute could help us sample that rare state. But we also need to know the relevant cellular conditions, binding partners, concentrations, etc. Otherwise we could spend enormous compute precisely simulating a system that doesn't match the biology we're trying to understand.
As a sidenote, AlphaFold is a bad example here. AlphaFold only succeeded because of the existence of the PDB dataset, which was collected over decades. EXPERIMENTS WERE NEEDED TO MAKE ALPHAFOLD NOT JUST MORE COMPUTE.
As you all know, I'm very optimistic about AI accelerating biology. But the path forward is a feedback loop: better experiments → better data → better models → better experiments.
Compute can help us get much more out of what we've measured and decide what to measure next. It doesn't automatically tell us which unmeasured details matter.
That's not how it works.
There are roughly two types of open problems in comp bio.
In some, exponentially growing compute doesn't get you far enough. In most others, compute availability doesn't matter at all.
That's why you need experiments.
If you had an exponential amount of compute you could solve simulate biology exactly and wouldn't need to run any experiments.
Some of you will think "of course, but that's an unrealistic assumption" and others will think "no, you need experiments even with that much compute. That's just How It's Done".
You would of course need evaluations once you have a solution, but you don't in principle need experiments to find it: Experiments are a way to save compute, dating from a time when it was much more expensive.
And I think the presentation is exponential but real systems aren't irreducibly exponential. AlphaFold worked, after all.
There is fundamentally so little that we know about *single cells*, not to mention how they operate within complex organisms.
Throw *infinite* compute at it and you will still get nowhere.
1
52
Carlos Outeiral retweeted
If you don't think that there is a bubble in AI x Bio valuations, just consider that the purported valuation of Isomorphic Labs is more than that of Revolution Medicines. Investors are acting as if preclinical Isomorphic already has an approved KRAS scale blockbuster.
Carlos Outeiral retweeted
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI"
That statement is still totally true.
First, the reasoning abilities of current AI systems are based non-auto-regressive search (which is what I have always advocated for). But AFAICT, they do it in token space, which is limited and inefficient. I have claimed that human-like reasoning must be a search in continuous representation space. It looks like the industry is moving towards that.
Second, the self-improvement methods, as currently practiced, only work for domains where the quality of outputs can be scored without human intervention, such as mathematics, code, and scenarios that can be simulated accurately. Not anything else. Humans and animals learn new skills way more efficiently than current RL methods.
Third, the multimodal capabilities of current AI assistants generally use separately-trained encoders (that are not LLMs). This is also what I've been advocating. Except that I think the best way to do this is with JEPA trained with self-supervised learning. The research community is clearly moving towards that (3000 papers on JEPA in just 4 years).
Fourth, if LLMs were a path to human-level AI, we would have domestic robots and Level-4 or Level-5 self-driving cars for consumers by now. And we don't. We certainly don't have cars that can learn to drive in 20 hours or practice like any teenager. We're still missing something pretty huge to claim human-level intelligence (let alone superhuman).
Sure, we now have computer systems that are impressive, very useful, and whose performance is superhuman in an increasing number of domains (coding being one of them).
But that's true of the entire history of progress in computer technology.
Lastly, there is a basic confusion about what intelligence actually is.
It is not the mere accumulation and regurgitation of existing declarative knowledge (which is essentially what LLMs do).
As Jean Piaget famously said, "intelligence is not what you know, it is what you do when you don't know."
It is your ability to solve new problem without any prior training, to act in previously-unknown scenarios, and to adapt very quickly to new situations with minimal training.
We're still far from that.
I hear rumours Anthropic solved one of these already @OpenAI
In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others.
Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
1
498
Carlos Outeiral retweeted
>be me
>discover effective altruism
>apparently normal charity is inefficient
>why donate to random sad thing when spreadsheet can tell you optimal sad thing
>fair enough
>buy mosquito nets
>save lives
>numbers look good
>feel powerful
>couple years later
>someone asks an innocent question
>why only count people alive today
>huh
>future people matter too
>obviously
>my grandchildren shouldn't matter less just because they haven't spawned yet
>reasonable.jpg
>keep following logic
>what about their grandchildren
>also yes
>what about people in 500 years
>sure
>5000 years
>why not
>500 million years
>starting to get weird but morality is morality
>open calculator
>humanity could survive for an astronomically long time
>could colonize galaxy
>could have trillions upon trillions of descendants
>maybe digital people too
>maybe simulated civilizations
>maybe dyson spheres full of happy uploaded minds
>calculator starts smoking
>realize currently living humans are rounding error
>8 billion people suddenly looking extremely beta
>future contains potentially 10^something people
>can't even fit beneficiaries in google sheets
>new moral priority unlocked
>protect the long-term future
>stop thinking in units of "people helped"
>start thinking in "fraction of cosmic endowment preserved"
>malaria?
>terrible
>but only kills existing humans
>AI extinction could delete the entire light cone
>nuclear war could permanently derail civilization
>bad institutions could lock in terrible values for ten million years
>someone invents wrong constitution in 2140
>quadrillions suffer
>better fund governance workshop now
>friend says maybe we should improve hospitals
>explain opportunity cost
>friend says hospitals are full of actual sick people
>explain scope sensitivity
>friend stops inviting me to dinner
>need to decide what to fund
>easy
>expected value
>suppose project has one in a million chance of preventing extinction
>sounds tiny
>but extinction destroys 10^50 future lives
>multiply
>mother of god
>$10 million project has expected value of several galaxies
>charity evaluation complete
>someone asks where the one-in-a-million number came from
>expert judgement
>which expert
>us
>how calibrated
>extremely thoughtfully
>reduce estimate to one in ten million to be conservative
>still beats curing cancer by 38 orders of magnitude
>epistemic robustness achieved
>someone says maybe project doesn't work
>assign 20% chance
>still astronomical
>maybe project makes problem worse
>assign 5% chance
>still astronomical
>why 5
>because 30 felt pessimistic
>publish 46-page report
>contains seventeen sensitivity analyses
>every sensitivity analysis begins after assuming intervention has positive sign
>critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities
>yes
>that's literally why it's important
>critic says the uncertainty might be structural rather than numerical
>make probability smaller
>critic says no, I mean maybe your model is wrong
>make probability smaller again
>critic begins rubbing temples
>discover AI safety
>perfect longtermist cause
>AI might kill everyone
>or create utopia
>or seize galaxy
>or tile universe with paperclips
>or create billions of conscious software minds
>finally a problem with numbers big enough for me
>start AI safety nonprofit
>mission: prevent dangerous AI
>hire smartest people available
>smartest people immediately start building better AI to understand dangerous AI
>interesting
>we must understand capabilities to understand safety
>we must scale models to study alignment
>we must race ahead so less responsible actors don't get there first
>we must deploy systems to learn how deployment can go wrong
>we must build the thing quickly because building the thing quickly is dangerous
>outsider asks why the people most worried about AI apocalypse all work at AI companies
>complicated field
>company releases stronger model
>very concerned
>company begins training even stronger model
>extremely concerned
>company raises $14 billion
>concern reaches unprecedented levels
>need to influence government
>future is at stake
>normal democratic process too slow
>politicians don't understand exponential curves
>public doesn't understand x-risk
>experts must guide them
>who counts as expert
>people who understand x-risk
>who understands x-risk
>our friends
>someone objects that this seems politically convenient
>explain we're representing future generations
>future generations unavailable for comment
>develop concept of value lock-in
>terrifying possibility that one ideology controls civilization forever
>therefore extremely important that civilization adopts correct values before lock-in
>whose values
>let's circle back
>begin with impartial morality
>end with small group of people deciding what quadrillions of hypothetical beings would want
>beautiful arc
>meanwhile actual humans keep doing annoying things
>voting wrong
>having parochial attachments
>loving family more than strangers
>caring about local community
>getting upset when told their suffering is cosmically negligible
>evolutionary biases everywhere
>explain that moral intuition cannot be trusted
>except intuition that future digital people count
>and intuition that extinction is uniquely bad
>and intuition that our probability estimates are sane
>and intuition that our institutional choices improve the future
>those intuitions survived peer review
>someone donates $5k to local homeless shelter
>inefficient
>could have funded 0.0000000000003% of an AI governance researcher
>think of all the simulated people you just killed
>okay maybe don't phrase it that way publicly
>PR team says "future generations deserve a voice"
>much better
>journalist asks what longtermism means
>say "future people matter"
>everyone agrees
>great
>journalist asks what follows from that
>well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures
>journalist raises eyebrow
>return to "future people matter"
>motte has entered the chat
>critic: of course future people matter
>me: glad we agree
>critic: I don't agree that your institute knows how to help them
>me: why do you hate our grandchildren
>eventually notice uncomfortable implication
>if future value dominates everything
>then helping people today mostly matters through effects on future
>education matters because future institutions
>health matters because future productivity
>democracy matters because future trajectory
>human beings slowly become instrumental variables in their own moral philosophy
>see starving child
>feel compassion
>check spreadsheet
>child's direct welfare contribution negligible
>but perhaps childhood nutrition improves national institutional quality
>compassion restored
>tell myself this is impartial altruism
>one day assistant asks obvious question
>"how do you know your intervention actually improves the far future?"
>silence
>open spreadsheet
>increase column width
>add confidence interval
>assistant asks again
>"no, I mean how do you know the sign is positive?"
>stare into cosmic light cone
>10^50 people staring back
>none of them exist
>none of them can tell me
>none of them can falsify my assumptions
>realize I have invented the perfect constituency
>infinitely important
>completely silent
>and always represented by me
This is super cool!
The pricing is the Answer to the Ultimate Question of Life, the Universe, and Everything.
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
1
2
331