"AlphaFold's Bayesian roots in Probability Kinematics," co-authored with directional statistics pioneer Prof. Kanti V. Mardia on his 90th birthday, is now published by AISTATS 2026. In 2018, CASP13 (the 13th edition of the "olympic games" of protein structure prediction) reported "[...] unprecedented progress in the ability of computational methods to predict protein 3D structure. The reasons are not yet fully clear [...]". DeepMind's AlphaFold, version 1, had changed protein structure prediction forever! AlphaFold1 was published as “Improved protein structure prediction using potentials from deep learning” by Nature in 2020. It uses deep networks to parameterize potentials over dihedral angles and pairwise distances, combining the two using a third reference potential. This approach was heuristically justified by referring to the classic work by the physicist John Kirkwood from 1935 on potentials of mean force for liquids. But there’s more to the story, and it builds on some fascinating work done well after 1935. First, AlphaFold1 formulates a potential that corresponds to a prior distribution over protein dihedral angles (phi, psi) using directional statistics (which deals with angles, directions, orientations and so on). This goes back to the classic work of David L. Dowe at the end of the 90ies, and our joint work with Kanti V. Mardia around 2010 on generative models based on directional Bayesian networks. Second, a non-local potential concerning pairwise distances D between amino acids is added. Why? In principle, the dihedral angles are enough to parameterize the 3D structure, but small errors quickly propagate, rendering them only accurate on a local scale. Third, a reference potential on D is subtracted. Why? Correctly combining distance and angular potentials requires taking into account the signal concerning distances already present in the angular potential. This approach was pioneered as a heuristic by Manfred Sippl in the 1990s, and in the 2010s formalized by us as a surprising application of generalized Bayesian updating called probability kinematics or Jeffrey’s updating (referring to the probability theorist Richard C. Jeffrey, 1926-2002). AlphaFold1 powered the whole probabilistic setup based on directional statistics and probability kinematics outlined above with deep networks, obtained a maximum a posteriori estimate using simple gradient descent, and subsequently aced CASP13. Our article connects a paradigm shift in protein structure prediction with the classic work by Richard C. Jeffrey (1950s, probability kinematics), Manfred Sippl (1990s, knowledge based potentials), and our joint work with Kanti V. Mardi (2010s, probability kinematics applied to directional priors), revealing generalised Bayesian updating as a potentially powerful method to formulate compositional deep models from simpler components. The figure shows the AlphaFold1 potential reformulated as a joint probabilistic model. Article: virtual.aistats.org/virtual/…

Give this man a cigar. LLMs are heuristic induction only - no deduction, no abduction (ie. model building based on a world representation, including causality and memory). They are eminently useful but not conscious, sentient or intelligent, and they will never be.

The world of Bayesian statistics looks upon the terminology drift in the world of machine learning with increasing horror...

New paper on arxiv! Tiny yet effective neural networks from using circulant weight matrices and spectral priors.

AGI crowd, listen and learn.