Nithin Chalapathi

1/ Can molecular AI move past hard-coded Graph Neural Networks and embrace scalable Transformers that discover molecular structure on their own? We demonstrate that you can train a 1B parameter Transformer model without any graph priors or physical inductive biases. And surprisingly, not only can it maintain competitive performance under equal compute on the Open Molecules 2025 dataset… it’s faster than a 6M parameter equivariant GNN, and exhibits scaling laws that don’t saturate. We use this as a starting point to investigate emergent internal representations, and find that it adaptively discovers molecular structure! Check out the interactive demo on our website: tkreiman.github.io/projects/… And our paper: arxiv.org/abs/2510.02259 In collaboration with @tobykreiman, @YutongBAI1002, Fadi, Elizabeth, and @EricQuCal. Here’s a video showing how the Transformer learns distance-aware attention patterns (purple gradient) that adapt to atomic environments 👇