What happens when you train a transformer on 123 million bacterial proteins Bacteria have been fighting viruses for billions of years. To do it, they have evolved a remarkable diversity of antiphage defense systems — molecular immune machines that detect and destroy invading phages. Yet fewer than 250 such systems had been experimentally validated. A new study in Science suggests we've barely scratched the surface. Mordret and coauthors asked a simple but powerful question: can language models trained on protein sequences and genomic context learn the "grammar" of bacterial immunity well enough to predict entirely unknown defense systems at scale? They developed three complementary deep learning models. ALBERTDF adapts the ALBERT transformer architecture to treat genes as words and genome neighborhoods as sentences — learning defensiveness from genomic context alone, without any sequence information. ESMDF fine-tunes the ESM2 protein language model with LoRA adapters to classify proteins as defensive or non-defensive directly from amino acid sequence — trained on a dataset of 123 million proteins drawn from 32,000 bacterial genomes. GeneCLRDF combines both signals through contrastive learning: it teaches the model that a gene's identity can be inferred either from its sequence or from the genomic neighborhood where it lives. This joint embedding achieves 99% precision and 92% recall on held-out benchmarks — far outperforming each approach independently. The models aren't just impressive on paper. The authors experimentally validated 12 antiphage systems with no prior link to immunity, in both E. coli and Streptomyces albus. Some carry canonical defense domains; others involve proteins with no known association to antiphage function whatsoever. Applied to over 32,000 bacterial genomes, GeneCLRDF predicts 2.39 million antiphage proteins. Around 1.5% of a typical bacterial genome is devoted to defense — three times previous estimates — and more than 85% of predicted protein families have no prior link to immunity. The implications are immediate. The predicted atlas — including ~23,000 candidate operon families — is a ready-made discovery pipeline for novel nucleases, molecular effectors, and antimicrobial mechanisms directly relevant to phage therapy and programmable biologics. Language models are turning the bacterial pangenome into an actionable resource. Paper: Mordret et al., Science (2026) — Science license | science.org/doi/10.1126/scie…