Florian Wünnemann

Aggregating Residue-Level Protein Language Model Embeddings with Optimal Transport 1. This study introduces a novel protein representation method using sliced-Wasserstein embedding (SWE) to aggregate residue-level outputs from protein language models (PLMs) into fixed-length, protein-level embeddings. SWE addresses limitations of traditional average pooling by capturing essential sequence information often lost in standard aggregation methods. 2. SWE interprets token-level embeddings as samples from a distribution and maps them onto a reference set using optimal transport, creating embeddings that retain critical information from variable-length protein sequences. This approach significantly outperforms average pooling, especially for longer sequences, by preserving details vital for accurate predictions. 3. Using SWE, small PLMs achieved comparable or superior performance to larger models with average pooling across several tasks, including drug-target interaction (DTI), enzyme commission (EC) prediction, and subcellular localization, making PLM use more accessible for researchers with limited computational resources. 4. This approach enhances interpretability, as the learned reference embeddings represent key features of protein-level tasks, providing insights into protein structure-function relationships and offering a scalable solution for protein-level prediction tasks. @rohitsingh8080 💻Code: github.com/navid-naderi/PLM_… 📜Paper: biorxiv.org/content/10.1101/… #ProteinLanguageModels #MachineLearning #Bioinformatics #OptimalTransport #ProteinRepresentation