Ivan Masmitja

PQN has been accepted as a Spotlight at ICRL 2025 🎉 👉If didn't have time to check it out, take a look at our 5-minute blog covering PQN’s key features, plus a Colab demo with JAX & PyTorch implmentations mttga.github.io/posts/pqn/ 🔎 If you want to dive deeper, we break down TD instability issues and how PQN tackles them without replay buffers or target networks in this other blog post: blog.foersterlab.com/fixing-… See you in Singapore! 🇸🇬

⚪I OPEN CALL  🪼¿Postdocs interesadxs cómo cambia la distribución de la biodiversidad debido al cambio global? 🤖¿Predocs amantes de la robótica submarina? 🚨En el ICM ofrecemos distintas plazas dentro del programa @Momentum_CSIC!  + info aquí ➡️ icm.csic.es/ca/ofertes-de-tr…

🚀 We're very excited to introduce Parallelised Q-Network (PQN), the result of an effort to bring Q-Learning into the world of pure-GPU training based on JAX! What’s the issue? Pure-GPU training can accelerate RL by orders of magnitude. However, Q-Learning heavily relies on replay buffers and target networks, making training computationally slow and memory-intensive on GPUs. As a result, researchers often prefer PPO, leaving Q-Learning behind 🥲 Our solution? Eliminate Q-Learning's legacy components. PQN challenges the standard DQN paradigm by training a Q-Network without replay buffers or target networks: just online Q-Learning with vectorised exploration and network normalization (layer or batch norm). Despite its simplicity, PQN sets a new strong baseline in many single-agent and multi-agent scenarios. Check out the thread for more details 🔥 📄 Paper: arxiv.org/abs/2407.04811 ⚙️ Code: github.com/mttga/purejaxql/ A fantastic collaboration with @mattiefoxcs and @benjamin_ellis3, and the support of the amazing people at @FLAIR_Ox directed by @j_foerst. Inspired by the groundbreaking work of @_chris_lu_ on compiling entire RL pipelines in GPU: github.com/luchris429/pureja…