My summer project is done! A 20 video, free course on post-training to accompany my book is all on YouTube with slides open for modification & re-use. ~12 hours of content covers the core foundations and some research areas I think will grow in importance. It was a fun time to review all the fundamentals again, as it is clear in the next 1-3 people the amount of people wanting to learn post training will likely 100X again from today, as we have already 100X'ed from two years ago. As AI agents get increasingly capable at coding and discussing these fundamentals (see the code exercises accompanying the book that I am refining with the community) I think developing clear intuitions for how models work and why is one of the most important skills going forward in AI. Still, learning the post-training math is the best way to battle test them. I personally just in this course am starting to master how forward/reverse KL relates to post-training topics. Thanks to all my viewers, and I'm happy to answer questions in the book discord or understand how to better teach the various reward models, on-policy distillation, new RL algorithms, etc. Plus, the book is 50% off right now with the code PBLambert on Manning to celebrate the launch. I'll share the relevant links below. Who's going to make this course for pretraining?

My book, Reinforcement Learning from Human Feedback is done! This is the book I wish I had when learning to fine-tune, align, & now post-train models since ChatGPT. The resource has been built by me finding time to study and document the fundamentals on nights and weekends since 2024. Transferring as much of the intuitions of building Olmo as I possibly can in the book format. The book is launching with an over 10 hour, full course with slidedecks, functional code for the training chapters, an example model completions library, and of course the free online web version. Physical orders from Manning will ship in 1-2 weeks, and Amazon a week or so after. Thanks for your support!