Isak Westerlund

understanding reasoning from pretraining to post-training standard LLM pipelines eval RL post-training largely in isolation from the massive pretraining phase. Instead, the authors quantify pretraining-to-RL interface by using chess as a controlled testbed to isolate how pretraining scale shapes RL performance. Training is performed sequentially across a complete pipeline. At each stage, the model is first pretrained on human chess games, then SFT on synthetic reasoning traces, and finally uses RL on chess puzzles with verifiable rewards They test this across 36 compute combos with models ranging from 5M to 1B parameters. Some findings are: 1) pretraining loss strongly predicts post-RL performance; the rate of RL improvement grows approximately linearly with number of pretraining tokens. 2) Mechanistically, RL does not simply sharpen existing SFT preferences; on hard puzzles, it surfaces correct moves that were nearly absent, a predictive pattern that also transfers to a 1B language model trained on math text!