๐ New post: Post-Training in Code: Fine-Tune Your Own Assistant with SFT and DPO
If you're about to fine-tune an LLM on your own data for the first time, my new post is the one I wish I'd had.
It's a single notebook that takes a base model all the way to an aligned assistant, and it's built so you can swap your own data in:
โ Stage 1 (SFT): instruction-to-response pairs, the chat template, and the prompt masking TRL handles for you with `assistant_only_loss=True`
โ Stage 1b (LoRA): the same run with adapters, plus the loss curves that explain why it came out ahead
โ Stage 2 (DPO): preference pairs, the implicit reward, and why the loss has to start at exactly 0.693
Post Link: buff.ly/ZHHWhfL
1
2
643
GPT-6 Astra shipping to a cybersecurity cohort first, not the public, tells you where this is heading. A few predictions for MLEs, on the record:
*๏ธโฃ Capability-gated access becomes standard. The frontier won't be "released" so much as granted.
*๏ธโฃ Post-training is the new pretraining, career-wise. The differentiating skill in 2027 won't be knowing transformer internals โ everyone will. It'll be reward design: building verifiers, shaping RLVR/GRPO signals, catching reward hacking before it ships. If you can't yet explain why DPO sometimes underperforms RLHF, that's your next study weekend.
*๏ธโฃ Long context kills half of RAG. At ~1M-token windows, "just stuff it in the prompt" beats a badly-tuned retriever for a lot of tasks.
*๏ธโฃ Evals eat the roadmap. When capability is gated and expensive, you can't A/B your way to confidence. Teams that invested in real, task-specific eval harnesses will move faster than teams sitting on a bigger model.
*๏ธโฃ The "prompt engineer" title quietly retires. Agentic, high-output models mean the value moves to system design โ tools, verifiers, guardrails, orchestration.
Which of these do you think ages worst? Tell me why ๐
Here's a fact that surprises almost everyone new to LLMs: the giant model that comes out of pretraining isn't a chatbot. Ask a raw base model "What is the capital of France?" and it might reply with "What is the capital of Germany?" โ because on the open web, that's a very plausible continuation. The knowledge is in there. The behavior of being a helpful assistant is not.
Part 5 of my GenAI Fundamentals series is about that gap โ how a base model becomes ChatGPT, Claude, or DeepSeek-R1:
โ Instruction tuning (SFT) โ teaching form: answer instead of extend, follow the instruction, stop when you're done
โ RLHF vs DPO โ the reward-model route vs. the shortcut that quietly took over most production pipelines, and why
โ KTO โ the unpaired cousin of DPO, for when you don't have clean preference pairs
โ RLVR + GRPO โ the reasoning layer behind DeepSeek-R1-style models, and how "verifiable rewards" actually work when you can check the answer
The mental model I use throughout: a base model is a brilliant, impossibly well-read intern who's read every book in the library but has never been taught how to answer a question. Pretraining filled their head. Post-training teaches them manners.
๐ buff.ly/BnalRJ6
I've spent the last two-plus months writing the GenAI Fundamentals series โ my attempt at building the "how do LLMs actually work" mental model from scratch, no hand-waving. Just added the fifth part.
Five parts, in order:
1. Understanding Transformers, the MLE Way โ what a transformer actually is before you write a line of code. ๐ buff.ly/dFUh4AH
2. The Transformer, Demystified โ Let's Actually Build One โ the from-scratch implementation: attention, positional encoding, the easy-to-get-subtly-wrong parts. ๐ buff.ly/G71JxyX
3. What is an LLM? Tokens, Embeddings, and the Big Picture โ raw text to next-token prediction, tokenization quirks included. ๐ buff.ly/M2Tj2aN
4. Pretraining 101: Data, Scale, and the Loss Function โ how random weights become a base model: cross-entropy, data pipelines, compute budgets. ๐ buff.ly/VQcz2fM
5. Post-Training 101: From Base Model to Assistant โ how that base model becomes ChatGPT/Claude/DeepSeek-R1: SFT, RLHF vs DPO, KTO, and the RLVR/GRPO reasoning era. ๐ buff.ly/4DrUbv9
The arc: build the intuition, build the thing, understand what it predicts, understand how it's trained, then understand how it becomes an assistant. Start at Part 1 if you've ever nodded along about "the transformer architecture" without being sure why attention works.
Recommenders are quietly going generative, and most RecSys content hasn't caught up. So I built the path from "why" to "here's the working code."
HSTU: How Meta Built a Trillion-Parameter Recommender That Actually Scales โ the architecture, the math, why teams are dropping two-tower stacks for it.
โ buff.ly/oZ5ltyH
From Random IDs to Semantic IDs: Building a Generative Recommender from Scratch โ RQ-VAE and TIGER on Steam Games with Qwen embeddings, codebook details included.
โ buff.ly/H0yclpx
HSTU From Scratch in PyTorch โ full build: data pipeline, three sub-layers, retrieval + rating loss, benchmarked against rectools on MovieLens-1M.
โ buff.ly/LOsLN3p
The arc: why generative retrieval wins โ items into semantic IDs โ train and validate the sequence model.
3
211
Most "understand LLMs" content stops at the diagram with the arrows. The goal of this series was the opposite: build real intuition, and in some cases build the thing itself.
GenAI Fundamentals, from the architecture up through pretraining โ each part standing on the one before it:
1. Understanding Transformers, the MLE Way โ attention, the residual stream, what's actually happening โ buff.ly/Xu90tEz
2. The Transformer, Demystified โ Let's Actually Build One โ coded from scratch, no hand-waving โ buff.ly/OH1u8tf
3. What is an LLM? Tokens, Embeddings, and the Big Picture โ text โ tokens โ vectors โ text, the full loop, with real configs and PyTorch โ buff.ly/qyFTCL1
4. Pretraining 101 โ Data, Scale, and the Loss Function โ where a pile of random weights becomes a base model โ buff.ly/tKBnGpv
Read top to bottom and you go from "I've heard of attention" to "I just coded attention." More coming on post-training next.
1
131
Today, Meituan open-sourced LongCat-2.0: a 1.6-trillion-parameter model that was both pretrained AND served entirely on domestic Chinese chips. No NVIDIA in the loop.
Beyond the geopolitics:
โ It's the first trillion-parameter-class model claimed to complete FULL pretraining on home-grown silicon. DeepSeek-V4-pro used domestic chips only for inference โ training still leaned on imported hardware.
โ The cluster: ~50,000 AI ASIC cards in "superpod" configuration. ASICs (not general-purpose GPUs) mean the entire software stack โ kernels, collective ops, the training framework โ had to be rebuilt around that hardware.
โ 1.6T params, a 1M-token context, open weights, reportedly comparable to Gemini 3.1 Pro.
The headline is "China trained a frontier model without US chips."
And the MLE story underneath is compute budgets, interconnect, and pretraining pipelines โ exactly the stuff I broke down in Pretraining 101 (Sorry, for the Plug).
๐ buff.ly/tKBnGpv
165
Quick test: can you explain the difference between pretraining, post-training, and fine-tuning?
Most people who use LLMs every day can't โ and that's because the terms get blurred together constantly.
So I started at the root. My newest piece is all about pretraining: the stage where a pile of random weights becomes a base model.
๐ buff.ly/tKBnGpv
It's Part 4 of my GenAI Fundamentals series. If the rest is fuzzy too, start at the beginning:
โ What transformers actually are: buff.ly/Xu90tEz
โ Building one from scratch: buff.ly/OH1u8tf
โ Tokens, embeddings, and the full LLM picture: buff.ly/qyFTCL1
Full series ๐ buff.ly/PTTyrDO
109
Pre-training. Post-training. Fine-tuning. RLHF.
Most people throw these terms around interchangeably โ and quietly aren't sure where one ends and the next begins.
My newest piece clears up the first and most misunderstood one: pretraining.
Pretraining 101 โ Data, Scale, and the Loss Function covers next-token cross-entropy, trillion-token data pipelines, and the scaling that makes it all work. From first principles.
Read it here ๐ buff.ly/tKBnGpv
It's Part 4 of my GenAI Fundamentals series. The earlier parts build the foundation:
โ Understanding Transformers, the MLE Way: buff.ly/Xu90tEz
โ The Transformer, Demystified โ Let's Actually Build One: buff.ly/OH1u8tf
โ What is an LLM? Tokens, Embeddings, and the Big Picture: buff.ly/qyFTCL1
Full series ๐ buff.ly/PTTyrDO
1
2
120
A recommendation system that optimizes itself with LLM agents โ no engineer hand-tuning the loss. That's not a thought experiment; it's a real paper this week - Google DeepMind's Self-Evolving Recommendation System.
So here's the question every RecSys engineer should be sitting with, and what I actually think:
*๏ธโฃ The tuning work shrinks. Hyperparameter sweeps, feature iteration, reranking heuristics โ a lot of that is exactly what an agent loop is good at automating. That part of the job gets smaller.
*๏ธโฃ The judgment work grows. Someone has to define the reward, catch the agent optimizing for the wrong metric, and notice when "engagement went up" actually means the feed got worse. That's not automatable yet.
*๏ธโฃ Understanding the system end-to-end becomes the moat. When the optimizer is a black-box agent, the engineer who knows why the funnel is built the way it is can debug it. The one who only ran the notebook can't.
*๏ธโฃ "I let the agent tune it" won't be an answer in a postmortem. Accountability still lands on a human.
Same pattern I keep coming back to: the model raises the floor, fundamentals raise your ceiling.
Full take on this for ML careers ๐ buff.ly/OcuLTbu
1
196
Just published: Pretraining 101 โ Data, Scale, and the Loss Function.
This is the part where a pile of random weights becomes a base model. Next-token cross-entropy, trillion-token data pipelines, and the scaling that ties it all together โ explained from first principles.
Read it here ๐ buff.ly/tJrFMWg
It's Part 4 of my GenAI Fundamentals series. If you're just finding it now, here's the rest of the build-up:
1. Understanding Transformers, the MLE Way โ buff.ly/GitWOjA
2. The Transformer, Demystified โ Let's Actually Build One โ buff.ly/qEqgs3L
3. What is an LLM? Tokens, Embeddings, and the Big Picture โ buff.ly/TFqwPU3
Each part builds on the last. More coming.
Full series ๐ buff.ly/C5SkvBD
"You get ChatGPT out of pretraining." Nope.
What pretraining actually produces is a *base model* โ a brilliant, impossibly well-read text-completer that has never been taught how to hold a conversation. Ask it "What is the capital of France?" and a base model might reply with *more questions*, because that's how quizzes look on the web. It's not broken. It's doing its job perfectly.
I just published a deep dive on what pretraining really is โ:
โ The one self-supervised loss function behind all of it
โ The data pipeline that matters more than the model code
โ The scaling-law math (Chinchilla, and the inference-economics twist)
โ Why "base" vs "instruct" checkpoints exist
โ A laptop-sized notebook that pretrains a GPT from scratch
Post Link: buff.ly/ghWlXVe
105
New post: HSTU From Scratch in PyTorch - A Complete Walkthrough
This is Part 9d of my RecSys for MLEs series, and it closes the Generative Recommenders arc.
If you're an ML engineer building or evaluating recommendation systems, this series is the resource I'd hand you on day one:
โ Foundations: matchmaking, content discovery, why recommendation is hard
โ Architecture: the 3-stage retrieval-ranking funnel inside YouTube, Netflix, TikTok
โ Engineering: building, training, and shipping rankers at scale
โ Edge cases: cold start, re-ranking, diversity, freshness
โ Frontier: sequential models, generative recommenders, HSTU at 1.5T parameters
The series so far, in order:
1. RecSys Fundamentals: The Art and Science of Digital Matchmaking: buff.ly/7KUXcwz
2. How Recommendation Systems Learned to Think: buff.ly/khShyrc
3. The 3-Stage Funnel Behind Every Modern Recommender System: buff.ly/FogrHtR
4. How YouTube Finds Your Next Video in Milliseconds: buff.ly/Yx2mkFj
5. From Candidates to Clicks: The Engineering Anatomy of Ranking: buff.ly/LU1MpgC
6. 3 Modern Approaches to Solving Cold Start in RecSys: buff.ly/AmS8T9k
7. Your Ranking Model Is Right. Your Recommendations Are Wrong (re-ranking): buff.ly/8B8hxz5
8. From RNNs to Transformers: Building Sequential Recommenders (Part 9a): buff.ly/qrzRhuJ
9. Semantic IDs + RQ-VAE + TIGER (Part 9b): buff.ly/W5gsGPS
10.Why Meta Replaced the Transformer in Their Recommendation System (Part 9c, HSTU): buff.ly/gZOV0qs
11. HSTU From Scratch in PyTorch - A complete Walkthrough: buff.ly/MQFB4cu
Start anywhere. Each post stands alone well.
4
14
704
In late 2022, recommendation teams everywhere hit the same wall.
LLMs had clean scaling laws: double the compute, get a predictably better model. DLRMs didn't. You could throw 10x more parameters and 10x more compute at a SASRec-style model and watch the metrics flatline.
Something was structurally broken between language and recommendation data. Nobody could quite articulate what.
Meta's answer was HSTU, and the answer turned out to be three things at once: softmax was destroying engagement intensity, positional encodings were ignoring time, and the inference cost of generative scoring was untenable without an architectural co-design.
My new post is a deep dive into what they changed and why each change was important.
Inside:
โ The three sub-layers of the HSTU block
โ Relative attention bias and the time-bucket trick
โ M-FALCON and how it works
Post Link:
203
Starting a series on how GenAI actually works โ transformers, pre-training, post-training, and everything in between.
Two posts are already up:
1/ Understanding Transformers, the MLE Way: buff.ly/QZF0jxk
2/ The Transformer, Demystified โ Let's Actually Build One: buff.ly/f2kD7W9
3/ What is an LLM?: buff.ly/Zm5udFW
More soon. ๐งต
46
An open-weights model just beat GPT-5.5 on long-horizon coding. At one-sixth the cost. Under an MIT license.
GLM-5.2 from Z.ai dropped yesterday: a 753B-parameter MoE with a 1M-token context window.
The numbers that matter:
- 62.1 on SWE-bench Pro โ past GPT-5.5's 58.6
- Near-ties Claude Opus 4.8 on agentic suites like FrontierSWE
- ~$5.80 per million tokens โ roughly 1/6 of closed-frontier pricing
Here's why I'm not shrugging this off as another model drop.
Long-horizon coding is the workload where closed models held their clearest lead. Multi-file, multi-step, dozens of tool calls without losing the plot. That's exactly where open weights were supposed to fall apart.
They didn't.
And after last week โ a US frontier model pulled offline โ "the best coding model is open, MIT-licensed, and runs on hardware I control" stopped being a perk. It is essential.
One caveat: these are vendor benchmarks until someone neutral reruns them.
But do run GLM-5.2 against your own eval set this week if you can. The math might genuinely flip.
Are you ready to self-host your coding agent yet?
1
129
When I started working with LLMs in production, I could name the parts (attention, embeddings, tokens) but I couldn't connect them.
I couldn't answer basic questions. Why not process one character at a time? What happens between my prompt and the response? Why does the model sometimes just stop mid-sentence?
Once I saw the pipeline end to end, it all clicked. KV caching made sense. Fine-tuning made sense. Even the gotcha screenshots stopped being surprising.
My new post is the explanation I wish I'd had:
โ Text to tokens to vectors to text, the full loop
โ Real model configs and runnable PyTorch
โ The practitioner payoff: why tokenization causes so much weird behavior
Part 3 of my genAI Fundamentals series. The next one is Pretraining 101.
Link: buff.ly/Zm5udFW
1
35
If you build with or evaluate LLMs, my new post is for you.
Tokenization sounds like a boring preprocessing detail. In practice it decides what a model can do, how much it costs to run, and why it fails on tasks you'd expect it to handle.
The post covers, with code and real numbers:
โ Why vocabulary size is a real design decision (32K โ 128K โ 256K)
โ How much of a model is just the embedding table (31% of GPT-2, ~7% of a 7B)
โ Weight tying: which models share the embedding and LM head, and which don't
โ Why multilingual cost varies so much per language
โ Why you can't swap a tokenizer without retraining from scratch
There's a companion Kaggle notebook so you can check every number yourself.
Part 3 of my genAI Fundamentals series.
Link: buff.ly/Zm5udFW
58
๐ New post: What is an LLM? Tokens, Embeddings, and the Big Picture
Part 3 of my genAI Fundamentals series is live.
Most "what is an LLM" explainers stop at "it predicts the next word." This one traces the whole pipeline, with real model numbers and PyTorch you can run.
What you'll get:
โ Why LLMs read tokens, not letters (and why that breaks the strawberry test)
โ The embedding table: a literal vocab_size ร d_model lookup, not magic
โ Weight tying, with a small worked example using actual numbers
โ The end-to-end path: tokenize โ embed โ transform โ LM head โ softmax โ sample
โ Why Hindi costs 3x the tokens of English for the same sentence
If you've ever wanted the one mental model that makes KV caching, fine-tuning, and context limits click, start here.
Link: buff.ly/Zm5udFW
132
I'm kicking off a series on GenAI, and over the coming weeks I'll be digging into the topics that actually make these models work โ from the architecture up through pre-training and post-training.
The goal is to go beyond the hand-wavy explanations and build real intuition (and in some cases, build the thing itself).
Two posts are already live to get us started:
1. Understanding Transformers, the MLE Way โ buff.ly/9NtwbEi
2. The Transformer, Demystified โ Let's Actually Build One โ buff.ly/mcyWgQC
More to come on what happens during training and why it matters. If these topics are on your radar, follow along โ and I'd love to hear which areas you'd most want me to cover.
43




