Jack retweeted
Been learning about LLMs for a while now. Spent some time this week building a tiny GPT from scratch, training it on 448K software arch Q&A examples.
256 token context, 8K vocab, ~2.9M params
Not very accurate yet, but a fun learning
colab.research.google.com/drβ¦
Jack retweeted
With Ai, Agents & Robots being spammed right left & center. The new moat is human. Build cafe, resturants, malls or even tools by human for human! π
A new term is echoing through AI circles these days: Loop Engineering π§Ά
Do you know what it actually means? β¬οΈ
For the last 2 years, the workflow looked like this:
You write a prompt π¨βπ», AI gives an answer π€
You write another prompt π¨βπ», AI gives another answer π€
You were in the loop π¦
Now the idea is different.
You build a system that:
- Finds work π
- Prompts the agent π€
- Checks the results π§ͺ
- Updates memory π
- Decides the next step π
Without waiting for you. You're no longer prompting the AI. You're designing the loop that prompts the AI.
The leverage point has moved. π―
Prompt Engineering taught us how to talk to models. π£οΈ
Loop Engineering is about teaching models how to keep working when we're not talking to them. πΆ
And that's a much bigger shift than most people realize. Have you tried it? π§
Ever wondered how LLMs understand negation if they read text one token at a time? π§Ά
Consider this sentence: β¬οΈ
I don't think the movie was terrible.
As the model reads:
π I don't think...
it starts expecting a negative opinion?
Then it sees:
π terrible
Now things look even more negative.
But the actual meaning isn't strongly negative at all.
In many conversations, "I don't think it was terrible" is closer to:
β
It was okay.
β
It wasn't that bad.
So how does the model get this right? π₯Ά
The trick is that LLMs don't lock in meaning as each token arrives. πΏ
Every new token can change how previous tokens are interpreted. π§Ά
When the model sees words like don't, not, never, or hardly, it uses attention to connect them with other relevant words in the sentence. π€―
In this case, attention helps the model understand that don't modifies the idea of terrible, changing the overall sentiment. π¨πΌβπ³
Meaning doesn't come from individual words.
It comes from how words attend to each other. π‘
And that's why a tiny token can completely flip the meaning of a sentence. π§
Bcz Attention is All You Need. π€―
We've all heard that modern LLMs have billions of parameters. π§Ά
Some of the latest models are rumored to have over a trillion. π€―
Now think about that for a second.
If a model has 1 trillion parameters, and each parameter takes 4 bytes, that's 4 TB just to store the weights. π₯Ά
And every time you ask the model a questionβ, those weights need to be available to the GPUs performing inference. π
So how is it possible for AI companies to serve millions of requests without needing racks of GPUs per conversation? β¬οΈ
The answer is quantization (mapping floating point weights into lower-bit integers like 8-bit or 4-bit using scale + zero-point to approximate values during inference) Just like instead of full value for pi we just limit to 3.14. π‘
Most weights don't actually need 32 bits of precision. You can often represent them using 8 bits, 4 bits, or even fewer, while preserving most of the model's quality. π€
Suddenly that 4 TB model becomes 1 TB.
Or 500 GB.
Or less.
Same model.
Same knowledge.
Far less memory and bandwidth.
The funny thing is that many people think AI inference is limited by compute. πΏ
In reality, a huge part of the challenge is simply moving model weights around fast enough. π’
Quantization works because the model usually doesn't need perfect numbers.
It just needs numbers that are accurate enough.
And that simple idea is one of the reasons trillion-parameter models are practical at all. π«±π»βπ«²πΌ
Spent the weekend shipping with the latest MiniMax M3 model π€ for $0
A few observations: β¬οΈ
- Very good at spec-driven development
- Strong backend engineering capabilities
- Great at troubleshooting and debugging
- Frontend is okay so far.
Have you tried it? What's been your experience? π
Do you know how a reasoning model actually works? π€
Does it have some special reasoning engine hidden inside? π§©
Not really. β¬οΈ
Imagine asking two AI models:
"What is 47 Γ 23?"
The first one instantly replies: 1081
The second one says:
47 Γ 20 = 940
47 Γ 3 = 141
940 + 141 = 1081
Same answer. β
But this time you can see how it got there.
That's essentially what a reasoning model is. It's not a new architecture.
It's the same LLM, trained to think step by step before answering.
How?
π§ Chain of Thought β learn to break problems into smaller steps.
π― Reward Models β reward good reasoning, not just correct answers.
π Reinforcement Learning β reinforce reasoning paths that lead to better outcomes
β
Self-Critique β review and fix its own mistakes before responding
The result?
Instead of making a fast guess, the model learns to spend extra tokens thinking.
Same brain.
Different training.












