Chapter 2Free
What just happened: tokens, the loop, the two phases, and "will it fit?"
Explain how a model writes one token at a time, why the first token takes longest, and whether a model file fits in GPU memory.
About 4 h 5 min
What you will do
- 01Guess firstBefore a new idea, write down what you think. It is never graded.
- 02Learn5 short lessons, one small step at a time.
- 03Run itRun four labs on your laptop, then break it 7 ways.
- 04Check10 questions. Pass with 8 right, including weights in GB and which phase caching shortened.
- 05ReviewFour quick questions from chapter 1.
In this chapter
- Tokens are pieces, not wordsAfter this you can say how your text becomes token IDs, count tokens three ways, and predict why a language or a format costs more.About 30 min
- One token at a timeAfter this you can explain the loop, read done_reason, set a seed and a cap on purpose, and say why thinking tokens cost what they cost.About 25 min
- Prefill and decodeAfter this you can read prefill and decode from your counters, say what limits each phase, and compare your numbers with real targets.About 25 min
- No memory between callsAfter this you can show that each call carries the whole chat, read the exact string Ollama builds, catch a silent cut from the counters, and order a prompt so caching pays.About 25 min
- Will it fit?After this you can size a model's weights and its cache in the right unit, read a real file and a real engine log, and say which tool fits where and why.About 30 min
- Practice: four labs on your laptopCount tokens three ways, run the loop by hand, time both phases, and fill a fit table.About 50 min
- Break itChange one thing, guess first, look, and explain.About 45 min
- Chapter check10 questions. Pass with 8 right, including weights in GB and which phase caching shortened.About 10 min
- ReviewFour quick questions from chapter 1.About 5 min
Stuck? Known errors for this chapter
Previous chapter: Chapter 1, Your first model, measured