Break it
Change one thing, guess first, look, and explain. Part of chapter 2: What just happened: tokens, the loop, the two phases, and "will it fit?"
Turn 2 without turn 1
What changes
Whether turn 1 is sent. Everything else stays the same: think: false, temperature: 0, num_ctx: 4096.
Run it
Run the script. It asks the question alone, then again with turn 1 in front. Nothing else changes.
Raw mode, with and without the template
What changes
Who writes the template. Three raw calls: the bare question, then role marks we wrote by hand, with and without the empty think block.
Run it
Run the script. Three raw calls, each at most 120 tokens. Calls 2 and 3 carry a template we wrote by hand.
Three ways to stop
What changes
How the loop may end. The same question three times: a 20-token cap, a stop string ".", and no limit.
Run it
Run the script. Watch done_reason and the end of each reply.
Thinking eats the budget
What changes
Thinking on, with a small cap: think: true and num_predict: 64. Same question as before.
Run it
Run the same script with --think. Watch content, eval_count and done_reason.
Same prompt, different words
What changes
The seed and the temperature. One sentence about the sea: 3 default calls, 3 with seed 42, then 5 at temperature 0.
Run it
Run the script. It says which outputs match, word for word.
The silent cut
What changes
num_ctx: 512 tokens. One call with a rule at the top, about 800 tokens of notes, and a question at the end.
Run it
Run the script. One chat call at num_ctx 512, thinking off.
Fix it with room
Give the prompt room: run the script at 2,048 tokens. All of it reaches the model, and the reply follows the rule.
Go further: make the cut loud
Make the cut loud: --strict adds two calls at 512, with "truncate": false and "shift": false. Both came back HTTP 400 in our run.
The memory knob
What changes
num_ctx again, now for memory: one call at 4,096 tokens, then one at 16,384. Watch the KV buffer and SIZE.
Run it
Run the script. After each call it prints ollama ps and the KV line of the server log.
Go further: two users
This changes a server setting, so undo it after. Set two slots, then restart Ollama. On Linux or Windows, follow the FAQ's steps.
Run it again
Run python3 mem_knob.py --only 4096 again. Does the KV line double?
Two slots, twice the cache
In our run the KV line doubled, from 448 to 896 MiB: Ollama makes cache room for every slot, even when one user is talking.
Undo it
Put the setting back when you are done, then restart Ollama.
Close the chapter
A model you have not met
Last task, no worked rows: qwen3:14b on a 16 GB card. Budget: 14.23 GiB, vLLM's 92% of the card's 15.47 GiB.
Your numbers
14,768,307,200 parameters. Ollama's 4-bit download: 9,276,198,565 bytes. Cache: 163,840 bytes per token.