What inference is, and your first model
After this you can run Qwen3-0.6B on your laptop with Ollama, and get an answer over HTTP from curl, JavaScript and Python.
Inference runs a trained file
Guess first Qwen3-0.6B is a small language model. You will download it in a few minutes. How big is the file?
Your team wants a "Draft reply" button in a support tool. A small model on one laptop writes it. First, get that model running.
A trained model is a file of numbers. Training set those numbers once, before you ever download the file.
The Qwen3-0.6B file holds 751,632,384 learned numbers, about 752 million.
Why "0.6B"? The file stores one table twice, to read text in and write text out. Counted once: about 596 million. That is the 0.6B.
Inference means: run that trained file on new input, like your prompt, and get an output, like an answer.
Common mistake: chatting teaches the model. It does not. Our run took the file's fingerprint (hash) before and after a chat: the same.
Quick check What happens during inference?
Quick check Where did Qwen3-0.6B's 751,632,384 numbers come from?
Get Ollama and the model
Ollama is a program that downloads models and runs them on your own computer. It also runs a small web server you can call.
Install it. macOS 14 or newer: the app, or the install script. Windows 10 22H2 or newer: OllamaSetup.exe. Linux: the install script.
Already have Ollama? ollama -v shows its version. If it is not 0.35.0, run the line above: it replaces your Ollama with 0.35.0.
Check it is on: curl http://localhost:11434/api/version. Ours replied 0.35.0. Any version number means the server is running.
The server listens on 127.0.0.1, port 11434. That is your own machine. No cloud is involved in this call.
ollama pull qwen3:0.6b downloads it: 522.7 MB. The terminal cuts that to 522, Ollama's site rounds to 523. Ours took 118 s.
ollama list shows the models you have downloaded. They live in ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux, C:\Users\%username%\.ollama\models on Windows.
The ID is not the hash the pull printed. It starts the hash of the manifest, a small file that lists the model's parts.
This is a 4-bit copy of Qwen3-0.6B (Q4_K_M). Each number uses fewer bits than the original. The next chapter explains why that matters.
4 bits each would be 376 MB. Some tables keep 6 or 16 bits, and each block adds a scale: 5.5 bits on average.
Quick check Which command downloads the model file but does not start a chat?
Quick check You run curl http://localhost:11434/api/version and curl stops with error code 7. What does it mean?
Your first call, with curl
Guess first You ask the model a question with curl. How does the answer come back?
The request is JSON sent to the endpoint /api/chat. It names the model and lists messages. Each has a role and content.
Add "think": false. Qwen3 thinks before it answers by default. We turn thinking off for now. Lesson 3 shows what it does.
Run it. The answer does not come in one block. It comes as many JSON lines, one by one. This is a stream.
The model writes text in pieces called tokens: often a word, sometimes part of one, like "'s" in "planet's". Ollama counts them for you.
1 your script posts, 2 Ollama loads the model if needed, 3 the model reads the prompt, 4 it writes tokens, 5 text streams back.
Step 6: the last line has "done": true and counters about the run. Times are in nanoseconds (billionths of a second). Lesson 2 reads them.
eval_count is how many tokens the model wrote. Ours: 27, while the stream had 26 lines of text. So count tokens with eval_count, not lines.
Which token had no line? The last one. When the model stops on its own, its last token only marks the end, with no text.
prompt_eval_cached_count: prompt tokens Ollama reused from its cache instead of reading them again. Ours: 0, on our first call.
Run it again right away and the number grows: our second call reused 22 of its 23 prompt tokens. The next chapter shows this cache.
The file first-call-once.sh adds "stream": false, so you get one JSON reply. It is easier for scripts. You lose the live pieces.
Quick check You run first-call.sh. How does the answer arrive?
Quick check In a streamed reply, where are the counters about the run?
The same call from JavaScript and Python
curl is good for a quick try. Real apps call the same endpoint from code. Same JSON in, same JSON out.
JavaScript: fetch is built into Node 18 and newer (node -v shows yours). POST the same JSON body, with stream: false and think: false.
fetch does not throw when the server answers 404. Check res.ok. Ollama puts the reason in an error field.
Run node first-call.js. It prints the answer and the number of tokens in it.
Python: urllib.request comes with Python, so there is nothing to install. Giving it data makes the request a POST.
HTTPError: the server said no, like 404. URLError: no server reached. Catch HTTPError first, because it is a kind of URLError.
Run python3 first_call.py. Same kind of reply as JavaScript and curl.
Three tools, one server, the same shape of reply. The words differ each run: the model picks words with some randomness. This is sampling.
This picture is our map for the chapter. Lesson 2 times each step.
Quick check In Node, await fetch(...) gets an HTTP 404 from Ollama. What happens?
Quick check Put the steps of one call in order: