Shapes and slices
After this you can read the slicing lines in a PyTorch script and say what shape comes out.
From lists to nested lists
Guess first nested = [[1, 2, 3], [4, 5, 6]]. What does nested[:, -1] give in Python?
Recap from lesson 2: xs[-1] is the last item. A slice such as xs[2:10] copies what exists, with no error. xs[10] raises IndexError.
A nested list is a list of rows: nested[1][-1] is 6. To get every row's last item, loop: [row[-1] for row in nested].
nested[:, -1] raises TypeError. The comma packs :, a slice of everything, and -1 into one tuple. A list takes one integer or one slice.
So what takes that tuple? A tensor: a typed array with a shape, like a Float32Array of 70 numbers read as shape (2, 5, 7).
Quick check xs = [10, 20, 30, 40]. What does xs[2:10] give?
Quick check rows = [[1, 2, 3], [4, 5, 6]]. Which line gives [3, 6]?
Reading logits[:, -1, :]
Guess first A model returns its scores as logits, a block of shape (2, 5, 7). What is the shape of logits[:, -1, :]?
A model's raw scores are its logits, chapter 2's score list: one score per row of its output list. Qwen3-0.6B has 151,936 rows.
Put 2 prompts of 5 tokens in a batch. A toy model with 7 tokens in its vocabulary gives shape (2, 5, 7).
Each slot between the commas picks along one axis. Slot 1 is :, which keeps the whole axis: every prompt in the batch stays.
Slot 2 is -1, the last position. Its scores are for the token that comes after the whole prompt: the next token.
Slot 3 is : again, so all 7 scores stay. The result has shape (2, 7): one row of next-token scores for each prompt.
A plain number picks one item and drops its axis. A slice keeps it, even one item long: logits[:, -1:, :] is (2, 1, 7).
Common mistake: logits[-1]. Its -1 lands in slot 1, the batch, and missing slots mean :. You get the last prompt's whole (5, 7) block.
flat_tensor.py fakes the logits with a plain list of 0 to 69, read as shape (2, 5, 7). It prints four slices.
Run it: four slices, four shapes, the same as the drawing showed.
Real code: nanoGPT, a GPT model in one short file, takes logits[:, -1, :]; Transformers, Hugging Face's model library, takes outputs.logits[:, -1].
A batch needs prompts of one length, so short ones get filler tokens: padding. A second tensor, the attention mask, marks each filler 0.
Padded on the right, a short prompt's -1 is filler. Transformers warns: pad on the left, so -1 is real text for every prompt.
Real code: Transformers finds right padding with attention_mask[:, -1] == 0. A 0 in the last column means that row ends in filler.
Quick check What does logits[:, -1, :] select?
Quick check logits has shape (3, 16, 151936). What is the shape of logits[:, -1, :]?
Quick check One batch holds a 9-token prompt and a 5-token prompt, padded on the right to 9 tokens. You take logits[:, -1, :]. What do you get for the short prompt?
Views, copies and memory
Guess first You write last = logits[:, -1, :], then last[0, 0] = 9.0. Does logits change?
Under the shape, the 70 numbers sit in one flat row, as in the Float32Array. logits[b, p, v] sits at b*35 + p*7 + v.
Strides are the jumps per axis: a prompt is 5 × 7 = 35 numbers, a position is 7, a score is 1.
By hand, logits[:, -1, :] is two runs of 7 numbers: places 28 to 34 for prompt 0, and 63 to 69 for prompt 1.
That slice is a view: it reads the same memory with a new start and new strides. Nothing was copied.
So writing through the view changes the original, its base: after last[0, 0] = 9.0, logits[0, 4, 0] reads 9.0 too.
view_write.py takes the last position twice: as a view, last, and as a copy, mine. Then it writes into each.
Run it: the write through the view changed logits. The write into the copy did not.
JavaScript's slice() copies, but a tensor slice is a view, like subarray(). To get numbers of your own, copy: .clone() in PyTorch.
Both files take the last rows as a view: subarray() in JavaScript, our last_position() in Python. The JavaScript twin also copies with slice().
Run the JavaScript twin: the write into a view row changed logits; the write into a slice() copy did not.
The catch: a view keeps the whole original alive. In 4-byte floats, our two-row slice needs 56 bytes but holds all 280.
One 4,096-token prompt on Qwen3-0.6B gives 2.49 GB of 4-byte scores. A view of the last position, 607,744 bytes, keeps all 2.49 GB.
Transformers copies the last position, so no view holds the big block. When it can, it also asks for one position only: logits_to_keep=1.
Quick check logits has shape (2, 5, 7) and strides (35, 7, 1). At which place in the flat row is logits[1, 2, 3]?
Quick check mine = logits[:, -1, :].clone(), then mine[0, 0] = 9.0. Does logits change?
Quick check You generate text one token at a time. In a simple loop with no cache, each step runs the model on the whole text so far and makes a new logits block. For one prompt, you keep every step's logits[:, -1, :] in a list. Memory grows far faster than 607,744 bytes a step. Why?