LLM Fundamentals: Tokens, Sampling & Non-Determinism.

Every Claude call rests on the same next-token loop: an autoregressive model predicts one token at a time, sampling from a probability distribution over the vocabulary conditioned on everything before it, appends that token, and repeats. Tokens are the smallest units the model processes; for Claude, one token is "approximately 3.5 English characters," though this varies by language, so /v1/messages/count_tokens should be used to budget usage rather than assuming a fixed character-to-token ratio.

The context window is "all the text a language model can reference when generating a response, including the response itself," a working-memory buffer, not the training corpus. Everything in a request counts toward it: system prompt, every message (including tool results, images, documents), tool definitions, plus all generated output including extended-thinking tokens. Current models (Opus 4.8/4.7/4.6, Sonnet 5/4.6, Fable 5) default to a 1M-token window at standard pricing; Sonnet 4.5 caps at 200K. More context is not automatically better: accuracy and recall can degrade as tokens grow, a documented phenomenon Anthropic calls "context rot."

Sampling and non-determinism sit on top of the same loop. temperature controls output randomness, higher increases variation, lower pushes toward the most probable tokens. The load-bearing fact for the exam: even at `temperature: 0`, output is not guaranteed reproducible, a property of the sampling process itself, not a configuration mistake.

Next-token generation is the mechanism everything else sits on. Nothing about tokens, context windows, or sampling changes this: the model always produces one token, appends it to the sequence, and conditions the next prediction on the updated sequence. Thinking tokens, tool-call arguments, and plain text are all generated through this same loop, just with different downstream handling.

Token counting is measured, not estimated. Use /v1/messages/count_tokens on representative inputs before sending a request, rather than dividing character count by a fixed ratio, since the ~3.5 chars/token figure varies by language and content. Context-window overflow is enforced server-side: if the input alone exceeds the window, the API returns a 400 error before generation starts; if input_tokens + max_tokens exceeds the window, generation can stop mid-response instead.

Sampling parameters are being phased out on the newest models, not just theoretically unreliable. On all six models that reject them - Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, Claude Opus 4.8, Claude Opus 4.7, and Claude Sonnet 5 - non-default temperature, top_p, and top_k values are rejected outright with an HTTP 400, on every request to those models, regardless of whether thinking is active. The replacement lever is output_config.effort (low/medium/high[default]/xhigh/max) combined with thinking: {type: "adaptive"}, which lets Claude decide whether and how much to think per request. This is a model-selection and cost-tuning decision in its own right; see Model Selection & Tradeoffs for the full tier/pricing/migration picture, this page stays focused on the tokens-and-sampling mechanics underneath it.

Fundamental prompting techniques sit on the same spectrum, differentiated by example count. Zero-shot supplies no examples and leaves output format to the model's judgment. Anthropic's documented technique is "few-shot or multishot prompting," wrapping "a few well-crafted examples" in <example> tags, which "improve accuracy and consistency." Single-shot (one example) is the standard midpoint between the two. This page covers the spectrum at the fundamentals level; the deep multishot technique, including the exact <sample_input>/<ideal_output> XML pattern and measured accuracy gains, lives in Prompt Engineering Techniques.