# LLM Fundamentals: Tokens, Sampling & Non-Determinism > Claude is autoregressive: it predicts one token at a time from a probability distribution conditioned on everything before it, then appends that token and repeats. Even at temperature: 0, Anthropic states results "will not be fully deterministic and identical inputs may produce different outputs across API calls," true for first-party and third-party inference alike. **Domain:** CCDVF-D5 · Model Selection & Optimization (16.8% of the exam) **Canonical:** https://claudearchitectcertification.com/concepts/llm-fundamentals-tokens-and-sampling **Last reviewed:** 2026-05-04 ## Quick stats - **Avg characters per token:** ~3.5 - **Exam domain:** CCDVF-D5 - **Domain weight:** 16.8% - **Models rejecting temp/top_p/top_k:** 6 - **Multishot example sweet spot:** 2-5 ## What it is Every Claude call rests on the same next-token loop: an autoregressive model predicts one token at a time, sampling from a probability distribution over the vocabulary conditioned on everything before it, appends that token, and repeats. Tokens are the smallest units the model processes; for Claude, one token is "approximately 3.5 English characters," though this varies by language, so /v1/messages/count_tokens should be used to budget usage rather than assuming a fixed character-to-token ratio. The context window is "all the text a language model can reference when generating a response, including the response itself," a working-memory buffer, not the training corpus. Everything in a request counts toward it: system prompt, every message (including tool results, images, documents), tool definitions, plus all generated output including extended-thinking tokens. Current models (Opus 4.8/4.7/4.6, Sonnet 5/4.6, Fable 5) default to a 1M-token window at standard pricing; Sonnet 4.5 caps at 200K. More context is not automatically better: accuracy and recall can degrade as tokens grow, a documented phenomenon Anthropic calls "context rot." Sampling and non-determinism sit on top of the same loop. temperature controls output randomness, higher increases variation, lower pushes toward the most probable tokens. The load-bearing fact for the exam: even at temperature: 0, output is not guaranteed reproducible, a property of the sampling process itself, not a configuration mistake. ## How it works Next-token generation is the mechanism everything else sits on. Nothing about tokens, context windows, or sampling changes this: the model always produces one token, appends it to the sequence, and conditions the next prediction on the updated sequence. Thinking tokens, tool-call arguments, and plain text are all generated through this same loop, just with different downstream handling. Token counting is measured, not estimated. Use /v1/messages/count_tokens on representative inputs before sending a request, rather than dividing character count by a fixed ratio, since the ~3.5 chars/token figure varies by language and content. Context-window overflow is enforced server-side: if the input alone exceeds the window, the API returns a 400 error before generation starts; if input_tokens + max_tokens exceeds the window, generation can stop mid-response instead. Sampling parameters are being phased out on the newest models, not just theoretically unreliable. On all six models that reject them - Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, Claude Opus 4.8, Claude Opus 4.7, and Claude Sonnet 5 - non-default temperature, top_p, and top_k values are rejected outright with an HTTP 400, on every request to those models, regardless of whether thinking is active. The replacement lever is output_config.effort (low/medium/high[default]/xhigh/max) combined with thinking: {type: "adaptive"}, which lets Claude decide whether and how much to think per request. This is a model-selection and cost-tuning decision in its own right; see Model Selection & Tradeoffs for the full tier/pricing/migration picture, this page stays focused on the tokens-and-sampling mechanics underneath it. Fundamental prompting techniques sit on the same spectrum, differentiated by example count. Zero-shot supplies no examples and leaves output format to the model's judgment. Anthropic's documented technique is "few-shot or multishot prompting," wrapping "a few well-crafted examples" in tags, which "improve accuracy and consistency." Single-shot (one example) is the standard midpoint between the two. This page covers the spectrum at the fundamentals level; the deep multishot technique, including the exact / XML pattern and measured accuracy gains, lives in Prompt Engineering Techniques. ## Where you'll see it in production ### Audit logging system Instead of assuming temperature: 0 gives replayable outputs for compliance review, the system logs the actual generated output at call time, since replay isn't guaranteed even at zero temperature. ### Multilingual classification pipeline Calls /v1/messages/count_tokens per input language before sizing max_tokens, instead of a single fixed characters-per-token heuristic that would misestimate non-English inputs. ## Comparison | Technique | Examples in prompt | Best for | Tradeoff | | --- | --- | --- | --- | | Zero-shot | 0 | Simple, well-understood tasks with an obvious output shape | Format and edge-case handling left entirely to the model's judgment | | Single-shot | 1 | Establishing one concrete pattern or format quickly | One example rarely covers every edge case or failure mode | | Multi-shot (few-shot) | 2-5, wrapped in tags | Consistent format/tone across ambiguous or edge-case-heavy tasks | More prompt tokens per call; only pays off when examples target observed failures | ## Decision tree 1. **Does this workload require exact, reproducible output across repeated calls (e.g. for compliance replay)?** - **Yes:** Design for it explicitly, log actual outputs, validate against a schema, don't rely on temperature: 0; Anthropic states it does not guarantee determinism, on any inference path. - **No:** Standard sampling behavior is fine as-is. 2. **Is the target model one of the six that reject non-default sampling params (Fable 5, Mythos 5, Mythos Preview, Opus 4.8, Opus 4.7, Sonnet 5)?** - **Yes:** Non-default temperature/top_p/top_k will be rejected with a 400; use output_config.effort and adaptive thinking instead. - **No:** Sampling parameters are still accepted, though output remains non-deterministic regardless. 3. **Is the task's correct output format ambiguous, or does it have known edge cases Claude has failed on before?** - **Yes:** Add 2-5 multishot pairs targeting exactly those observed failures. - **No:** Zero-shot is likely sufficient; added examples would just cost tokens for no accuracy gain. 4. **Do you need to budget max_tokens or estimate cost before sending a request?** - **Yes:** Call /v1/messages/count_tokens on representative inputs rather than estimating from a fixed character ratio. - **No:** No pre-flight token count needed for this call. ## Exam-pattern questions ### Q1. A compliance team wants to log Claude's output once and treat future identical requests as "replayable" for audit purposes, using temperature: 0 to guarantee this. Will this work? No. Anthropic states that even at temperature: 0, "results will not be fully deterministic and identical inputs may produce different outputs across API calls," true for both first-party and third-party inference. Named distractor: "yes, as long as top_p and top_k are also left at default", leaving other sampling params at default doesn't change the underlying non-determinism guarantee. ### Q2. A developer sets temperature: 0.7 on a request to claude-opus-4-8 and receives an HTTP 400 error. What's the cause, and what's the fix? Opus 4.8 rejects non-default temperature/top_p/top_k outright, on every request, regardless of thinking state. The fix is to control output via output_config.effort and thinking: {type: "adaptive"} instead. Named distractor: "lower the temperature value below 0.7 until it's accepted", wrong, any non-default value is rejected on this model, not just values above some threshold. ### Q3. A pipeline estimates max_tokens budgets by dividing each input's character count by 4, across English, Japanese, and code inputs. What's wrong with this approach? Tokens are roughly 3.5 characters for English but vary meaningfully by language and content type; a fixed ratio is unreliable across mixed-language inputs. The correct approach is calling /v1/messages/count_tokens on the actual input. Named distractor: "the ratio is fine as long as you round up", rounding doesn't fix a ratio that's wrong by a different, unpredictable margin per language. ### Q4. An extraction task with a genuinely ambiguous output format is producing inconsistent field names run to run under zero-shot prompting. What's the recommended fix per Anthropic's documented technique? Add 2-5 multishot examples wrapped in tags, chosen to cover the specific failure modes observed, not generic passing cases. Named distractor: "lower the temperature to reduce variation", temperature tuning doesn't fix an underspecified output *format*, and is being phased out as a lever on the newest models entirely. ### Q5. After migrating a workload to Sonnet 5, a call using thinking: {type: "enabled", budget_tokens: 4000} starts returning a 400 error. What changed? Manual extended thinking with budget_tokens is rejected on Sonnet 5; it requires thinking: {type: "adaptive"} instead, tuned via output_config.effort. Named distractor: "the budget_tokens value of 4000 is too low for this model", the parameter itself is rejected on this model generation, not the specific value supplied. ## FAQ ### Q1. Does temperature: 0 guarantee identical outputs across calls? No. Anthropic explicitly states this is false, even at 0, results are not fully deterministic, on both first-party and third-party inference. ### Q2. Is a token the same thing as a word or a character? No. A token is roughly 3.5 English characters on average but varies by language and content; use /v1/messages/count_tokens for an exact count rather than a fixed conversion ratio. --- **Source:** https://claudearchitectcertification.com/concepts/llm-fundamentals-tokens-and-sampling **Last reviewed:** 2026-05-04 **Evidence tiers**, 🟢 official Anthropic doc / API contract · 🟡 partial doc / inferred · 🟠 community-derived · 🔴 disputed.