# Debugging LLM System Failures

> A Claude application fails in two structurally different places: the integration layer (the call never reached a valid model turn - bad auth, malformed request, rate limit) or model output (the call returned 200, but the content is wrong - refused, truncated, or malformed). The error object only exists on non-2xx responses; a 200 with a bad stop_reason is not an HTTP error, so your code has to detect it itself.

**Domain:** CCDVF-D4 · Evaluation, Testing & Debugging (2.6% of the exam)
**Canonical:** https://claudearchitectcertification.com/concepts/debugging-llm-system-failures
**Last reviewed:** 2026-05-04

## Quick stats

- **Typed HTTP error codes:** 11
- **Exam domain:** CCDVF-D4
- **Also tested in:** CCA-P D4 + CCA-A D7
- **Default SDK retries:** 2
- **Triage layers:** 3

## What it is

A production Claude call can fail in two structurally different places, and each needs a different fix. The integration layer covers everything before a valid model turn happens: bad auth, a malformed request, a rate limit, or a transient overload - these surface as non-2xx HTTP status codes with a typed error object (type, message, request_id). Model output failures are different in kind: the call returned 200, was billed, and completed - but the content itself is wrong, refused, truncated, or malformed. Confusing the two leads to the most common production debugging mistake: wrapping every call in a generic retry loop and shipping a truncated or refused response as if it were valid.

Origin isolation - deciding which of these two layers a given failure belongs to, before writing a fix - is the core skill this objective tests. It composes with two further steps: identifying the specific error TYPE once the layer is known, and selecting a recovery strategy that matches that type (retry with backoff, fix the request, raise max_tokens, adjust the prompt/policy, or enforce structure) rather than applying one generic reaction to every failure.

## How it works

Typed, predictable HTTP codes drive the first triage split. 400 invalid_request_error, 401 authentication_error, 402 billing_error, 403 permission_error, 404 not_found_error, 409 conflict_error, and 413 request_too_large are all non-retryable - the request itself is broken, so blind retry just repeats the failure. 429 rate_limit_error, 500 api_error, 504 timeout_error, and 529 overloaded_error are transient - retry with backoff genuinely helps. Getting this split backwards (retrying a 400, or treating a 429 as a code bug) is the most-tested distractor pattern.

SDKs retry automatically, but only within that transient set, and only twice by default. Connection errors, 429s, and 5xx responses are retried with exponential backoff honoring retry-after, configurable via a max_retries option per client. 429 responses also expose x-ratelimit-remaining-* headers for proactive throttling before a limit is hit. Catch typed SDK exceptions (e.g. Python anthropic.NotFoundError, anthropic.RateLimitError, anthropic.APIStatusError for any other non-2xx, anthropic.APIConnectionError for a network failure before any response existed) rather than string-matching error messages, and handle the most specific exception first. Every response - success or failure - carries a request-id for support correlation, the primary trace artifact for isolating a failed call.

Model-output failures live inside a 200 and surface only through stop_reason. stop_reason: "refusal" means Claude declined for safety - the call is 200 and billed, and the output may not match your schema because the refusal message takes precedence over schema constraints. stop_reason: "max_tokens" means truncation - the response may be incomplete and not match your schema either. Neither raises an exception; both require branching on stop_reason before the payload is trusted, which is precisely what a try/except around typed exceptions alone will not catch.

Structured Outputs and Strict Tool Use close the remaining JSON-quality gap, but they are two separate mechanisms, not one setting. Structured Outputs is configured via output_config.format on the request and constrains the model's final text response to a JSON Schema. Strict Tool Use is configured with a top-level strict: true field on an individual tool definition (alongside name, description, and input_schema) and constrains that tool's tool_use.input to its schema. output_config.format does not take a tool-level strict: true parameter - they are configured independently, on different objects, for different output paths (final response vs. a specific tool call), even though both compile to grammar-constrained decoding through the same underlying pipeline. Neither overrides a refusal or rescues a truncated response. One residual gap even under strict mode: enum/const values can differ only in leading-word capitalization with no error and no distinguishing stop_reason, so compare enum output case-insensitively rather than assuming exact-match. Exceeding schema complexity limits (20 strict tools per request, 24 optional params, 16 union-typed params) pushes the failure back to the integration layer entirely - a 400 before generation ever starts.

Origin isolation also covers prompt-design and model-selection failures, one level above HTTP/stop_reason mechanics. Not every wrong answer is a refusal or truncation - a well-formed, schema-compliant 200 can still be a hallucination (confident fabrication with no refusal signal), a prompt failure (ambiguous instructions producing a technically valid but off-target answer), or a model mismatch (a lighter/cheaper model handling a task past its reliability ceiling for that complexity). None of these raise a typed exception or an unusual stop_reason - they surface only through evaluation against ground truth or ground-truth comparison against a stronger model, which is why an architect's diagnostic toolkit extends past dev-level error handling into eval-driven root-causing.

## Where you'll see it in production

### Ticket-extraction pipeline under load

Triage order: (1) did the call raise RateLimitError/APIStatusError - integration layer, fix with backoff/queuing; (2) if 200, read stop_reason first - max_tokens means raise the budget, refusal means fix policy/prompt, not retry; (3) only if stop_reason is end_turn/tool_use and JSON still fails to parse is it a genuine content-quality issue, fixed with output_config.format or strict: true.

### Non-technical practitioner triage

A delivery consultant gets a vague, off-target answer from a client-facing Claude workflow - the CCAOF-D7-O1 framing treats this as a prompt-iteration problem (add specificity, examples, explicit format) before escalating it as a system-reliability incident.

## Comparison

| Failure class | HTTP status | Signal | Fix approach |
| --- | --- | --- | --- |
| Integration - transient | 429 / 500 / 504 / 529 | Typed error, retryable | Retry with backoff (SDK does this automatically, twice by default) |
| Integration - non-retryable | 400 / 401 / 402 / 403 / 404 / 409 / 413 | Typed error, permanent | Fix the request/auth/schema; blind retry repeats the failure |
| Model output - refused | 200 | stop_reason: "refusal" | Not an API error; adjust policy/prompt, don't retry the same call |
| Model output - truncated | 200 | stop_reason: "max_tokens" | Raise the token budget; response may not match schema |
| Model output - malformed/non-compliant | 200 | end_turn/tool_use but JSON fails to parse or match schema | output_config.format / strict: true; compare enums case-insensitively |

## Exam-pattern questions

### Q1. A call returns 429 rate_limit_error. What's the correct recovery strategy?

Retry with exponential backoff, honoring retry-after - this is exactly what the SDK does automatically (twice by default). The distractor "the request itself is malformed and needs to be rewritten" misclassifies a transient, retryable error as a permanent one; 429 means slow down, not fix the payload.

### Q2. A call returns 400 invalid_request_error. Will retrying the identical payload after a short delay eventually succeed?

No - 400 is non-retryable; the request itself is broken and needs to be fixed, not resent. The distractor "retrying with the same payload after a short delay will eventually succeed" treats a permanent client error as if it were transient like a 429 or 529.

### Q3. A call returns 200 with stop_reason: "refusal". Is this a 4xx client error that a retry loop should catch?

No - it's a successful, billed 200 response; Claude declined for safety, and the output may not match your schema because the refusal message overrides schema constraints. The distractor "this is a 4xx client error that should be retried with an adjusted prompt" misplaces a model-output failure in the integration layer, where typed-exception handling would never catch it.

### Q4. A pipeline sets output_config.format on the request and separately sets strict: true on one of its tool definitions. Does either guarantee a schema-valid response when stop_reason is "refusal"?

No - output_config.format (Structured Outputs, constrains the final response) and strict: true (Strict Tool Use, a tool-level field constraining that tool's input) are two independent mechanisms that both use constrained decoding for the case where generation completes cleanly; neither overrides a refusal or rescues a truncated (max_tokens) response. The distractor "strict: true is a parameter of output_config.format" conflates two separately-configured mechanisms into one.

### Q5. A well-formed, schema-compliant 200 response confidently states a fact that is entirely fabricated - no refusal, no truncation, no exception raised. What kind of failure is this, and how is it caught?

A hallucination - a model-output content-quality failure that produces neither an HTTP error nor an unusual stop_reason, so it can only be caught via evaluation against ground truth, not exception handling. The distractor "this raises APIStatusError because the content failed internal Anthropic validation" invents a validation mechanism that doesn't exist for factual content; there is no API-level fact-check.

## FAQ

### Q1. Does a 200 status mean the response content can be trusted?

No - 200 only means the call completed and was billed. Refusal, truncation, hallucination, and schema-noncompliant content can all arrive inside a 200; check stop_reason and, beyond that, evaluate content quality separately.

### Q2. Do typed SDK exceptions (RateLimitError, APIStatusError, etc.) replace the need to check stop_reason?

No - typed exceptions only fire on non-2xx responses. A successful call still needs its stop_reason inspected before the payload is trusted; exception handling and stop_reason branching cover different failure layers.

---

**Source:** https://claudearchitectcertification.com/concepts/debugging-llm-system-failures
**Last reviewed:** 2026-05-04

**Evidence tiers**, 🟢 official Anthropic doc / API contract · 🟡 partial doc / inferred · 🟠 community-derived · 🔴 disputed.