Observability & Monitoring for Agents at Scale.

Observability for a Claude-based agent fleet spans layers an architect must monitor separately: platform-level usage/cost accounting (what Anthropic exposes natively across all API usage), Managed Agents session tracing (a Console Tracing view plus native span events, native to agents run on the Managed Agents platform specifically), and application-level execution tracing for an agent an architect orchestrates themselves directly against the Messages API, a layer Anthropic does not natively instrument.

That last gap is exactly the "observability challenge at scale" this objective tests: a single request's console log tells you little about why a multi-agent pipeline degraded across thousands of concurrent runs, and Managed Agents tracing does not help if the agent in question is not running on that platform. A scalable monitoring strategy layers execution tracing on top of, not instead of, the native cost APIs, and picks the right native tracing tool for the runtime the agent actually executes on.

Anthropic's Usage & Cost Admin API gives an organization granular historical usage/cost data - token counts (uncached input, output, cache creation/reads), usage by model/workspace/API key, and cache-efficiency analysis - intended for cost reconciliation and chargeback reporting, across all API usage regardless of runtime.

Two separate Analytics APIs exist and are easy to confuse. The Claude Code Analytics API (developer productivity metrics, via an Admin API key) is distinct from the Claude Enterprise Analytics API (org-wide chat/Projects/Code engagement and adoption, via a claude.ai Analytics API key) - architects must provision the correct one for what they're actually trying to observe.

Managed Agents now ships native session tracing. Managed Agents sessions emit session, span, and agent events (e.g. span.model_request_start / span.model_request_end) for observability into agent progress, and the Console's Managed Agents section renders a session list plus a Tracing view - a chronological, step-level replay of what a given session did (content, tool calls and results, timestamps, token usage), restricted to Developers/Admins. This closes the execution-tracing gap for agents that run on the Managed Agents platform - it does not extend to a self-built agent loop calling the raw Messages API directly.

Anthropic does not natively provide distributed tracing for a self-orchestrated agent built directly on the Messages API (your own agent loop, not a Managed Agents session). This remaining gap is filled by the vendor-neutral OpenTelemetry GenAI semantic-conventions project, which as of 2026 defines a versioned span model for agent traces (inference, embeddings, retrieval, memory, execute-tool spans) - still labeled "Development" status, so span names can change. The Claude Agent SDK / Claude Code CLI also ships built-in OTel instrumentation, recording spans around each model request and tool execution independent of Managed Agents.

The recommended OTel pattern (for the Messages-API-direct case Managed Agents tracing doesn't cover): wrap each agent run in a root span, nest a child span per LLM call/tool call/retrieval lookup, so a single trace reconstructs the full decision path an agent took - the shape that scales past "read one console log." Pair this with structured, per-turn logging of stop_reason (max_tokens vs end_turn) so silent truncations are distinguishable from clean completions at the dashboard level, not just in individual transcripts.

Third-party LLM observability platforms (Langfuse, Helicone, Datadog LLM Observability, and others) build on top of or alongside these OTel conventions to add prompt/response logging, eval scoring, and cost dashboards purpose-built for LLM apps - a reasonable augmentation layer where Anthropic's native tooling (Managed Agents tracing included) stops.