# Model Selection and Cost Routing

> A cost/latency router across the current Claude lineup: Haiku 4.5 for cheap classification and routing decisions, Sonnet 5 as the default workhorse for the bulk of production traffic, and Opus 4.8 reserved for explicitly-defined hard or high-stakes tasks. Escalation criteria are data-driven, not vibes, and output_config.effort is tuned per tier before ever reaching for a bigger model. Non-interactive bulk work moves to the Message Batches API for a 50% discount.

**Sub-marker:** CCD-F.4
**Domains:** CCDVF-D5 · Model Selection & Optimization, CCDVF-D2 · Applications & Integration, CCDVF-D6 · Prompt & Context Engineering
**Exam weight:** CCD-F D5 - Model Selection & Optimization (16.8%)
**Build time:** 18 minutes
**Source:** 🟡 Developer-cert production pattern
**Canonical:** https://claudearchitectcertification.com/scenarios/model-selection-and-cost-routing
**Last reviewed:** 2026-07-12

## In plain English

Not every request deserves the most expensive model. A one-line classification and a multi-step agentic refactor have wildly different intelligence requirements, and routing both through the same top-tier model is the single most common way teams overspend on Claude. This scenario is the router that sits in front of your requests: a cheap first pass decides how hard the task actually is, then the request is sent to the cheapest model tier that can do it well, with an explicit, data-driven escalation path rather than a blanket 'always use the best model' default.

## Exam impact

Model Selection & Optimization (D5) is 16.8% of CCD-F, the second-heaviest domain. It tests whether you can justify a model choice with cost/latency/quality tradeoffs rather than defaulting to the newest, most expensive model for everything. D2 (Applications & Integration) covers the router's request path; D6 (Prompt & Context Engineering) covers tuning effort and prompts per tier before escalating model tier.

## The problem

### What the customer needs
- Simple, high-volume requests answered cheaply and fast.
- Hard, high-stakes requests get the intelligence they actually need.
- A graceful fallback when the highest tier is rate-limited, instead of a hard failure.

### Why naive approaches fail
- Routing every request to Opus 4.8 'to be safe' multiplies cost 5x with no measurable quality gain on the 80% of traffic that's routine.
- Routing every request to Haiku for cost savings degrades quality on genuinely complex requests that need more reasoning depth.
- No fallback tier on a 429/529 from the top model fails the whole request instead of degrading gracefully.

### Definition of done
- Routing decision made in < 200ms via a cheap classification pass
- Escalation criteria documented and measured, not guessed
- Blended per-request cost tracked and trending down as routing improves
- Rate-limit on any one tier degrades to the next tier, never a hard failure

## Concepts in play

- 🟢 **Model selection** (`model-selection-tradeoffs`), Tier selection criteria
- 🟢 **Accuracy/latency/cost tradeoffs** (`accuracy-latency-cost-tradeoffs`), Router decision logic
- 🟡 **Prompt caching** (`prompt-caching`), Shared prefix across tiers
- 🟡 **Batch API** (`batch-api`), Non-interactive bulk work

## Components

### Router, Haiku 4.5 classification pass

A single cheap Haiku 4.5 call (or a lightweight heuristic) classifies request complexity/intent before the real request is dispatched to the right tier.

**Configuration:** router_resp = client.messages.create(model="claude-haiku-4-5", max_tokens=16, output_config={"effort":"low"}, ...). Output is a tier label, not the final answer.
**Concept:** `model-selection-tradeoffs`

### Workhorse Tier, Sonnet 5 - default

The default destination for the large majority of production requests: near-Opus quality on coding/agentic work at a fraction of the cost.

**Configuration:** model="claude-sonnet-5", output_config={"effort":"high"} (the default; tune down for routine work).
**Concept:** `model-selection-tradeoffs`

### Escalation Tier, Opus 4.8 - explicit criteria

Reserved for requests that meet a documented bar: long-horizon agentic work, high-stakes correctness requirements, or tasks that measurably fail on Sonnet 5 in the eval harness.

**Configuration:** model="claude-opus-4-8", output_config={"effort":"high"} or "xhigh" for the hardest coding/agentic subset.
**Concept:** `model-selection-tradeoffs`

### Fallback Handler, graceful tier step-down

On a 429/529 from the escalation tier, retries the request one tier down rather than failing outright.

**Configuration:** except RateLimitError: retry with model=WORKHORSE_MODEL instead of the escalation tier.
**Concept:** `accuracy-latency-cost-tradeoffs`

## Build steps

### 1. Classify the request with a cheap Haiku pass

The router call itself must be cheap. Keep max_tokens tiny, effort low, and ask only for a tier label, not an explanation.

**Python:**

```python
def classify_tier(request_text: str) -> str:
    resp = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=16,
        output_config={"effort": "low"},
        system="Classify the request as one of: simple, standard, complex. Reply with one word.",
        messages=[{"role": "user", "content": request_text}],
    )
    return resp.content[0].text.strip().lower()
```

**TypeScript:**

```typescript
async function classifyTier(requestText: string): Promise<string> {
  const resp = await client.messages.create({
    model: "claude-haiku-4-5",
    max_tokens: 16,
    output_config: { effort: "low" },
    system: "Classify the request as one of: simple, standard, complex. Reply with one word.",
    messages: [{ role: "user", content: requestText }],
  });
  const block = resp.content[0];
  return block.type === "text" ? block.text.trim().toLowerCase() : "standard";
}
```

Concept: `model-selection-tradeoffs`

### 2. Route to a tier and set per-tier effort

Map the classification to a model + effort pair. Sonnet 5 is the default for 'standard'; escalate only 'complex' to Opus.

**Python:**

```python
TIER_MAP = {
    "simple": ("claude-haiku-4-5", "low"),
    "standard": ("claude-sonnet-5", "high"),
    "complex": ("claude-opus-4-8", "high"),
}

def route_request(tier: str):
    model, effort = TIER_MAP.get(tier, TIER_MAP["standard"])
    return model, effort
```

**TypeScript:**

```typescript
const TIER_MAP: Record<string, [string, "low" | "medium" | "high" | "xhigh"]> = {
  simple: ["claude-haiku-4-5", "low"],
  standard: ["claude-sonnet-5", "high"],
  complex: ["claude-opus-4-8", "high"],
};

function routeRequest(tier: string) {
  return TIER_MAP[tier] ?? TIER_MAP.standard;
}
```

Concept: `accuracy-latency-cost-tradeoffs`

### 3. Dispatch on the chosen tier

The router's decision feeds a normal Messages API call - the routing logic and the actual request are two separate calls, not one combined prompt.

**Python:**

```python
def dispatch(request_text: str, system_prompt: str):
    tier = classify_tier(request_text)
    model, effort = route_request(tier)
    return client.messages.create(
        model=model,
        max_tokens=4096,
        output_config={"effort": effort},
        system=system_prompt,
        messages=[{"role": "user", "content": request_text}],
    )
```

**TypeScript:**

```typescript
async function dispatch(requestText: string, systemPrompt: string) {
  const tier = await classifyTier(requestText);
  const [model, effort] = routeRequest(tier);
  return client.messages.create({
    model,
    max_tokens: 4096,
    output_config: { effort },
    system: systemPrompt,
    messages: [{ role: "user", content: requestText }],
  });
}
```

Concept: `model-selection-tradeoffs`

### 4. Fall back a tier on rate limit or overload

Catch typed rate-limit/overload exceptions on the escalation tier and step down one tier rather than surfacing a hard failure to the user.

**Python:**

```python
def dispatch_with_fallback(request_text: str, system_prompt: str):
    tier = classify_tier(request_text)
    model, effort = route_request(tier)
    try:
        return client.messages.create(model=model, max_tokens=4096,
            output_config={"effort": effort}, system=system_prompt,
            messages=[{"role": "user", "content": request_text}])
    except (anthropic.RateLimitError, anthropic.APIStatusError) as e:
        if model == "claude-opus-4-8":
            return client.messages.create(model="claude-sonnet-5", max_tokens=4096,
                output_config={"effort": "high"}, system=system_prompt,
                messages=[{"role": "user", "content": request_text}])
        raise
```

**TypeScript:**

```typescript
async function dispatchWithFallback(requestText: string, systemPrompt: string) {
  const tier = await classifyTier(requestText);
  const [model, effort] = routeRequest(tier);
  try {
    return await client.messages.create({
      model, max_tokens: 4096, output_config: { effort },
      system: systemPrompt, messages: [{ role: "user", content: requestText }],
    });
  } catch (e) {
    if (model === "claude-opus-4-8" &&
        (e instanceof Anthropic.RateLimitError || e instanceof Anthropic.APIError)) {
      return client.messages.create({
        model: "claude-sonnet-5", max_tokens: 4096, output_config: { effort: "high" },
        system: systemPrompt, messages: [{ role: "user", content: requestText }],
      });
    }
    throw e;
  }
}
```

Concept: `accuracy-latency-cost-tradeoffs`

### 5. Move non-interactive bulk work to the Batches API

Anything latency-insensitive (nightly re-scoring, bulk classification) should run through Message Batches for the 50% discount rather than live requests.

**Python:**

```python
def submit_bulk_job(items: list[dict]):
    return client.messages.batches.create(requests=[
        {"custom_id": item["id"], "params": {
            "model": "claude-sonnet-5", "max_tokens": 1024,
            "messages": [{"role": "user", "content": item["text"]}],
        }} for item in items
    ])
```

**TypeScript:**

```typescript
async function submitBulkJob(items: { id: string; text: string }[]) {
  return client.messages.batches.create({
    requests: items.map((item) => ({
      custom_id: item.id,
      params: {
        model: "claude-sonnet-5", max_tokens: 1024,
        messages: [{ role: "user", content: item.text }],
      },
    })),
  });
}
```

Concept: `model-selection-tradeoffs`

### 6. Log routing decisions for a data-driven escalation policy

Every dispatched request logs the chosen tier, tokens, cost, and a downstream quality signal, so escalation criteria come from measured data, not intuition.

**Python:**

```python
def log_routing_decision(request_id, tier, model, usage, quality_signal=None):
    telemetry.insert({
        "request_id": request_id, "tier": tier, "model": model,
        "input_tokens": usage.input_tokens, "output_tokens": usage.output_tokens,
        "quality_signal": quality_signal,
    })
```

**TypeScript:**

```typescript
async function logRoutingDecision(
  requestId: string, tier: string, model: string,
  usage: { input_tokens: number; output_tokens: number },
  qualitySignal?: number,
) {
  await telemetry.insert({
    request_id: requestId, tier, model,
    input_tokens: usage.input_tokens, output_tokens: usage.output_tokens,
    quality_signal: qualitySignal ?? null,
  });
}
```

Concept: `accuracy-latency-cost-tradeoffs`

## Decision matrix

| Decision | Right answer | Wrong answer | Why |
|---|---|---|---|
| Default model tier for production traffic | claude-sonnet-5 | claude-opus-4-8 for every request | Sonnet 5 is the current cost/quality default; Opus should be an explicit escalation, not the baseline. |
| Routing decision model | claude-haiku-4-5, tiny max_tokens, low effort | the same model as the actual request, run twice | The router only needs a tier label, not the answer - it should be the cheapest, fastest call in the pipeline. |
| Escalation criteria | measured failure patterns on the workhorse tier, tracked over time | gut feeling that a task 'seems hard' | Data-driven escalation avoids both over- and under-spending; an eval harness (Scenario CCD-F.5) is the source of that data. |
| Rate-limit handling on the top tier | step down to the workhorse tier and retry | surface a hard failure to the user | A slightly lower-quality answer beats no answer, and Opus rate limits should not take down the whole request path. |

## Failure modes

| Anti-pattern | Failure | Fix |
|---|---|---|
| CCDF-11 · Blanket top-tier routing | Every request, regardless of complexity, is sent to claude-opus-4-8 'to be safe'. | Route routine traffic through Sonnet 5 by default; reserve Opus for a documented, measured escalation criterion. |
| CCDF-12 · No fallback on rate limit | A 429 from the escalation tier propagates as a user-facing error. | Catch the typed rate-limit exception and retry one tier down before failing. |
| CCDF-13 · Expecting determinism from temperature | Code sets temperature=0 on Opus 4.8 expecting identical outputs every run. | Sampling parameters are removed on Opus 4.7+ and Sonnet 5 (400 error). Even where they existed, temperature=0 never guaranteed identical outputs - use prompting for consistency instead. |
| CCDF-14 · Stale token-cost assumptions across tokenizers | Cost model assumes Sonnet 5 token counts match Sonnet 4.6 for the same text. | Sonnet 5 uses a new tokenizer producing roughly 30% more tokens for the same input than Sonnet 4.6 - re-baseline with count_tokens rather than reusing an old multiplier. |

## Implementation checklist

- [ ] Router uses Haiku 4.5 with tiny max_tokens and low effort (`model-selection-tradeoffs`)
- [ ] Sonnet 5 is the documented default tier for standard traffic
- [ ] Opus 4.8 escalation criteria are written down and measured, not assumed (`accuracy-latency-cost-tradeoffs`)
- [ ] Rate-limit/overload on the top tier falls back one tier before failing
- [ ] Non-interactive bulk work runs through the Batches API (`batch-api`)
- [ ] Per-tier cost and quality signals logged for ongoing router tuning

## Cost &amp; latency

- **Haiku 4.5:** $1 / $5 per MTok, Router classification pass and any simple, high-volume traffic.
- **Sonnet 5:** $3 / $15 per MTok ($2/$10 intro through 2026-08-31), Default workhorse tier for the bulk of production requests.
- **Opus 4.8:** $5 / $25 per MTok, Escalation tier only, gated by measured criteria, not the default path.
- **Blended cost estimate (80/15/5 split):** ~$4.20 / MTok input blended, 80% Haiku+Sonnet routine traffic, 15% Sonnet standard, 5% Opus escalation - illustrative, tune against your own traffic mix.

## Domain weights

- **CCDVF-D5 · Model Selection & Optimization (16.8%):** Router + tier map + escalation criteria
- **CCDVF-D2 · Applications & Integration (33.1%):** Dispatch + fallback request path
- **CCDVF-D6 · Prompt & Context Engineering (11.0%):** Per-tier effort tuning

## Practice questions

### Q1. A team routes every production request through Opus 4.8 because 'it's the smartest model'. Cost is 5x a comparable Sonnet-5-default setup with no measurable quality difference on 80% of traffic. What's the fix?

Introduce a cheap router (Haiku 4.5) to classify request complexity, default routine traffic to Sonnet 5, and reserve Opus 4.8 for an explicit, measured escalation criterion rather than routing everything through the top tier by default.

### Q2. Your escalation tier (Opus 4.8) returns a 429 during a traffic spike. The request currently fails outright. What should happen instead?

Catch the typed RateLimitError and retry the request one tier down (Sonnet 5) rather than surfacing a hard failure - a slightly different answer beats no answer, and the top tier's rate limit shouldn't take down the whole path.

### Q3. You're running a nightly job re-scoring 50,000 stored items with no user waiting on the result. Should this go through live Messages API requests?

No. Route it through the Message Batches API, which offers a 50% cost discount for exactly this kind of latency-insensitive, non-interactive workload.

## FAQ

### Q1. Isn't running a Haiku classification call before every request just adding cost and latency?

The router call is deliberately tiny (max_tokens ~16, effort low), so it's a small fraction of a cent and tens of milliseconds. The savings from routing 80% of traffic away from Opus far outweigh that overhead.

### Q2. How do I decide the actual escalation criteria, not just 'when it feels hard'?

Feed the eval harness (Scenario CCD-F.5) results: track which request patterns fail or score poorly on the workhorse tier, and route exactly those patterns to the escalation tier. This turns the router into a measured policy instead of a guess.

### Q3. Should effort tuning happen before or after considering a bigger model?

Before. Tuning output_config.effort on the current tier is cheaper than jumping a model tier, and often closes the gap - reserve the tier jump for cases where effort tuning alone doesn't get there.

### Q4. Does prompt caching change the router's economics?

Yes - if the same system prompt and tools are shared across all three tiers, caching that shared prefix reduces the cost delta between tiers even further, since the fixed overhead per request drops regardless of which model serves it.

## Production readiness

- [ ] Router classification accuracy periodically spot-checked against human judgment
- [ ] Cost dashboard broken down by tier, trending as routing improves
- [ ] Fallback path tested under simulated rate-limit conditions
- [ ] Escalation criteria documented and reviewed, not tribal knowledge
- [ ] Batches API used for all latency-insensitive bulk workloads

---

**Source:** https://claudearchitectcertification.com/scenarios/model-selection-and-cost-routing
**Last reviewed:** 2026-07-12

**Evidence tiers**, 🟢 official Anthropic doc · 🟡 partial doc / inferred · 🟠 community-derived · 🔴 disputed.