What this system is, and its 5 parts
Think of this as the long-running version of the support agent. The one that keeps making sense on turn 15 just like it did on turn 1. When a customer comes back five messages later, this agent still knows who they are, what they decided, and what they explicitly asked you not to do. It exists because most production conversations are not three turns; they are fifteen, and a system that quietly forgets the customer ID by turn eight is worse than no system at all. Everything below is how to make a conversation continue feeling like a conversation, even after the underlying context has been compressed.
Case-Facts Block
immutable customer state, top of promptPinned at the very top of every system-prompt iteration. Holds customer_id, decision_made, contact_preference, escalation_requested, policy_cap. Survives history compression and is re-read every turn. That is the entire point.
Configurationsystem: f"CASE_FACTS:\n customer={cust_id} · decision={decision} · contact={pref} · escalated={escalated}". Updated by hooks after any state-changing tool call. Never paraphrased; always exact.Concept: case-facts-block →Session State Manager
decisions + flags between turnsTracks the structured state that case-facts cannot: which clarification questions have been answered, which tool results are still in play, whether the customer has explicitly asked for a human. Updated post-each-tool-call. Read by the hook before any subsequent tool dispatch.
Configurationstate: {clarifications_answered: [order_id, refund_or_credit], last_tool_result, escalation_requested: false, contact_preference: 'email'}. Persisted in session store, loaded into prompt as a serialized block.Concept: session-state →History Summarizer
turns 2-14 → 3 lines at turn 15Watches conversation length. When the message list exceeds 15 entries, replaces turns 2-N-1 with a single 3-line summary preserving decisions, not transcripts. Case-facts stays untouched at the prompt top. The summary lives in the message list, not in case-facts.
Configurationif len(messages) > 15: summary = compress_to_3_lines(messages[1:-1]); messages = [messages[0], {role: 'user', content: summary}, messages[-1]]. Keeps token count flat while preserving decision continuity.Concept: context-window →Clarification Gate Hook
PreToolUse · prerequisite blockSits between Claude's tool_use request and tool execution. If a downstream tool needs verified_id and case-facts.verified_id is null, exits 2 with a deterministic message routing Claude to call get_customer first. This is the difference between probabilistic prompt language and 100% prerequisite enforcement.
ConfigurationHook fires before process_refund / update_account / escalate_to_human. Reads case_facts.verified_id and conversation flags. Exit 2 with stderr message routes Claude back; no leakage, no exceptions.Concept: hooks →
Stop-Reason Loop Control
branch on the field, not the textReads stop_reason after every API response. end_turn → exit cleanly. tool_use → execute, append result, continue. max_tokens → save partial state and escalate (never silently truncate). Never branches on response text containing 'done' or 'goodbye'.
Configurationwhile True: resp = client.messages.create(...). if resp.stop_reason "end_turn": return. if resp.stop_reason "tool_use": dispatch + append. if resp.stop_reason "max_tokens": persist + escalate.Concept: agentic-loops →
The problem
- Pick up the conversation at turn 15 and still see the customer ID, decision, and contact preference from turn 2.
- Be honored immediately when they say 'I want to speak to a human'. No negotiation, no 'let me try first'.
- Not be re-asked the same clarifying question three turns after they already answered it.
- Single-block message history hits lost-in-the-middle by turn 9; the agent loses the order ID and re-asks.
- Prompt-only clarification language ('don't repeat questions') leaks 8% of cases; the agent re-asks anyway.
- Sentiment-triggered escalation creates 50% false positives: angry-but-valid customers get escalated unnecessarily.
- ✓ Turn-15 retrieval of customer_id + prior decision = 100% (case-facts pinned, never summarized)
- ✓ Repeat-clarification rate < 1% (programmatic prerequisite block, not prompt language)
- ✓ False-escalation rate < 5% (policy-gap + explicit-request triggers only, sentiment ignored)
- ✓ p95 turn latency < 5s including hook overhead and history compression
One run, traced end to end
8 steps to production
Define the case-facts anchor block
Pin the immutable customer facts at the very top of the system prompt. These survive compression, are re-read every turn, and are never paraphrased. The block is the single load-bearing pattern of the whole scenario. Get this wrong and turn 15 forgets turn 2.
Concept: case-facts-block →from anthropic import Anthropic
client = Anthropic()
def build_system_prompt(case_facts: dict) -> str:
return f"""You are a conversational support agent.
CASE_FACTS (immutable; re-read every turn; never paraphrased):
- customer_id: {case_facts['customer_id']}
- decision_made: {case_facts.get('decision_made', 'none')}
- contact_preference: {case_facts.get('contact_preference', 'unset')}
- escalation_requested: {case_facts.get('escalation_requested', False)}
- policy_cap: ${case_facts.get('cap', 500)}
Constraints:
- Branch on stop_reason. Never on response text.
- If escalation_requested is True: route to human queue, no negotiation.
- If a clarifying question was already answered, do not re-ask (state below)."""Build the session-state structure
Case-facts holds immutable customer state; session-state holds the conversational state. Answered clarifications, last tool result, escalation flag. Together they replace the lost-in-the-middle problem with structural retrieval. Loaded into the prompt as a serialized block right after CASE_FACTS.
Concept: session-state →Wire the PreToolUse clarification hook
Programmatic prerequisite enforcement. Before any account-modifying tool, the hook checks case-facts + session-state. Missing prerequisites → exit 2 with a structured stderr message; Claude reads it and routes to the prerequisite tool first. Prompt language alone leaks 8%; this hook is 100%.
Concept: hooks →Run the loop on stop_reason, not text
The single most-tested distractor in this scenario is parsing response text for 'done'. Claude can return text + tool_use in the same message; the structured stop_reason field is the only authoritative termination signal. Branch on it. Always.
Concept: agentic-loops →Compress conversation history at turn 15
When the message list exceeds 15 entries, replace turns 2 through N-1 with a single summary that preserves decisions, not transcripts. Case-facts stays at the prompt top, untouched. The summary lives in the message list. This frees ~40% of tokens with zero decision loss.
Concept: context-window →Honor explicit human-handoff requests immediately
When the customer says 'speak to a human', the agent does not negotiate. The hook flips session_state.escalation_requested → true. The next tool dispatch is escalate_to_human; everything else is blocked. Sentiment is orthogonal. Angry customers with valid requests still get the answer first.
Concept: escalation →Cache the system prompt + tools
System prompt + tool definitions are stable across turns; only case-facts and session-state change. Mark the stable parts with cache_control: ephemeral and pay ~90% less for those bytes on every turn after the first. With 5-min TTL on continuous traffic, hit rate stays above 70%.
Concept: prompt-caching →Audit-log the conversation arc
Every closed conversation writes a structured row: customer_id, turn_count, tool_calls_in_order, escalation_reason (if any), elapsed_ms_total, csat. Skip the full transcript. The structured trace is enough to replay any failure and is 50× smaller. Store for 90 days minimum.
Concept: evaluation →9 decisions the exam turns into distractors
progressive summarization of customer_id + amount
case-facts block at top of prompt + session-state block
negotiate ('let me try once more') or suggest alternatives
set escalation_requested → block all tools except escalate_to_human
keep all messages OR summarize the case-facts block
compress turns 2 through N-1 into 3 lines at turn 15; case-facts stays untouched
escalate on negative sentiment
process the request normally; sentiment does not trigger escalation
By turn 9, agent has summarized cust_4711's order ID to 'a recent order'. Treats turn 10 as a new conversation.
Pin CASE_FACTS at top of system prompt. Re-read every turn. Never paraphrased. Compression only touches the message list, never the case-facts block.
System prompt says 'do not re-ask answered questions'. Agent re-asks 'which order?' on turn 4 and turn 8. 8% leakage.
Track answered clarifications in session_state. PreToolUse hook checks state and blocks downstream tools if a prerequisite clarification is unanswered. Deterministic, not probabilistic.
50 turns fill the context window. Lost-in-the-middle effect drops the order ID. Agent makes contradictory recommendations.
At turn 15, summarize turns 2 through N-1 into 3 lines preserving decisions only. Case-facts stays at prompt top. Frees ~40% tokens with zero decision loss.
User said 'I do not want to be contacted by phone' on turn 3. Agent suggests phone callback on turn 9.
Persist session_state with contact_preference + decision flags. Read into prompt every turn alongside case-facts.
Angry customer with a valid refund request is escalated because tone is negative. 50% false-positive rate, customers learn that anger = faster service.
Escalation triggers only on (a) policy gap, (b) tool limit, (c) explicit user request. Sentiment is logged for reporting but never gates escalation.
Cost & latency
12 avg turns × (cached system + tools + dynamic case-facts/session-state + accumulating history). Cache hits ~70% on stable preamble + tools.
Pre-cache: ~$0.05. With ephemeral cache on stable system + tools: ~$0.022. ~55% reduction on long conversations.
Streaming first token in ~150ms. Tool round-trips 1.5-2s each. Average 1.5 tool calls per turn + hook (<100ms) + compose.
12 verbose message blocks (≈8K tokens) → 3-line summary (≈250 tokens). Frees the long-tail conversations from lost-in-the-middle and OOM-on-history.
5-min ephemeral TTL on stable preamble + tool definitions. Continuous chat traffic keeps cache warm. Per-turn case-facts/session-state stays fresh, as it should.
Check every gate before release
5 exam-pattern questions
Work through one question at a time, check the architecture, then move through the set.
By turn 8 of a long conversation, your agent has lost the customer's order ID and refund amount. The agent treats turn 9 as if it were turn 1. What is the architectural fix?
Frequently asked
Why pin case-facts in the system prompt instead of passing them as a tool result?
System prompt content is re-read and weighted highest by the model. Tool results live in the message list, which can be compressed, summarized, or fall victim to lost-in-the-middle as the conversation grows. Case-facts must survive every single turn unchanged, so it lives at the structural top of the prompt. The only place that's immune to summarization and attention drift.
What's the right threshold for triggering history compression?
~15 messages is a defensible default. Below that, full history fits comfortably and the cost of compression isn't justified. Above it, lost-in-the-middle starts degrading recall on details from turns 4-8. Tune per workload: customer-support averages 8-12 turns and rarely needs compression; technical-support can run 30+ and benefits from compressing earlier (turn 10).
If case-facts is immutable, how do I update it when the customer changes their decision?
Case-facts is immutable per-iteration, not immutable forever. After every state-changing tool call (e.g., customer switches from refund to store credit), the harness re-builds case-facts with the updated values and pins the new version on the next turn. Within one turn, it's fixed; between turns, it's the deliberate, hook-controlled write path.
Why a PreToolUse hook for clarification, instead of putting the rule in the prompt?
Prompt-only enforcement is probabilistic (~92% in this scenario). Hooks are deterministic. They read structured state, exit 2, and route Claude back. For business-bearing guarantees (don't re-ask answered questions, don't process unverified accounts), the 8% leak from prompt-only is unacceptable. Use prompts for tone and persona; use hooks for hard guarantees.
Should I cache the case-facts block?
No. Case-facts changes whenever the customer makes a decision or a tool updates state. Caching it kills hit rate. Split the system prompt into a stable preamble (role + constraints, cached with cache_control: ephemeral) and a dynamic block (case-facts + session-state, fresh every turn). You get ~70% hit rate on the cached portion and zero staleness on the dynamic portion.
What if the customer switches topics mid-conversation (refund → tech support)?
Re-route through triage. Push current case-facts (customer_id + decisions made so far) to the new specialist's task string, and either context-switch this agent or spawn a sub-agent for the new intent. Trying to handle multi-intent in a single specialist erodes accuracy and pollutes the case-facts block. The original intent's decisions get tangled with the new intent's evidence.
How do I test that conversation continuity is actually working?
Two-step regression: (1) On turn 1, customer says 'My order is #123, I want a refund.' Verify case-facts is updated. (2) On turn 8, after compression has fired and verbose history is summarized, ask 'Which order was that again?'. The agent must NOT re-ask; it must read case-facts and answer immediately. If it re-asks, your case-facts block isn't being re-read every turn. That's the bug to chase.
What goes in the audit log for a long conversation?
Per closed conversation: customer_id, turn_count, ordered list of tool_calls (just names + timestamps), per-turn stop_reasons, compression_fired_at_turn (if any), escalation_reason (if any), elapsed_ms_total, csat (if surveyed). Skip the full transcript. The structured trace is 50× smaller and replays any failure path. Retain 90 days minimum for production debugging and exam-style retrospectives.