The problem
What the customer needs
- Show text as it's generated, even on turns that also call tools.
- Execute a tool only once its full input has arrived, never on a partial JSON fragment.
- Run independent tool calls concurrently, not one at a time.
Why naive approaches fail
- Calling
json.loads()on eachinput_json_deltachunk as it arrives throws on every partial fragment until the block closes. - Splitting parallel tool_result blocks across multiple user messages trains the model to stop batching tool calls, silently degrading throughput.
- Ignoring
stop_reason: 'pause_turn'on a long server-tool turn truncates the response instead of resuming it.
- Every tool call executes against a complete, successfully parsed JSON input
- Parallel tool_use blocks in one response execute concurrently and return in one user message
- pause_turn resumes automatically without an injected 'Continue' message
- Text renders to the client with no added latency from the tool-handling logic
The system
What each part does
3 components, each owns a concept. Click any card to drill into the underlying primitive.
Delta Accumulator
index-keyed JSON buffer
Buffers input_json_delta fragments per content-block index. Never attempts to parse a tool's input until that block's content_block_stop event has fired.
Configuration
buffers[event.index] += event.delta.partial_json on each input_json_delta; JSON.parse(buffers[index]) only at content_block_stop.
Tool Executor
concurrent - single-batch return
Runs every completed tool_use block from a response concurrently, then returns all tool_result blocks in one user message so parallel tool use keeps working on the next turn.
Configuration
results = await Promise.all(toolUses.map(execute)); messages.push({role: 'user', content: results}) - one message, every result.
Loop Controller
stop_reason dispatcher
Re-opens a new stream after tool_use, and resumes automatically (no injected user text) after pause_turn by resending the accumulated assistant + user history.
Configuration
if stop_reason 'tool_use': execute + continue. if stop_reason 'pause_turn': resend messages as-is, no extra prompt.
Data flow
5 steps to production
Define a small, precisely-described tool set
Tool count and description quality drive routing accuracy under streaming just as much as in a non-streaming call. Keep the set small and each description explicit about when to call it.
tools = [
{
"name": "lookup_order",
"description": (
"Look up an order by order_id. Use this when the user asks about "
"order status, shipping, or contents. Returns 404-shaped error if "
"order_id is not found."
),
"input_schema": {
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"],
},
},
]Accumulate tool input keyed by content-block index
A response can interleave multiple content blocks. Buffer input_json_delta text per index and only parse it once that block's content_block_stop event arrives.
def route_stream_events(stream):
buffers = {}
tool_uses = {}
for event in stream:
if event.type == "content_block_start" and event.content_block.type == "tool_use":
tool_uses[event.index] = {"id": event.content_block.id, "name": event.content_block.name}
buffers[event.index] = ""
elif event.type == "content_block_delta" and event.delta.type == "input_json_delta":
buffers[event.index] += event.delta.partial_json
elif event.type == "content_block_delta" and event.delta.type == "text_delta":
yield ("text", event.delta.text)
elif event.type == "content_block_stop" and event.index in tool_uses:
tool_uses[event.index]["input"] = json.loads(buffers[event.index])
yield ("tool_use", tool_uses[event.index])Execute completed tool calls concurrently, return as one batch
Every tool_use block gathered from a single response executes concurrently. All of their results go back in exactly one user message, preserving Claude's ability to keep issuing parallel calls.
async def execute_and_continue(messages, tool_calls, response_content):
results = await asyncio.gather(*(execute_tool(t) for t in tool_calls))
tool_results = [
{"type": "tool_result", "tool_use_id": t["id"], "content": r}
for t, r in zip(tool_calls, results)
]
messages.append({"role": "assistant", "content": response_content})
messages.append({"role": "user", "content": tool_results}) # single batch
return messagesResume on pause_turn without inventing a user message
A long server-tool turn can stop with pause_turn. Resend the trailing assistant response as-is; the API detects the pending server_tool_use and resumes automatically.
def handle_pause_turn(messages, response):
messages.append({"role": "assistant", "content": response.content})
# Do NOT append an extra "Continue." user message - just re-request.
return client.messages.create(model="claude-sonnet-5", max_tokens=4096,
tools=tools, messages=messages)Drive the full loop, capping iterations
Wrap steps 2-4 in a loop that opens a new stream after every tool_use turn and terminates on end_turn, with an iteration cap as a safety net (not the primary control).
async def run_streaming_tool_loop(user_text, max_iter=10):
messages = [{"role": "user", "content": user_text}]
for _ in range(max_iter):
response = None
tool_calls = []
async for kind, payload in stream_and_route(messages):
if kind == "text":
send_to_client(payload)
elif kind == "tool_use":
tool_calls.append(payload)
elif kind == "final":
response = payload
if response.stop_reason == "end_turn":
return response
if response.stop_reason == "tool_use":
messages = await execute_and_continue(messages, tool_calls, response.content)
continue
if response.stop_reason == "pause_turn":
response = await handle_pause_turn(messages, response)
continue
return {"status": "iteration_cap"}The four decisions
| Decision | Right answer | Wrong answer | Why |
|---|---|---|---|
| Where to buffer tool input JSON | keyed by content-block index, parsed only at content_block_stop | parse each input_json_delta chunk immediately | Fragments are not valid JSON on their own; interleaved blocks corrupt a single shared buffer. |
| Returning parallel tool results | all tool_result blocks in one user message | one user message per tool_result | Splitting results across messages silently trains the model to stop issuing parallel tool calls. |
| Handling pause_turn | resend the assistant turn as-is, no injected text | append a 'Continue.' user message | The API detects the trailing server_tool_use block and resumes on its own; extra text just adds noise. |
| Manual loop vs Tool Runner | manual loop for full control over streaming + tool execution ordering | always default to the beta Tool Runner | The Tool Runner is convenient but its streaming variant does not auto-resume pause_turn as of current SDK versions; a manual loop handles it explicitly. |
Where it breaks
3 failure pairs. Each maps to an exam-style question - the naive move on the left, the disciplined fix on the right.
Code calls json.loads() on every input_json_delta chunk as it streams in.
Buffer fragments per content-block index; parse only once content_block_stop fires for that index.
Each tool_result is sent back in its own separate user message.
CCDF-06Batch every tool_result from one response into a single user message.
Loop returns to the caller on stop_reason == 'pause_turn', truncating the answer.
Resend the assistant turn as-is to resume; only end_turn (or the iteration cap) ends the loop.
Cost & latency
Growing history across two tool round-trips before end_turn; tool defs cached after the first turn.
New stream open + tool execution time; concurrent execution keeps this from multiplying per tool.
Ship checklist
Two passes. Build-time gates verify the code; run-time gates verify the system in production.
Build-time
- Tool input buffered by content-block index, parsed only at content_block_stop↗ streaming
- Parallel tool_use blocks executed concurrently↗ tool-calling
- All tool_result blocks for a turn returned in a single user message
- pause_turn resumes without an injected user message↗ agentic-loops
- Loop has an iteration cap as a safety net, not the primary control
- Text deltas render to the client with no added latency from tool routing
Run-time
- Test with a tool whose input JSON exceeds one SSE chunk (forces multi-fragment accumulation)
- Test two independent tools called in parallel in one response
- Test pause_turn resumption end-to-end with a long server-tool run
- Alert on iteration-cap hits as a routing/description quality signal
- Verify tool_result batching survives a code review checklist, not just a manual test
Five exam-pattern questions
Your streaming handler calls `JSON.parse()` on every `input_json_delta` fragment and throws on most of them. What's the fix?
index and parse only once that block's content_block_stop event has fired. A single input_json_delta chunk is not valid JSON on its own.After adding a second tool, the model stops issuing parallel tool calls even though the tools are independent. You send each tool_result back in its own user message. Why does this matter?
tool_result blocks across multiple messages silently trains Claude to stop batching tool calls in one response. Return every tool_result from a turn in a single user message.A long research turn using server-side web search stops with `stop_reason: 'pause_turn'`. What should the client do?
server_tool_use block and resumes automatically.Frequently asked
Should I use the SDK's Tool Runner instead of a manual loop for streaming + tools?
pause_turn in current SDK releases - it ends the loop and returns the paused message. A manual loop handles this explicitly, which is why this scenario builds one.Do I need to stream tool_use content blocks to the UI at all?
text blocks to the UI for perceived responsiveness; tool_use blocks are backend bookkeeping. Some UIs show a 'using tool X' indicator, which you can emit as soon as content_block_start reports a tool_use block, before its input is even complete.What happens if two tools are called in the same response but one fails?
tool_result with is_error: true and an informative message for the failed one, alongside normal results for the others, all in the same batched user message. Claude will typically retry differently or explain the failure rather than silently proceeding.