Long Document Processing.

Walk the document; split at section / paragraph / sentence boundaries (in that precedence). Aim for 500-2000-token chunks; add 10-20% overlap between adjacent chunks so a clause spanning a boundary stays whole in at least one chunk. Each chunk gets a deterministic chunk_id (hash of content) and a page number for citation.

import hashlib
from typing import TypedDict

class Chunk(TypedDict):
    chunk_id: str
    page: int
    text: str

def chunk_document(pages: list[str], target_tokens: int = 1200, overlap: float = 0.15) -> list[Chunk]:
    """Semantic chunking with overlap. Split at section, paragraph, sentence."""
    chunks = []
    buffer = ""
    page_buffer_started = 1

    for page_num, page_text in enumerate(pages, start=1):
        for paragraph in split_into_paragraphs(page_text):
            # If adding this paragraph would exceed target, emit current buffer
            if approx_tokens(buffer + paragraph) > target_tokens and buffer:
                chunks.append({
                    "chunk_id": hashlib.md5(buffer.encode()).hexdigest()[:12],
                    "page": page_buffer_started,
                    "text": buffer.strip(),
                })
                # Carry forward the last 15% as overlap
                tail = buffer[-int(len(buffer) * overlap):]
                buffer = tail + paragraph + "\n\n"
                page_buffer_started = page_num
            else:
                buffer += paragraph + "\n\n"

    if buffer.strip():
        chunks.append({
            "chunk_id": hashlib.md5(buffer.encode()).hexdigest()[:12],
            "page": page_buffer_started,
            "text": buffer.strip(),
        })
    return chunks

def split_into_paragraphs(page_text: str) -> list[str]:
    return [p for p in page_text.split("\n\n") if p.strip()]

def approx_tokens(text: str) -> int:
    return len(text) // 4  # rule of thumb

At ingest time, embed each chunk once with a strong embedding model (Voyage-3 or OpenAI text-embedding-3-small) and store in a vector index. Index by chunk_id; the embedding becomes the search key. Re-embedding only fires on content change (hash-keyed cache). For a 200-page document at ~500 chunks, embedding is a ~$0.05 one-time cost.

When the agent asks a question (e.g., 'what's the indemnification clause?'), embed the question, retrieve the top-K=5 most-similar chunks, and pass ONLY those into context. The full document never enters the prompt. K=5 is the sweet spot. Bigger K dilutes context with marginally-relevant chunks; smaller K misses the right one. Re-rank top-20 with Claude Haiku if precision matters.

Every prompt iteration starts with a CASE_FACTS block: doc_id, extracted_count, last_clause_id, decisions already made. The block is rebuilt from durable state every turn. It survives summarization, model swaps, and session resets. Critically, EXACT VALUES ($247.83, not ~$250; cust_4711, not the customer) stay verbatim. The conversation history below it CAN be summarized; the case-facts cannot.

When stop_reason 'max_tokens', the harness writes the current case-facts + last extracted record + chunk position to a durable store, then starts a fresh session and re-loads the checkpoint as its case-facts. Because case-facts are at the prompt top, the new session continues exactly where the old left off. No data loss; no manual intervention; no need for the agent to even know.

Every extraction tool emits a citations: [{ chunk_id, page }] array; the schema makes citations REQUIRED. The model can't extract a clause without pointing at the chunks that supported it. Downstream consumers (auditors, reviewers, regulators) click any extracted value and see the exact paragraph in the original. Audit-grade provenance, structurally enforced.

When the use case is 'extract every payment clause from 1000 contracts overnight', the Batch API earns its 50% discount. Submit at 6 PM, results ready at 6 AM. No real-time retry inside the batch; failures get resubmitted in the next batch with their specific error in the next message. Combined with prompt caching on the system prompt + tool registry, bulk extraction cost drops 95%+ vs naive sync calls.

Aggregate accuracy hides per-document-type weakness. Stratify by doc_type (MSA vs DPA vs SOW), by clause_type, by page-section (front/middle/back). Surface the worst stratum. Pair with an adversarial test set of 50 documents where the requested clause is GENUINELY absent. The right behaviour is clause_text='not_found' with empty citations, NEVER an invented clause. Hallucinated extractions = audit-fail.