What this deep dive covers
The coordinator owns semantic decomposition (not lexical), enumerates every relevant sub-domain before spawning anything, and dispatches research subagents in parallel via a synchronous-fork-then-join pattern. Coverage gaps live in the decomposition step. Not in the subagents.
What it is
The coordinator is the single hub that receives the user's research query and turns it into a fan-out of self-contained subagent tasks. Its first and most load-bearing job is semantic decomposition: looking at a topic like creative industries and enumerating every sub-domain that matters (visual arts, music, writing, film, performing arts), not just the first one that surfaces from a keyword scan. Lexical decomposition stops at the words you can see in the query. Semantic decomposition asks what the user actually needs covered.
Once decomposition is complete, the coordinator dispatches subtasks via a synchronous-fork-then-join pattern. All N research subagents fire at once through asyncio.gather (Python) or Promise.all (TypeScript). The coordinator awaits the full set, then proceeds to verification. Latency becomes max(subagents) instead of sum(subagents). That parallelism is the architectural payoff and the reason hub-and-spoke beats a single mega-loop on any decomposable task.
Routing is strict hub-and-spoke: subagents never call each other. If subagent B depends on a finding from subagent A, the answer is not for B to import A. The answer is A returns to the coordinator, the coordinator constructs B's task prompt with A's finding embedded, and B starts in a fresh context. Every cross-subagent edge passes through the hub. This is what keeps isolation, parallelism, and visibility intact at the same time.
How it works
Step 1 is decomposition. The coordinator reads the query, identifies the topic class, and produces an explicit list of sub-domain tasks. For impact of AI on creative industries the list must include visual arts, music, writing, film, and performing arts. The decomposition function is unit-testable and should be reviewed before any spawn happens; a coverage bug here cannot be recovered downstream by tuning subagents or upgrading models.
Step 2 is fan-out. Each task becomes a messages.create call with its own system prompt, scoped tools whitelist, and a self-contained messages body. No history is inherited. The coordinator wraps the fan-out with a Semaphore(MAX_PARALLEL=5) to bound concurrency and a per-subagent retry budget (default 2) for timeout handling. stop_reason from each subagent tells the coordinator whether the response is complete, max-tokens partial, or tool-use mid-flight.
Step 3 is join. The coordinator awaits the full gather, inspects each subagent's status_code, and decides per-result: accept, retry with a narrower query, or transparently mark a gap. Successful findings get pooled and handed to the verification subagent. Timeouts return structured error context, not silence: {status: 'timeout', query, partial_results, alternatives}. The coordinator uses that envelope to make a real decision; an empty [] masquerading as success would force it to guess.
The 4 decisions
User asks about creative industries. How do you decompose?
Lexical splits miss every sub-domain that wasn't named. Semantic decomposition asks what the user actually needs covered and enumerates the full set before spawning.
Right answerSemantic enumeration: visual arts, music, writing, film, performing arts
Subagent B needs a finding from Subagent A. How does B get it?
Direct calls break isolation, kill parallelism (B blocks on A even when independent), and hide the dependency from the coordinator's orchestration graph.
Right answerA returns to coordinator. Coordinator embeds A's finding in B's task prompt
Final report is missing 4 of 5 sub-domains. Where do you debug first?
If the coordinator never enumerated music or writing, no subagent could research them. Decomposition is the load-bearing step. Fix it first.
Right answerThe coordinator's decomposition function. The bug is upstream
How many subagents to fan out?
Unbounded fan-out hits API concurrency limits, rate-limit backpressure, and coordinator-side context contention. Cap, measure, tune to your workload.
Right answerBounded by Semaphore(MAX_PARALLEL=5). Diminishing returns past 5-7
9 decisions the exam turns into distractors
Lexical split on the words creative and industries
Semantic enumeration: visual arts, music, writing, film, performing arts
A calls B directly with the finding
A returns to coordinator. Coordinator embeds A's finding in B's task prompt
Tune subagent prompts or upgrade their model
The coordinator's decomposition function. The bug is upstream
Unbounded. Spawn one per sub-domain regardless of count
Bounded by Semaphore(MAX_PARALLEL=5). Diminishing returns past 5-7
Coordinator splits creative industries on the word creative. Spawns one subagent for creative writing. Misses music, film, visual arts, performing arts entirely.
Replace lexical split with a semantic enumerator. For each topic class, list ALL relevant sub-domains in code, not the first one the regex catches.
Coordinator awaits subagent 1 before spawning subagent 2. Latency = sum(subagents) ~ 25s for 5 tasks. Parallel architecture wasted.
Use asyncio.gather(*tasks) (Python) or Promise.all(tasks) (TS). All N subagents fire at once. Latency drops to max(subagents) ~ 5s.
Researcher A imports Researcher B and passes a finding directly. B inherits A's call-site context. Coordinator loses visibility on the dependency.
Route through the coordinator. A returns its finding; coordinator builds B's task prompt with the finding embedded. Hub-and-spoke is non-negotiable.
Coordinator spawns 50 subagents at once for a long taxonomy. Anthropic API rate-limits half of them; retry storms compound the problem.
Wrap fan-out in Semaphore(MAX_PARALLEL=5). Set a retry budget per subagent (default 2). Beyond that, accept partial data and mark the gap.
Web-search subagent times out, returns []. Coordinator interprets empty list as no information exists. Final report has a silent gap with no acknowledgement.
Return {status: 'timeout', query, partial_results, alternatives}. Coordinator inspects status_code, retries narrower or marks the gap transparently in synthesis.
5 exam-pattern questions
Work through one question at a time, check the architecture, then move through the set.
A research system decomposes 'impact of AI on creative industries' into three subtopics: visual arts, music, writing. The web-search subagent finds results for all three. The synthesis subagent produces a report covering only visual arts. Why?
Frequently asked
Should decomposition happen at runtime or be hardcoded?
Hybrid. For known topic classes (creative industries, healthcare, finance), maintain a hardcoded sub-domain map. For novel topics, fall back to a lightweight model-generated decomposition with a generic structure (academic literature, industry case studies, empirical adoption data). Hardcoded is faster and tested. Model-generated is adaptive but needs review.
How does the coordinator know when all subagents are done?
asyncio.gather / Promise.all resolves when every task has either returned or raised. The coordinator inspects each result's stop_reason and status_code to decide accept, retry, or mark a gap. There is no polling. The runtime owns the join.
Can the coordinator update its decomposition mid-run if a subagent surfaces an unexpected sub-domain?
Yes, but only between phases. Phase 1 (research) completes; coordinator inspects the pooled findings; if a missing sub-domain emerges, the coordinator can spawn a follow-up research subagent before verification. Mid-phase decomposition changes break the parallel model and create coordination chaos. Wait for the join.