D4.10 · Domain 4 · Prompt Engineering · 20% of CCA-F

Multi-Instance & Multi-Pass Review Architectures.

5 min read·7 sections·Tier A

A multi-pass review architecture runs more than one independent Claude pass over the same artifact and reconciles the results, instead of trusting a single evaluation - it covers two related patterns: the evaluator-optimizer loop (generate, then a separate evaluator critiques, repeat) and fleet-style parallel review (many independent reviewer instances, each verifying its own findings, reconciled before a human sees them). Reconciliation happens by independently reproducing and verifying each finding, not by majority vote. Anthropic: code-review blog

Anthropic production dataDomain 4CCA-F
Multi-Instance & Multi-Pass Review Architectures, hero illustration featuring Loop mascot in a warm gallery scene.
Domain D4Prompt Engineering · 20%
On this page
01 · Summary

TLDR

A multi-pass review architecture runs more than one independent Claude pass over the same artifact and reconciles the results, instead of trusting a single evaluation - it covers two related patterns: the evaluator-optimizer loop (generate, then a separate evaluator critiques, repeat) and fleet-style parallel review (many independent reviewer instances, each verifying its own findings, reconciled before a human sees them). Reconciliation happens by independently reproducing and verifying each finding, not by majority vote. Anthropic: code-review blog

3
Review tiers (Claude Code)
D4
Exam domain (CCA-F)
16% -> 54%
PR review-comment lift
<1%
Findings marked incorrect
84% (avg 7.5)
Large-PR finding rate
02 · Definition

What it is

A multi-pass (or multi-instance) review architecture runs more than one independent Claude pass over the same artifact, code, a plan, a generated output, and reconciles the results, rather than trusting a single evaluation. This spans two related but distinct patterns: the evaluator-optimizer workflow (a generator LLM and a separate evaluator LLM loop until a quality bar is met) and fleet-style parallel review (many independent reviewer instances explore the same change in parallel, each verifying its own findings, aggregated and ranked before a human sees them).

Anthropic runs this pattern in production on its own pull requests via Code Review (and its deeper ultrareview mode), not as a research demo, but as a default review layer on Anthropic's own codebase. The two patterns solve different problems: evaluator-optimizer improves a single generated artifact against explicit criteria through iteration; fleet review increases confidence in a judgment (did this diff introduce a bug) by decorrelating the reviewers.

03 · Mechanics

How it works

Evaluator-optimizer is a two-role loop. One LLM call generates a response while another provides evaluation and feedback, iterating until the bar is met. It is the right fit when there are clear evaluation criteria and iterative refinement demonstrably improves the result - both when a human could articulate useful feedback and when the LLM can generate that same feedback itself.

Structurally, multi-pass review runs the same task many times, which is the shape of parallelization's "voting" variant (Anthropic's workflow taxonomy contrasts it with sectioning, which splits one task into independent parallel subtasks). That shape is not the same thing as the reconciliation mechanism: Anthropic's canonical description of voting aggregates redundant attempts for higher-confidence output, which reads as consensus/majority-style aggregation - but that generic description is not how production multi-pass review architectures actually resolve disagreement, see below.

Claude Code's review stack has three depths. /review is a fast, single-pass, read-only PR review. /code-review (or /code-review <pr#>) runs a multi-agent review of a diff and can apply fixes with --fix. /code-review ultra (alias /ultrareview) launches a fleet of reviewer agents in a remote sandbox, where every reported finding is independently reproduced and verified before it ever surfaces, prioritizing real bugs over style nitpicks.

Reconciliation is verification, not majority-vote aggregation. Rather than resolving disagreement across passes by counting votes or taking a consensus, Anthropic's production design has each finding independently reproduced and verified before it is reported - a finding stands or falls on proof, regardless of how many passes did or didn't flag it. Internally this raised the share of PRs receiving substantive review comments from 16% to 54%; on PRs over 1,000 lines, 84% get findings (avg. 7.5) versus 31% on small PRs (avg. 0.5); fewer than 1% of findings are marked incorrect by engineers. It does not auto-approve, merge stays a human call.

The pattern generalizes past code review. The same "many independent parallel agents, condense and reconcile before reporting up" shape appears in Anthropic's multi-agent Research system (subagents explore independently, compress findings for a lead) and in the C-compiler project, where automated tests served as the ground-truth reconciliation mechanism across parallel agents' work.

Multi-Instance & Multi-Pass Review Architectures mechanics, painterly diagram featuring Loop mascot.
04 · In production

Where you'll see it

Anthropic's own PR review (dogfooding)

Code Review dispatches a team of agents per PR, verifies each finding before surfacing it, and raised substantive-comment coverage from 16% to 54% internally.

Iterative report or content generation

An evaluator-optimizer loop checks a drafted report against a fixed rubric and feeds back critique until the quality bar is met, before a human ever sees it.

05 · Compare

Side-by-side

PatternMechanismReconciliationBest for
/reviewSingle-pass, read-only PR reviewNone - one opinionSmall or trivial diffs, a fast check
/code-reviewMulti-agent review of a diff, can --fixMultiple agents on one diff, findings pooledEveryday PRs that need substantive review
/code-review ultra (ultrareview)Fleet of reviewer agents in a remote sandboxEach finding independently reproduced and verified before surfacingLarge or security-sensitive changes (auth, encryption keys)
Evaluator-optimizerGenerator LLM + separate evaluator LLM loopEvaluator feedback loop until a quality bar is metIterative content generation against a rubric, not diff review
06 · On the exam

Question patterns

Multi-Instance & Multi-Pass Review Architectures exam trap, painterly cautionary scene featuring Loop mascot.
A one-line typo-fix PR triggers `/code-review ultra` by default. What is wrong with this default?
Fleet review is overkill for a trivial diff - scale agent count and depth to PR size and risk. Named distractor: "ultra review is always the safest default" - it wastes cost without improving signal on a change this small; /review (fast single-pass) is the right fit.
A generated compliance report needs to hit a fixed rubric before delivery, and no diff or PR is involved. Which pattern fits, not fleet code review?
The evaluator-optimizer workflow: a generator LLM produces the report while a separate evaluator LLM checks it against the rubric and feeds back critique in a loop. Named distractor: "spin up N parallel reviewer instances and vote" - that is the wrong pattern outside a diff-review context and has no explicit rubric-checking loop.
Two independent reviewer instances flag conflicting severities for the same finding on a PR. How does Anthropic's production Code Review architecture resolve the disagreement?
Not by majority vote - each finding is independently reproduced and verified before it is ever surfaced, so disagreement is resolved by proof, not by counting votes. Named distractor: "take whichever pass has the higher-severity vote" - that describes a voting or consensus scheme Anthropic's design explicitly does not use here.
A large, security-sensitive auth refactor (roughly 1,200 lines) gets the fast single-pass `/review` to save time before merge. What is the risk?
Large, high-stakes diffs are exactly where fleet review earns its cost - Anthropic cites a one-line auth change that looked routine but would have broken production authentication, caught only by a fleet of independently-verifying reviewer agents. Named distractor: "single-pass review is fine as long as a human also reads the diff" - on large or complex diffs a single pass has much lower substantive-comment coverage than fleet review (the 16% vs 54% lift Anthropic reports).
A team assumes running the identical review prompt three times and picking the majority answer counts as a "multi-pass review architecture." What is missing?
Independence and an explicit reconciliation step. Re-running the same prompt without diversifying the pass (different agents or instances exploring independently) and without a verification-before-reporting step just correlates the same blind spot three times. Named distractor: "running the same prompt more than once is sufficient for multi-pass review" - true multi-pass requires independent passes plus reconciliation (verification or evaluator feedback), not repetition alone.
07 · FAQ

Frequently asked

Is evaluator-optimizer the same as fleet code review?
No. Evaluator-optimizer is a generator and evaluator loop for iterative refinement against criteria, such as a rubric; fleet review runs many independent reviewer instances over the same artifact once and reconciles via verification, not iterative regeneration.
Does fleet review auto-approve or auto-merge?
No. Anthropic's Code Review surfaces one overview comment plus inline findings; merge approval stays a human call.
08 · Practice with AI

Work this with your AI

Work this concept hands-on with Claude Code, Codex, or claude.ai. Copy a prompt, paste it into your assistant, and practise in tandem. Each one keeps you active (explain it back, get drilled, or build) rather than just reading.

  • Drill it like the exam (scenario MCQs)
    Practice in the exam's scenario-MCQ format with trap awareness.
  • Explain it back (Feynman)
    Build durable, transferable understanding of a concept you can half-state.
  • Test me, adapting the difficulty
    Active recall practice on a concept you think you know.
  • Check my prerequisites first
    Before studying a concept that keeps not sticking.
  • Find the high-leverage 20%
    When a domain feels too big and you are short on time.
Self-check

Test yourself

Three diagnostic questions on this primitive. Reveal each answer when you have a guess. Want a full 60-question mock? Open the mock hub →

Q1A one-line typo-fix PR triggers `/code-review ultra` by default. What is wrong with this default?
Fleet review is overkill for a trivial diff - scale agent count and depth to PR size and risk. Named distractor: "ultra review is always the safest default" - it wastes cost without improving signal on a change this small; /review (fast single-pass) is the right fit.
Q2A generated compliance report needs to hit a fixed rubric before delivery, and no diff or PR is involved. Which pattern fits, not fleet code review?
The evaluator-optimizer workflow: a generator LLM produces the report while a separate evaluator LLM checks it against the rubric and feeds back critique in a loop. Named distractor: "spin up N parallel reviewer instances and vote" - that is the wrong pattern outside a diff-review context and has no explicit rubric-checking loop.
Q3Two independent reviewer instances flag conflicting severities for the same finding on a PR. How does Anthropic's production Code Review architecture resolve the disagreement?
Not by majority vote - each finding is independently reproduced and verified before it is ever surfaced, so disagreement is resolved by proof, not by counting votes. Named distractor: "take whichever pass has the higher-severity vote" - that describes a voting or consensus scheme Anthropic's design explicitly does not use here.
Last reviewed: 2026-05-04·Refresh cadence: monthly
D4.10 · D4 · Prompt Engineering

Multi-Instance & Multi-Pass Review Architectures, complete.

You've covered the full ten-section breakdown for this primitive, definition, mechanics, code, false positives, comparison, decision tree, exam patterns, and FAQ. One technical primitive down on the path to CCA-F.

More platforms →