What this system is, and its 5 parts
Think of this as Claude Code working inside your CI without anyone watching. It runs every time a pull request opens, reviews each changed file in isolation, and posts inline comments back on the PR. No interactive prompts, no human at a keyboard. The agent is invoked as a one-shot command (claude -p), it returns a structured JSON verdict, and the workflow turns that JSON into the comments your team actually reads. The whole point is that PR review at scale needs a reviewer that does not get tired by file 14, and a CI-native agent that runs per file is exactly that.
GitHub Actions Workflow
.github/workflows/claude.ymlTriggers on pull_request events. Authenticates with the Claude Code GitHub App. Loops over the changed files (via gh pr diff --name-only) and dispatches a per-file claude -p invocation. Owns retries, concurrency, and the eventual gh pr review post.
Configurationon: pull_request. Steps: actions/checkout → install Claude Code CLI → for each changed file, run claude review pr -p --output-format json --custom-instructions .github/claude-context.md. Concurrency: max 4 parallel files; the rest queue.Concept: claude-md-hierarchy →
claude -p Headless Invocation
non-interactive, one-shot per fileRuns Claude Code in headless mode. The -p flag disables the interactive REPL; the agent processes a single task, emits output to stdout, and exits. No human at a keyboard, no waiting on prompts. This is the CI primitive. Without -p, the workflow hangs.
Configurationclaude review pr -p --output-format json --custom-instructions ".github/claude-context.md" --files "src/auth/login.ts" --max-turns 6Concept: context-window →
Per-File Session Isolation
one claude -p invocation per changed fileEach changed file gets its OWN headless session. No shared context across files. This is the single biggest architectural decision: 14 isolated sessions × small context > 1 session × 14 files of accumulated context. The latter triggers lost-in-the-middle by file 8-10; the former does not.
ConfigurationLoop in workflow YAML: for f in $(gh pr diff --name-only); do claude review pr -p --files $f >> review-$f.json; done. Each file's review is independent; the parent workflow aggregates JSON.Concept: subagents →
Structured JSON Output Contract
--output-format json + jq parsingClaude emits a structured object per file: { file, verdict (approve | request_changes | comment), issues: [{ line, severity, message, suggestion? }], summary }. The workflow's next step parses with jq and posts inline comments via gh pr review --comment-line. No regex parsing of free-form prose.
ConfigurationSchema (per file): { file: string, verdict: 'approve'|'request_changes'|'comment', issues: [{line: int, severity: 'blocker'|'nit'|'praise', message: string, suggestion?: string}], summary: string }Concept: structured-outputs →PR Comment Poster + Allowed-Tools Gate
gh pr review + explicit tool whitelistFinal workflow step reads the aggregated JSON, runs gh pr review --request-changes --body $SUMMARY and gh pr review --comment-line N $MSG per issue. Critical: the claude -p invocation declares --allowed-tools Read,Grep,Glob,Bash (no wildcard, no Edit, no Write). CI agents never need write access to the repo; they read and report.
Configuration--allowed-tools "Read,Grep,Glob,Bash(git diff,git log,gh pr diff)". Wildcards in CI are a red flag. They expand the blast radius of any prompt-injection in PR content.Concept: tool-calling →
The problem
- PR review on every pull request without a human running the CLI manually. Claude Code triggered by GitHub Actions on pull_request.
- Per-file feedback that doesn't lose track of conventions established earlier in the diff. File 14 must agree with file 3 on style and tests.
- Structured output the workflow can parse into PR review comments, not free-form prose with brittle regex.
- Single session for all 14 files → context pollutes → file 14 contradicts file 3 (lost-in-the-middle).
- Forgetting -p → workflow hangs waiting for interactive input → CI timeout after 6 hours, no review posted.
- Free-form text output → next step uses regex to extract issues → brittle, breaks on every Claude wording change.
- ✓ Per-file PR review fires on every pull_request event
- ✓ Each file reviewed in its own claude -p session (isolated context)
- ✓ Output is JSON (--output-format json); workflow parses with jq, posts comments via gh CLI
- ✓ allowed_tools is an explicit list in YAML (no * wildcard)
- ✓ Project context flows in via custom_instructions reading .github/claude-context.md
One run, traced end to end
8 steps to production
Scaffold the GitHub Actions workflow
Create .github/workflows/claude.yml. Trigger on pull_request events. Install the Claude Code GitHub App via /install-github-app (one-time, generates the OAuth token stored as secrets.CLAUDE_CODE_OAUTH_TOKEN). Checkout the PR head ref, install the CLI, then dispatch the per-file review loop.
Concept: claude-md-hierarchy →# .github/workflows/claude.yml
name: Claude PR review
on:
pull_request:
types: [opened, synchronize]
jobs:
review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2
- name: Install Claude Code CLI
run: npm i -g @anthropic-ai/claude-code
- name: Per-file review
env:
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
GH_TOKEN: ${{ github.token }}
run: ./.github/scripts/per-file-review.shRun claude -p with --output-format json per file
Loop over the changed files (gh pr diff --name-only). For each one, run claude review in headless mode (-p), with --output-format json, --allowed-tools explicit, and --custom-instructions pointing at a markdown file that holds the project's CLAUDE.md context. Capture each file's JSON output to disk; aggregate at the end.
Concept: structured-outputs →Inject project context via custom_instructions
Don't pass the entire project CLAUDE.md to claude -p. Too much. Instead, create .github/claude-context.md (committed to the repo) with the CI-relevant slice: stack, code style, what counts as a blocker vs a nit, what to skip (generated files, lockfiles). The --custom-instructions flag injects this into every per-file session.
Concept: claude-md-hierarchy →Lock allowed_tools. No wildcards in CI
In CI, the agent processes content from PR authors. Including authors outside your org. That content is untrusted. Wildcard --allowed-tools '*' lets a prompt-injection in a PR body or commit message escalate to write access on your repo. Always declare an explicit list: Read, Grep, Glob, and a NARROW Bash whitelist. Never Edit, Write, or open-ended Bash.
Concept: tool-calling →Parse JSON and post inline PR comments
Aggregate the per-file JSON outputs, then transform into gh pr review calls. One inline comment per issue (line + body), one summary review at the end (approve / request_changes / comment based on whether any blockers fired). The gh CLI handles the GitHub REST mechanics; you just feed it structured input.
Concept: structured-outputs →Cap concurrency and add a per-PR cost budget
Per-file fan-out is parallel by default. But uncapped parallelism can exhaust the GitHub Actions concurrent-job limit and stack up token spend. Cap at ~4 parallel files. Add a per-PR token budget (env var checked at the start of each file) that aborts further reviews if the running PR would exceed the cap. Cost predictability beats marginal latency wins.
Concept: context-window →Use Batch API for nightly audits, sync API for blocking review
PR review is blocking. The developer is waiting; sync API is the right call. But you also want a nightly audit pass (drift detection, security regression scan) that doesn't need to finish in minutes. That's where the Batch API earns its 50% discount: submit overnight, results in 24h, review the next morning. Two different APIs for two different latency budgets.
Concept: batch-api →Add a CI cost-guard hook + alerting
Wrap the claude -p invocation in a PreToolUse hook that aborts the review if the PR is over a token-budget threshold (e.g. >100K tokens of diff). This protects you from a runaway 10-million-line PR exhausting your monthly Claude budget in one CI run. Pair with a workflow alert to a Slack channel when the hook fires.
Concept: hooks →9 decisions the exam turns into distractors
One claude -p session reviewing all 14 files together
Per-file independent claude -p sessions, aggregated at the end
Free-form prose, parsed downstream with regex
--output-format json (parsed with jq → gh pr review)
Wildcard --allowed-tools '*' or unscoped Bash
Explicit --allowed-tools list (Read, Grep, Glob, narrow Bash)
Use one API for both
Sync API for blocking pre-merge; Batch API for non-blocking overnight
Workflow runs ONE claude -p over all 14 changed files. By file 14, lost-in-the-middle has dropped the conventions established on file 3. Inline comments on file 14 contradict the comments on file 3.
Per-file independent sessions. Loop the 14 files, run one claude -p invocation per file, aggregate the JSON outputs at the end. Each file gets a fresh, focused context.
Workflow invokes claude review without -p. The CLI starts in interactive mode, waits for input, and the GitHub Actions runner times out after 6 hours with no review posted.
Always pass -p for non-interactive headless execution. CI hangs are silent failures; the -p flag is what makes Claude Code CI-safe in the first place.
Workflow asks Claude to 'review the file and post inline comments'. Output is free-form prose. Next step uses regex to extract issues. Every Claude phrasing change breaks the regex; the workflow silently posts nothing.
Always pass --output-format json. The output is a structured contract: { file, verdict, issues[], summary }. Workflow parses with jq and posts via gh pr review --comment-line.
Workflow uses --allowed-tools '*' for convenience. A prompt-injection in a PR description tricks the agent into running Bash(rm -rf .) or writing a malicious file. Repo state corrupted; PR reviewer can't tell what happened.
Always declare an explicit allowed-tools list: Read,Grep,Glob,Bash(git diff,git log,gh pr diff). CI agents never need write tools. Wildcards expand the prompt-injection blast radius; explicit lists cap it.
claude -p runs without --custom-instructions. Claude doesn't know the project's stack, code style, or what counts as a blocker. Reviews are generic; flags style-correct code as 'should use named exports' in a default-exports codebase.
Commit .github/claude-context.md with the CI-relevant slice of CLAUDE.md (stack, severity rubric, files to skip). Pass it via --custom-instructions .github/claude-context.md on every claude -p invocation.
Cost & latency
8 files × ~10K input tokens (file diff + context-md) + ~2K output (JSON verdict) ≈ $0.04-0.10 + parallel overhead. Cap at ~$0.15 with the cost-guard hook to bound runaway 50-file PRs.
50 PRs × 8 files × ~5K tokens (audit-only, lighter prompt) at Batch API 50% discount. Result ready next morning; no developer waiting.
.github/claude-context.md is stable across all per-file invocations within a single PR. Mark it cache_control: ephemeral; 5-min TTL keeps it warm across files in one workflow run.
Per-file ~8-15s × 4 parallel = ~30s + JSON aggregation + gh pr review posts. Fast enough that the developer doesn't context-switch away while waiting.
A 200-file PR (rare but real. E.g. lockfile bumps) without the hook would burn ~$3-6 in one workflow run. The hook denies before the loop starts; cost reverts to $0.
Check every gate before release
5 exam-pattern questions
Work through one question at a time, check the architecture, then move through the set.
A GitHub Actions CI pipeline runs Claude Code to review PRs by processing all 14 modified files in a single claude -p session. After 8 files the output becomes repetitive and misses issues, and by file 14 an inline comment contradicts a decision made on file 3. What is the architectural fix?
Frequently asked
Is claude -p the same as the SDK?
Functionally similar; ergonomically different. The SDK is for code that needs programmatic control (custom orchestration, response streaming, tool definitions in code). claude -p is for shell-driven workflows (CI, cron, dev scripts) where the input is a prompt and the output is a structured response. CI workflows almost always want -p; bespoke automation almost always wants the SDK.
What's the maximum file count per PR before this approach breaks down?
No hard limit, but ~30 files is the ergonomic ceiling for blocking pre-merge review. Past that, sync-API latency adds up (~30s × ceil(N/4) parallel). For 30+ file PRs, switch to Batch-API audit (results next morning) and use the sync API only for files that touch security-critical paths (auth/, payments/, infra/). Two-track review.
Can I run claude -p on the same PR every time it's updated?
Yes. That's the synchronize event. The workflow trigger should be on: pull_request: types: [opened, synchronize]. Synchronize fires on every push to the PR head ref. Idempotency: each run reviews the *current* diff, so old comments stay until they're stale; if you want to dismiss outdated reviews, add a gh pr review --dismiss step that targets reviews on commits no longer at HEAD.
How do I keep the cost predictable if the team merges 100+ PRs a day?
Three levers, in priority order: (1) PreToolUse cost-guard hook that denies any single PR over a token budget. Protects from outliers; (2) --max-turns capped at 4-6. Bounds the worst case per file; (3) Concurrency cap (xargs -P 4 or JS semaphore). Bounds parallel Claude API calls in flight. With all three, monthly cost variance stays inside ±10%.
Should the CI workflow have write access to the repo?
No. Read-only on the repo, write-only on PR comments and reviews. GitHub Actions permissions: block: contents: read, pull-requests: write. The CI agent reads code, runs gh pr diff, posts comments. It never needs to push commits. Same logic that bans write tools in --allowed-tools applies at the GitHub permission layer.
What's in .github/claude-context.md vs the project's main CLAUDE.md?
The CI slice, not the whole thing. Main CLAUDE.md targets developers running Claude Code interactively. It covers full stack, conventions, examples, troubleshooting (~300-500 lines). .github/claude-context.md is the trimmed CI rubric: stack one-liner, severity definitions, files to skip, output-format reminder (~40-80 lines). Smaller context = faster review + cheaper tokens.
Can I run claude -p reviews with custom Skills?
Yes, and you should. Create .claude/skills/code-reviewer/SKILL.md with the team's review rubric, allowed tools, and output schema. The CI workflow invokes the Skill explicitly: claude review pr -p --skill code-reviewer .... Skills are version-controlled (live in the repo), so the rubric evolves with the codebase and CI behaviour stays in sync.
