On this page
TLDR
Prompt engineering is a craft of seven techniques that turn a brittle one-shot prompt into a production-grade contract: few-shot examples, iterative refinement against an eval suite, anti-fabrication schemas, structured templates, explicit do-don't constraints, output anchoring via tools, and test-driven prompt edits. The exam trap is treating any one technique as 'the answer'; production prompts compose all seven.
What it is
Prompt engineering is the discipline of shaping Claude's output by writing a prompt, measuring it against test cases, observing where it fails, and tightening the prompt until the eval score plateaus. It is not writing one clever sentence. A production prompt converges over 5-15 iterations against a frozen suite of 20-50 cases, of which 3-5 are known-failure cases that anchor the regression boundary. Anthropic illustrates this workflow in A typical eval workflow: its teaching example compares scores of 7.66 and 8.7 after revising a prompt. These are example scores, not expected gains for every task.
The seven load-bearing techniques are few-shot prompting (showing 2-5 input-output pairs so Claude can pattern-match the harder cases), iterative refinement (the Draft → Test → Observe → Refine → Anchor loop), anti-fabrication patterns (nullable fields, unclear enum values, citation-required answers), structured templates (role + objective + tone + tools + constraints + format + escalation), constraint elicitation (concrete do-don't lists with worked examples), output anchoring (force a tool_use schema, do not ask 'output JSON' in prose), and test-driven prompting (every change validated against a frozen eval suite, regression-free or roll back).
The reason every Domain 4 question on the exam is a composition question, never a single-technique question, is that real production prompts are layered. A refund agent uses a structured system-prompt template, three few-shot examples covering the sarcasm-style edge case, an unclear enum option for the refund-reason field, a forced tool_choice for output anchoring, and a 50-case eval that gates every prompt PR. Strip any one layer and the leak rate climbs above 5%. In production testing, natural-language 'output JSON' prompts leak structure ~15% of the time; tool-anchored prompts leak 0%.
How it works
Iterative refinement is the spine. You start with a baseline prompt that almost works, build a small eval set (20-50 cases is the sweet spot, of which 3-5 are deliberate failure cases you have already seen Claude get wrong), and run the prompt against the suite to get a score. Score is the only signal that matters; subjective 'this feels better' is theater. A baseline score gives you a reference for comparing prompt revisions on the same test cases. This is the point of Code based grading, a lesson in Anthropic’s Claude with Google Cloud Vertex AI course. Each refinement targets the lowest-scoring case, you write a tighter instruction or add an example, then re-run the entire suite to make sure no other case regressed.
Few-shot prompting gives Claude worked examples of the response you want. In Anthropic’s Providing examples lesson, you wrap each example in <sample_input> and <ideal_output> XML tags, you choose 2-5 examples that cover the failure modes (sarcasm, ambiguity, edge formats), and you optionally add a one-line explanation of *why* the example is ideal. Use correct, high-quality input/output pairs, including difficult cases. The lesson recommends learning from high-scoring evaluation outputs. Check the effect on your own test set; it does not establish a universal percentage improvement.
Output anchoring and anti-fabrication are where natural-language prompting reaches its ceiling. Asking 'please output JSON' in prose leaks structure roughly 15% of the time under load. The fix is tool_use with a JSON schema and tool_choice: forced, the API constrains token generation to match the schema, so structure is guaranteed at 100%. Fabrication is a separate problem, schemas guarantee shape, not truth. Per the structured-data-extraction scenario, when a source is genuinely silent, give the model two honest exits: nullable fields (`type: ['string', 'null']`) and an `unclear` / `not_provided` enum value. Without these escape hatches, required-string fields force fabrication and the leak rate climbs above 5%.
Where you'll see it
Refund agent prompt PR gate
Every prompt change PR runs against a 60-case golden suite (20 happy path, 20 edge, 10 escalations, 10 known-failure). The CI fails the PR if the average score drops or any previously-passing case regresses. This is an illustrative regression gate: testing a prompt once is insufficient evidence that later edits preserve behavior. Measure each revision rather than expecting a fixed gain after a fixed number of iterations.
Sarcasm-aware sentiment classifier
For a sentiment classifier that struggles with ironic praise, add a few correctly labelled sarcastic examples using <sample_input> and <ideal_output> pairs. Re-run the same evaluation and inspect both improvements and regressions. This illustrates the approach in Anthropic’s Providing examples lesson; the size of any improvement depends on the task.
Show 1 more examples
Contract-extraction anti-fabrication
A legal-ops extractor was fabricating termination_clause text when the contract was silent. The fix was schema-level, not prompt-level: nullable strings plus an unclear enum option in the tool input schema. Per the structured-data-extraction scenario doc, fabrication rate dropped from 8% to under 0.5% once the model had an honest exit. The system prompt remained almost unchanged.
Code examples
from anthropic import Anthropic
import json
client = Anthropic()
SYSTEM = """You classify tweet sentiment as positive, negative, or unclear.
DO emit one of: positive | negative | unclear.
DON'T add commentary or explanation.
DON'T pick positive when the tone is sarcastic.
Here are example input-output pairs:
<sample_input>I love how my flight was delayed three hours.</sample_input>
<ideal_output>negative</ideal_output>
<sample_input>Best coffee I've had all week, genuinely.</sample_input>
<ideal_output>positive</ideal_output>
<sample_input>idk what to make of this product</sample_input>
<ideal_output>unclear</ideal_output>"""
def classify(tweet: str) -> str:
resp = client.messages.create(
model="claude-opus-4-5",
max_tokens=10,
system=SYSTEM,
messages=[{"role": "user", "content": tweet}],
)
return resp.content[0].text.strip().lower()
def run_eval(cases_path: str) -> float:
cases = [json.loads(line) for line in open(cases_path)]
correct = sum(1 for c in cases if classify(c["tweet"]) == c["expected"])
score = correct / len(cases)
print(f"Score: {score:.2%} ({correct}/{len(cases)})")
return score
# Iterate: edit SYSTEM, re-run, compare. Ship only on regression-free improvement.
v1_score = run_eval("sentiment_cases.jsonl")from anthropic import Anthropic
client = Anthropic()
# Anti-fabrication: nullable + 'unclear' enum exit.
EXTRACT_TOOL = {
"name": "extract_refund_decision",
"description": "Extract refund decision from a customer ticket.",
"input_schema": {
"type": "object",
"properties": {
"decision": {
"type": "string",
"enum": ["approve", "deny", "escalate", "unclear"],
},
"refund_reason": {"type": ["string", "null"]},
"amount_usd": {"type": ["number", "null"]},
"confidence": {"type": "number", "minimum": 0, "maximum": 1},
},
"required": ["decision", "confidence"],
},
}
def extract(ticket: str) -> dict:
resp = client.messages.create(
model="claude-opus-4-5",
max_tokens=512,
tools=[EXTRACT_TOOL],
tool_choice={"type": "tool", "name": "extract_refund_decision"},
messages=[{"role": "user", "content": ticket}],
)
for block in resp.content:
if block.type == "tool_use":
return block.input
raise RuntimeError("forced tool_choice did not fire")
# 'unclear' + nullable fields let the model say 'I don't know' honestly.
result = extract("Customer wants a refund. No order ID provided.")
# Expected: {"decision": "escalate", "refund_reason": null, "amount_usd": null, "confidence": 0.3}Looks right, isn't
Each row pairs a plausible-looking pattern with the failure it actually creates. These are the shapes exam distractors are built from.
Adding 'output valid JSON, do not include any other text' to the system prompt to enforce structure.
Natural-language prompting for structure leaks ~15% of the time under load.
In production testing, the only structural guarantee is tool_use with a JSON schema and tool_choice: forced. Prompt instructions are advice, not enforcement.
Spending three days hand-tuning the wording of the system prompt to fix a 12% accuracy gap.
Concrete examples can clarify a distinction that repeated wording changes leave ambiguous.
Add correctly labelled examples for the error pattern and compare results on the same evaluation set; a particular improvement is not guaranteed.
Using Claude itself to grade 100% of your eval cases because it scales better than humans.
As explained in [Evaluation](/concepts/evaluation), model-based grading is not reproducible across model versions and fails on deterministic checks (tool sequences, JSON schemas).
Use code-based grading for anything measurable; reserve model grading for subjective qualities like tone.
Required-string field for refund_reason so the schema always returns a value the downstream code can parse.
Required strings force fabrication when the source is silent.
Per the structured-data-extraction scenario, fabrication climbs above 5% without an unclear enum option or a nullable type. Always give the model an honest exit.
Iterating the prompt 20+ times against the same 10 hand-picked test cases until the score reaches 100%.
That is overfitting: see [Evaluation](/concepts/evaluation).
Score plateau on a tiny fixed suite masks production blind spots. Grow the suite to 50+ externally-validated cases and pair with shadow-mode production evals.
Decision tree
Are you anchoring the output format?
tool_use with a JSON schema and tool_choice: forced. Structure is guaranteed at 100%, no parsing risk.Do you have a frozen 20-50 case eval suite, of which 3-5 are known-failure cases?
Is your accuracy gap concentrated in specific edge cases (sarcasm, ambiguity, format variance)?
<sample_input> / <ideal_output> examples that cover those exact cases. Re-test to see whether the examples help without making other cases worse.Is the model fabricating values when the source is silent?
unclear / not_provided enum option. Per the structured-data-extraction scenario, this drops fabrication from 8% to under 0.5%.required list.Question patterns
6 V2 questions wired to this concept. Tap an answer to check it instantly - you'll see whether it's right and why - then expand the full breakdown for the mental model and all four rationales.
Two of your tools have similar names (fetch_data and get_data). The model picks the wrong one 30% of the time. What is the best first fix?
Frequently asked
Showing 5 of 5 questions
<sample_input> and <ideal_output> label the example input and its intended answer, keeping both separate from your instructions. Plain bullets blur into the surrounding instructions.