What Anthropic teaches in this course
Generative AI is not uniformly capable or uniformly unreliable. It is strong and weak along four predictable axes, and the same underlying mechanism that produces a strength often produces the matching weakness. Anthropic's framework names those axes Next Token Prediction, Knowledge, Working Memory, and Steerability. Each one is a continuum, not a switch. Your job before delegating any task is to locate it on each continuum and decide what verification or context to supply. That move is what Anthropic calls calibrated trust, and it is the single most exam-relevant idea in the course.
Next Token Prediction answers *where do AI answers come from?* The model is writing what statistically comes next, one fragment at a time. On well-worn paths (summarize, reformat, explain a common concept) the patterns are dense and the output is reliable. On novel territory the same fluent prose keeps coming, but accuracy thins. Fabrication concentrates in specificity; names, dates, statistics, citations, URLs, quotes. Confident tone is not an accuracy signal; smoothness and correctness are independent variables. Product features (citations, uncertainty signaling, constrained generation, generator-verifier loops) push the edge out, but the verification habit is yours to build.
Knowledge answers *what does the model actually know?* Everything it learned came from training data and is frozen at a cutoff date. Mainstream, well-documented, stable topics land in the capability zone. Rare, post-cutoff, niche, local, or contested topics drift toward the edge. The characteristic failures are staleness (true-at-training-time is not true-now), uneven coverage (minority languages and recent developments suffer), inherited bias (the model's sense of default reflects training-data blind spots), and source amnesia (I read this somewhere is not a citation). Web search, RAG/retrieval, MCP, and tool use are all mitigations that extend knowledge at runtime; if you are not using them you are relying entirely on what the model absorbed.
Working Memory answers *what is the AI paying attention to right now?* This is the context window; your instructions, uploaded docs, prior responses, all in one finite container the model rereads every turn. Unlike the other three properties, this one has a cliff rather than a gradient. When the window overflows, oldest material falls off silently. Attention is not uniform across the window either; the lost-in-the-middle effect means buried instructions carry less weight than top-or-tail ones. The model does not learn from your corrections; it only responds to what is currently in context. Memory features, projects, compaction, larger windows, and skills push the cliff further out, but front-loading critical material and chunking long work are your operator-side defenses.
Steerability answers *how much am I in control?* Fine-tuning taught the model to treat your input as a request and follow rules, which gives precise control over format, role, length, and tone. But instructions are pattern-matched, not understood. Short, concrete, verifiable asks (respond as a table, under 100 words) land cleanly. Long reasoning chains drift; abstract asks like be insightful get patchy results; native arithmetic precision is brittle without code execution. The two characteristic failures are reasoning drift (small errors compound and the model does not notice) and letter-over-spirit (instruction honored literally but uselessly). When an instruction is followed literally but uselessly, restate the goal, not the instruction; repeating be concise louder will not fix what was really an intent problem.
The diagnostic move that turns this into a working tool: most real-world AI failures are two properties colliding, not one. A hallucinated citation is Next Token Prediction meeting a Knowledge gap (the model generates what a plausible citation looks like while the training data is sparse). Long-conversation drift is Working Memory meeting Steerability (early constraints fade as the window fills, and steerability follows whatever instructions are most salient now). Confidently-wrong arithmetic is Next Token Prediction meeting Steerability without code execution. Before reaching for a prompt fix, name which two properties are at play. The fix follows automatically from the diagnosis: verify specifics, re-supply context, offload to a tool, or invite explicit pushback.
Two training stages give every model its character. Pretraining reads vast amounts of text and learns one thing: predict what comes next. The result is a document completer with no concept of you or of helping. Fine-tuning is a second round on curated examples of helpful behavior plus reward signals shaped by human preferences. That second stage leaves fingerprints: sycophancy (the model validates your framing and backs down under light pushback), verbosity (thoroughness scored well in training, so essays come back when you wanted bullets), over-caution (conservative safety training means hedging on requests that are actually fine), and loose calibration between stated confidence and actual reliability. These are not bugs in one model; they are training artifacts that appear across all of them; knowing them puts you in control.
You'll walk away with
- Why generative AI is fundamentally a prediction system, not a search engine, and what that implies for fluency vs accuracy
- How pretraining and fine-tuning give each model its character, and the four behavioral fingerprints those stages leave (sycophancy, verbosity, over-caution, loose calibration)
- How to locate any task on each of the four property continuums and predict where it will struggle before you run it
- How to diagnose real failures by naming which two properties collided, then choose a targeted mitigation
- How this framework connects to the 4D Framework (Delegation, Description, Discernment, Diligence) as two halves of one calibrated-trust system
14 lessons, with our annotations
6 ideas to carry into practice
Lines worth keeping
Next Token Prediction; where do answers come from?
Capability: well-worn paths (summarize, reformat, explain). Limitation: novel territory and specificity (names, dates, citations, URLs). The same mechanism produces fluency and hallucination.
Related concept →Knowledge; what does it actually know?
Capability: frequent, recent-in-training, consistent topics. Limitation: rare, post-cutoff, niche, local, contested. Mitigations: web search, RAG, tool use, MCP.
Related concept →Working Memory; what is it paying attention to right now?
Capability: material fits comfortably, session is current, context supplied. Limitation: very long docs/conversations, cross-session continuity, lost-in-the-middle. The cliff is silent; you will not always be warned.
Related concept →Steerability; how much am I in control?
Capability: short, concrete, verifiable instructions. Limitation: long reasoning chains, abstract asks, native precision. Failures: reasoning drift and letter-over-spirit.
Related concept →Sycophancy
People prefer agreeable responses, so the model learns to validate your framing and back down under light pushback even when it was right. Counter by explicitly inviting disagreement: genuinely push back if you think I am wrong.
Verbosity
Thoroughness scored better in training, so the default is longer answers. Counter with explicit length constraints (one sentence, under 100 words, bullets only).
Over-caution
Conservative safety training means hedging on requests that are actually fine. Counter by stating the legitimate context up front so the model has a frame for why the ask is reasonable.
Loose calibration
Stated confidence and actual reliability are not tightly coupled. The model can sound certain while being wrong. Verify specifics independently regardless of tone.
How this course shows up on the exam
Builds the calibrated-trust mental model behind D1 task statements about hallucination, knowledge cutoffs, and context limits, and grounds D2 prompt-engineering decisions in why each technique exists. The four properties (Next Token Prediction, Knowledge, Working Memory, Steerability) are the diagnostic vocabulary the exam expects you to apply.
Blueprint weight27% (D1) + 18% (D2)Check the pattern
Long-document extraction stalls at chunk 18 every single time, regardless of which model you use. What is happening?
Frequently asked
What are the four properties of generative AI in Anthropic's capabilities and limitations framework?
Next Token Prediction (where answers come from), Knowledge (what the model knows), Working Memory (what it is paying attention to right now), and Steerability (how much you are in control). Each sits on a continuum from capability to limitation, and most real-world failures are two of them colliding rather than one acting up alone.
Why does an AI hallucinate citations when it sounds so confident?
Confident tone and accuracy are independent variables in a generative model. The model writes what a plausible citation looks like using Next Token Prediction; when the underlying Knowledge is sparse on that niche topic, it generates citation-shaped text that may or may not point to a real paper. Fabrication concentrates in specificity; names, dates, journal titles, URLs; so verify those independently no matter how smooth the prose sounds.
What is the difference between Knowledge and Working Memory in an AI model?
Knowledge is what the model absorbed during training and is frozen at a cutoff date. Working Memory is the context window: what the model is paying attention to *right now*; your prompt, uploaded docs, prior turns. Knowledge fails through staleness and uneven coverage; Working Memory fails through silent truncation, lost-in-the-middle, and the blank slate between sessions. The mitigations are different: web search and RAG for Knowledge, front-loading and projects/memory for Working Memory.
How do I stop the AI from agreeing with me when I want honest feedback?
That behavior is sycophancy, a fingerprint left by fine-tuning on human preference data; people prefer agreeable responses, so the model learns to validate your framing. Counter it by explicitly inviting disagreement in the prompt: I want you to genuinely disagree if you think I am wrong; do not agree just because I sounded confident. The pattern only changes when you give the model permission to push back.
Why does my AI assistant ignore the rules I set 20 messages ago?
That is long-conversation drift; Working Memory meeting Steerability. Your early constraints have either fallen out of the context window (silent truncation) or are now buried so deep in the conversation that lost-in-the-middle is suppressing them. The fix is to either re-supply the critical constraints in the current turn, move them into a system prompt or Project so they stay persistent, or start a fresh conversation with the essentials front-loaded.
Are these four properties going to change as models get better?
The properties stay; the boundaries move. Context windows grow, hallucination rates drop, features close gaps. But generative AI will keep being a predictor whose fluency runs ahead of its accuracy, with uneven knowledge frozen at a cutoff, working inside a finite window, following instructions through a gap between words and intent. That is why the framework is durable on purpose; it remains useful even when version numbers change.
When should I add web search or RAG to my AI workflow?
Whenever your task lives in the Knowledge limitation zone: rare topics, post-cutoff events, niche regulations, local information, fast-moving fields, contested claims, or anywhere staleness is a real risk. Web search routes around the cutoff for time-sensitive questions; RAG and MCP connect the model to documents it never trained on (your wiki, a specialized database). If the task is in the capability zone; mainstream, stable, well-documented; the absorbed knowledge is usually enough.
How does this framework relate to the 4D Framework (Delegation, Description, Discernment, Diligence)?
They are two halves of one calibrated-trust system. The 4Ds are what *you* do; the four properties are what you are responding to when you do them. Next Token Prediction sharpens Discernment (fluency and accuracy are independent). Working Memory sharpens Description (context is leverage, the model does not remember). Steerability sharpens Delegation (you know where control is tight and where it is loose). Knowledge sharpens all of them by telling you when to hand off and when to bring the context yourself.