CCAOF-D6.1 · Governance, Risk & Responsible Use15% of CCA-A8 min read

Identifying Appropriate vs Inappropriate AI Use Cases.

Before building a workflow around Claude, judge whether a task is one Claude is allowed and well-suited to do, not just whether it is technically possible. Anthropic's Usage Policy designates legal, financial, and employment-related tasks as "high-risk," requiring human-in-the-loop oversight and AI disclosure specifically when the output is consumer-facing; a separate, small set of categories (surveillance and biometric profiling, weapons design, malicious cyber operations, deceptive political content) is prohibited outright regardless of who consumes the output.

Loading your plan…
Mental modelThe trap is treating "Claude can produce this output" as equivalent to "this is an appropriate use case." Two outputs that look identical (a legal-sounding answer) can sit in different risk tiers depending on who consumes it directly, consumer-facing versus an internal analyst who reviews and re-packages it.
Identifying Appropriate vs Inappropriate AI Use Cases, hero illustration featuring Loop mascot in a warm gallery scene.
Share
On this page
01 · Summary

TLDR

Before building a workflow around Claude, judge whether a task is one Claude is allowed and well-suited to do, not just whether it is technically possible. Anthropic's Usage Policy designates legal, financial, and employment-related tasks as "high-risk," requiring human-in-the-loop oversight and AI disclosure specifically when the output is consumer-facing; a separate, small set of categories (surveillance and biometric profiling, weapons design, malicious cyber operations, deceptive political content) is prohibited outright regardless of who consumes the output.

3 (legal, financial, employment)
High-risk categories named
CCA-A D6 (15%)
Exam domain
2 (human review + AI disclosure)
High-risk safeguards
Consumer-facing output
Safeguard trigger
4
Prohibited category groups
3 (specified, verifiable, error-tolerant)
Task-suitability checks
02 · Definition

What it is

Before building a workflow around Claude, a business user has to judge whether the task is one Claude is *allowed* and *well-suited* to do, not just whether it is technically possible. These are two separate questions. Anthropic's Usage Policy is the authoritative framework for permissibility - what should and should not run on Claude at all, applying to everyone using Anthropic's products, consumer and enterprise alike. Passing the policy check is necessary but not sufficient: a task can be fully permitted by policy and still be a poor fit for an LLM in practice, which is a separate, task-suitability judgment about the nature of the task itself.

The policy sorts tasks along two independent axes an associate needs to check separately: whether the task category itself is designated high-risk or prohibited, and whether the specific output reaches an end user directly or gets reviewed and re-packaged first. A task can be high-risk and low-risk in effect depending on that second axis, while a prohibited category stays prohibited no matter how it is delivered. On top of that, a third, independent check applies regardless of policy category: is the task itself well-specified, verifiable, and tolerant of the kind of error an LLM can make? A permitted task that is vague, unverifiable, and error-intolerant (e.g., an ungoverned final decision with no review step) is still a bad candidate for Claude even though nothing in the Usage Policy blocks it.

03 · Mechanics

How it works

High-risk categories are named explicitly. Anthropic designates certain categories as "high-risk use cases," ones with public welfare and social equity implications, explicitly including legal, financial, and employment-related uses of Claude. Landing in one of these categories does not mean the task is off-limits, it means extra conditions attach before it ships.

Those extra safeguards are human-in-the-loop oversight and AI disclosure. High-risk use cases carry additional safeguards: a human-in-the-loop oversight path and a requirement to disclose to end users that they are interacting with an AI system.

The August 2025 update scoped those safeguards to consumer-facing output. Anthropic clarified that the extra requirements apply specifically when the model's output is consumer-facing, not for business-to-business interactions. An internal analyst using Claude to draft an employment-policy summary for HR sits in a different risk tier than a consumer-facing chatbot giving individualized legal or financial advice, even though both are "employment" or "legal" in category.

A separate set of categories is prohibited outright, regardless of consumer-facing or B2B framing. The policy prohibits surveillance, tracking, profiling, and biometric-identity inference; weapons design; malicious cyber operations; and deceptive political or election content, no safeguard level makes these appropriate. Law enforcement use is permitted for a carefully tailored set of use cases (back-office support, document summarization) but restricted for surveillance or profiling, showing the two axes, category and delivery context, apply even within one type of user.

The Usage Policy is a living document. Anthropic frames it as revised periodically as capabilities and regulation evolve, so current guidance can date quickly, an architect or business user should verify against the live policy page before quoting specifics rather than relying on a memorized snapshot.

Task-suitability is a separate check, and policy-permitted does not mean well-suited. Even inside a fully allowed category, ask three things about the task itself: is it well-specified (a clear, bounded definition of what a correct output looks like, versus an open-ended, ambiguous ask), is it verifiable (can a human or a downstream system actually check the output before it has consequences, versus a one-shot answer nobody reviews), and is it tolerant of the kind of error an LLM can make (a wrong draft that gets edited is low-cost; a wrong number that silently drives a payment is not). A task can clear every Usage Policy check and still be the wrong fit for Claude if it fails these three - the policy answers "is this allowed," not "is this a good idea."

Identifying Appropriate vs Inappropriate AI Use Cases mechanics, painterly diagram featuring Loop mascot.
Exam trap

The trap is treating "Claude can produce this output" as equivalent to "this is an appropriate use case." Two outputs that look identical (a legal-sounding answer) can sit in different risk tiers depending on who consumes it directly, consumer-facing versus an internal analyst who reviews and re-packages it. A second, related trap is treating "the Usage Policy allows it" as equivalent to "this is a good use case" - permissibility and task-suitability (well-specified, verifiable, error-tolerant) are separate checks, and a policy-permitted task can still be a poor fit for an LLM. The exam is likely to test both: whether a candidate recognizes when a task crosses into a high-risk category requiring human oversight and disclosure, and whether policy clearance alone is being mistaken for appropriateness.

04 · In production

Where you'll see it

Employee benefits chat widget

Consumer-facing eligibility Q&A is employment-related and reaches the employee directly, so it needs a human-in-the-loop review path for edge cases and a clear AI-disclosure notice.

Internal HR FAQ drafting

The same underlying task, used internally with an HR specialist reviewing before publishing, is a materially lower-risk, B2B-style use of Claude even though the subject matter is identical.

05 · When to use

Decision tree

01

Does the task fall into an explicitly prohibited category (surveillance, tracking, profiling, biometric-identity inference, weapons design, malicious cyber operations, deceptive political or election content)?

YesStop, this is prohibited regardless of consumer-facing or B2B framing; no safeguard level makes it permitted.
NoNot prohibited outright, continue to the high-risk check.
02

Does the task fall into a designated high-risk category (legal, financial, or employment-related)?

YesIt may need the high-risk safeguards, check whether the output is consumer-facing to know if they apply.
NoStandard usage-policy terms apply, no high-risk safeguard requirement from this axis.
03

Is the model's output consumer-facing (an end user is the direct recipient), rather than reviewed and re-packaged by an internal analyst first?

YesIf also high-risk, the additional safeguards, a human-in-the-loop review path and a clear AI-disclosure notice, are required as of the August 2025 policy clarification.
NoSame underlying task, lower risk tier, an internal reviewer re-packaging output before anyone else sees it is a materially different, B2B-style use even when the task category is identical.
04

Is this a law-enforcement use case?

YesPermitted only for a carefully tailored set of use cases such as back-office support or document summarization, restricted for surveillance or profiling regardless of the other axes.
NoThis axis does not apply.
05

Assuming the task clears the Usage Policy, is it well-specified, verifiable, and tolerant of the errors an LLM can make?

YesPolicy-permitted AND well-suited - proceed.
NoPolicy permission alone doesn't make this a good use case; redesign the task (narrow the spec, add a verification/review step, lower the blast radius of an error) before treating it as appropriate.
06 · On the exam

Question patterns

Identifying Appropriate vs Inappropriate AI Use Cases exam trap, painterly cautionary scene featuring Loop mascot.

6 V2 questions wired to this concept. Tap an answer to check it instantly - you'll see whether it's right and why - then expand the full breakdown for the mental model and all four rationales.

Question 1 of 6 · D6Choose the best answer

A benefits team wants to deploy a Claude-powered chat widget that gives individual employees direct, consumer-facing answers about health-plan eligibility with no human review step. How should this be classified?

07 · FAQ

Frequently asked

Showing 3 of 3 questions

Help someone pass

Share this concept.

One share is one less person stuck on the same question.

Last reviewed: ·Refresh cadence: monthly