Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 6

Question 50

A credit operations team is assembling a labeled benchmark from reviewers who initially disagree on ambiguous classifications. Which dataset-quality measure would show that the reference answers have been adjudicated consistently?

  1. Attack success rate and safe handling of intentionally unanswerable prompts
  2. Expert-rated factual fidelity and calibrated grader agreement on sampled explanations
  3. Inter-annotator agreement and unresolved-label rate after adjudication
  4. Zero schema violations accepted by the downstream consumer across generated cases
  5. Per-stratum pass rates with no uncovered critical scenario classes

Explanation

Correct Answer

C:

Label reliability is necessary before model score differences can be interpreted; measure inter-annotator agreement and unresolved-label rate after adjudication at the release gate; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth.

Incorrect Answers

A:

Red-team cases make trust-boundary performance observable; add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; this tests attack success rate and safe handling of intentionally unanswerable prompts; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

B:

A semantic rubric tests meaning while allowing legitimate variation in language; use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; this tests expert-rated factual fidelity and calibrated grader agreement on sampled explanations; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

D:

Executable contract tests deliver reproducible feedback at scale; apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; this tests zero schema violations accepted by the downstream consumer across generated cases; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

E:

Stratified coverage prevents abundant easy cases from hiding unsupported workflows; build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; this tests per-stratum pass rates with no uncovered critical scenario classes; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

Question 51

Production performance stagnates because changes memorize the public development samples. How should the design be corrected?

  1. Include hostile prompt-injection and no-answer probes
  2. Protect a separate holdout set for regression decisions
  3. Use deterministic code checks for objective output contracts
  4. Stratify a golden dataset by real workload segments

Explanation

Correct Answer

B:

Maintain untouched holdout cases and periodic refreshes representing real traffic, with clear version and leakage controls; untouched data estimates whether improvements transfer beyond the examples used during tuning; verify the repair using holdout quality stability across successive prompt versions.

Incorrect Answers

A:

Testing normal prompts alone cannot estimate resilience to malicious or unsatisfiable requests; add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; that control is assessed by attack success rate and safe handling of intentionally unanswerable prompts; the incident instead requires holdout quality stability across successive prompt versions.

C:

Subjective model grading is unnecessary and less reliable for an objectively checkable parser contract; apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; that control is assessed by zero schema violations accepted by the downstream consumer across generated cases; the incident instead requires holdout quality stability across successive prompt versions.

D:

A random sample alone can underrepresent important low-frequency and high-consequence events; build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; that control is assessed by per-stratum pass rates with no uncovered critical scenario classes; the incident instead requires holdout quality stability across successive prompt versions.

Question 52

A decision-support portal shows model recommendations beside cited source facts. User testing examines whether people understand what is advisory versus binding. Which result would establish trustworthy communication?

  1. Task completion by users of tested assistive technologies on representative flows
  2. Quality and escalation parity across predeclared language segments
  3. Time to resolve substantiated challenges and proportion of cases with documented review
  4. User comprehension of advisory status and source-citation validity

Explanation

Correct Answer

D:

Transparent decision tests measure whether users understand what the system did and did not decide; measure user comprehension of advisory status and source-citation validity at the release gate; distinguish evidence, uncertainty, and advisory status with accessible citations and a path to authoritative decisions.

Incorrect Answers

A:

Direct accessibility testing measures practical use rather than formal content correctness only; include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; this tests task completion by users of tested assistive technologies on representative flows; the actual release condition is user comprehension of advisory status and source-citation validity.

B:

Group-specific evaluation reveals concentrated harm masked by pooled scores; evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; this tests quality and escalation parity across predeclared language segments; the actual release condition is user comprehension of advisory status and source-citation validity.

C:

Remedy effectiveness measures whether affected people can obtain meaningful reconsideration; provide a contest and correction process with accountable human ownership for significant impacts; this tests time to resolve substantiated challenges and proportion of cases with documented review; the actual release condition is user comprehension of advisory status and source-citation validity.

Question 53

A knowledge service adds a lightweight reranking stage before passing passages to Claude. In trials with ambiguous employee queries, which evidence would justify the added cost and latency?

  1. Grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths
  2. Job completion reliability and duplicate-submission rate during slow-system simulations
  3. Deadline compliance and correct indication of omitted enrichment
  4. Accuracy by difficulty segment and unit cost under p95 constraints

Explanation

Correct Answer

A:

A quality-latency frontier shows whether an extra ranking pass is worth its overhead; measure grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths at the release gate; retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model.

Incorrect Answers

B:

Durable job identity decouples user interaction time from background work duration; acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; this tests job completion reliability and duplicate-submission rate during slow-system simulations; the actual release condition is grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.

C:

Timed fault injection confirms graceful degradation without inventing evidence; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; this tests deadline compliance and correct indication of omitted enrichment; the actual release condition is grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.

D:

Joint segmented reporting reveals whether expensive processing is spent where it helps; use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; this tests accuracy by difficulty segment and unit cost under p95 constraints; the actual release condition is grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.

Question 54

A project copilot spans weeks of discussion and must retain agreed constraints while avoiding stale chit-chat. What is the most targeted architecture choice?

  1. Attach provenance metadata to selected context
  2. Retrieve only relevant material instead of full archives
  3. Allocate input and output token budgets per workflow step
  4. Compact extended sessions without discarding commitments

Explanation

Correct Answer

D:

Store a structured, reviewable summary of decisions, open issues, and provenance; inject it at checkpoints rather than replaying every chat turn; a long-horizon test checks memory of obligations rather than raw conversation recall; evaluate the design with retention of signed-off requirements after long simulated conversations.

Incorrect Answers

A:

Supply source identity, version, effective date, and retrieval relevance alongside each chunk so the model can attribute evidence correctly; provenance metadata does not guarantee the upstream search selected the right document in the first place; this alternative would measure citation correctness by effective-date version in temporal regression questions; it does not establish retention of signed-off requirements after long simulated conversations for the described use case.

B:

Apply query-driven retrieval and evidence selection before prompting Claude, retaining citations and permission filters; broad context stuffing can degrade relevance and cost even when a large context window is available; this alternative would measure answer support and retrieved-token volume on a sampled policy-question set; it does not establish retention of signed-off requirements after long simulated conversations for the described use case.

C:

Reserve output capacity and bound each stage's retrieved context and response length according to the deliverable structure; merely expanding every context window can increase cost while leaving output truncation unaddressed; this alternative would measure complete mandatory report sections under the worst supported input size; it does not establish retention of signed-off requirements after long simulated conversations for the described use case.

Question 55

A refund investigation service allows flexible evidence gathering but keeps identity verification and final approval outside the exploratory component. Which release-gate outcome best demonstrates the boundary is working?

  1. Disputed-case resolution with every refund still passing the identity and approval gates
  2. Rate of recommendations backed by all required completed stage checks
  3. Correct citations to currently effective policy versions across update tests
  4. Completed valid classifications per batch window and cost per accepted record

Explanation

Correct Answer

A:

This tests flexible investigation and deterministic control at the same time; measure disputed-case resolution with every refund still passing the identity and approval gates at the release gate; place bounded exploratory reasoning inside one stage of a deterministic state machine; enforce approval and write permissions outside the agent.

Incorrect Answers

B:

Stage-completion evidence verifies that mandatory controls were not left to agent discretion; explicit state transitions preserve required order, testing points, and deterministic branch conditions; Claude can supply judgment inside selected stages; this tests rate of recommendations backed by all required completed stage checks; the actual release condition is disputed-case resolution with every refund still passing the identity and approval gates.

C:

Version-aware citation checks show whether answers rely on authoritative current evidence; retrieval from a versioned, access-filtered source grounds answers in current documents and can expose the authoritative citation; this tests correct citations to currently effective policy versions across update tests; the actual release condition is disputed-case resolution with every refund still passing the identity and approval gates.

D:

Throughput and unit economics measure the benefit that a user-facing latency metric would miss; batch-oriented orchestration handles throughput, retries, and cost planning while preserving consistent model instructions for a noninteractive workload; this tests completed valid classifications per batch window and cost per accepted record; the actual release condition is disputed-case resolution with every refund still passing the identity and approval gates.

Question 56

A connected search service and telemetry exporter silently transfer document snippets outside the approved region. Which intervention targets the defect rather than its symptoms?

  1. Minimize personal data and specify retention purpose
  2. Validate FedRAMP authorization against exact service boundaries
  3. Map data residency commitments to actual processing paths
  4. Implement accountable access, deletion, and retention processes

Explanation

Correct Answer

C:

Trace storage, inference, retrieval, logs, backups, and subprocessors against contractual and legal location requirements; a full data-flow audit finds secondary exports that primary-storage checks miss; verify the repair using verified geographic processing paths for all sensitive request components.

Incorrect Answers

A:

Broad encryption does not justify collecting more personal data than the task needs; limit processing to necessary authorized fields, define a lawful purpose and retention controls, and document relevant privacy obligations; that control is assessed by sensitive-field inclusion rate in prompts and retention-policy compliance; the incident instead requires verified geographic processing paths for all sensitive request components.

B:

A general government-cloud label or vendor marketing statement is not equivalent to FedRAMP authorization; verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection; that control is assessed by documented authorization-boundary mapping for every service handling regulated data; the incident instead requires verified geographic processing paths for all sensitive request components.

D:

Deleting the original row does not automatically erase embeddings, caches, or observability copies; map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion; that control is assessed by deletion-request completion across all inventoried repositories within policy deadlines; the incident instead requires verified geographic processing paths for all sensitive request components.

Question 57

An industrial inspection reviewer needs to trace one recommendation back to the exact sensor evidence and processing revisions. Meanwhile, an expense-intake pipeline receives images and tables with inconsistent units and field names before the model call. Which TWO controls directly solve the traceability and input-quality needs? Choose TWO.

  1. Normalize incoming documents into a validated input contract
  2. Validate model output against a business schema before side effects
  3. Link each output to source and processing trace IDs
  4. Persist workflow state and make retries idempotent
  5. Use a queue and asynchronous workers for bursty workloads

Explanation

Correct Answers

A:

An input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; a schema-conformance gate reveals extraction failures before they can contaminate reasoning; its corresponding workstream should be validated by percentage of ingested documents meeting schema checks before any Claude call.

C:

Correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review; end-to-end provenance lets reviewers investigate disputed outcomes without assuming the final text is self-evidencing; its corresponding workstream should be validated by fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

Incorrect Answers

B:

Use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; output parsing constrains generated data but does not by itself manage retry idempotency or transport durability; its natural acceptance measure is invalid write payloads rejected before execution, with zero partial supplier mutations, which does not satisfy the separate workstreams being assessed.

D:

Persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; durable-state control prevents duplicate effects but cannot decide whether a refund is substantively authorized; its natural acceptance measure is repeated and interrupted test runs yielding exactly one committed credit per approved case, which does not satisfy the separate workstreams being assessed.

E:

Decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; queueing absorbs bursts but does not improve a weak answer rubric or justify a higher-privilege tool set; its natural acceptance measure is job completion rate and queue age during the peak intake window, which does not satisfy the separate workstreams being assessed.

Question 58

Front-end requests expire during spikes although background computation later completes successfully. Which corrective change best addresses the root cause?

  1. Normalize incoming documents into a validated input contract
  2. Persist workflow state and make retries idempotent
  3. Validate model output against a business schema before side effects
  4. Use a queue and asynchronous workers for bursty workloads

Explanation

Correct Answer

D:

Decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; measuring backlog and completion under load tests the exact burst tolerance the design promises; verify the repair using job completion rate and queue age during the peak intake window.

Incorrect Answers

A:

Input normalization addresses upstream ambiguity, not the authorization needed before writing to a finance ledger; an input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; that control is assessed by percentage of ingested documents meeting schema checks before any Claude call; the incident instead requires job completion rate and queue age during the peak intake window.

B:

Durable-state control prevents duplicate effects but cannot decide whether a refund is substantively authorized; persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; that control is assessed by repeated and interrupted test runs yielding exactly one committed credit per approved case; the incident instead requires job completion rate and queue age during the peak intake window.

C:

Output parsing constrains generated data but does not by itself manage retry idempotency or transport durability; use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; that control is assessed by invalid write payloads rejected before execution, with zero partial supplier mutations; the incident instead requires job completion rate and queue age during the peak intake window.

Question 59

The first pilot fails because critical exception and privacy requirements were never gathered from operating teams. Which corrective change best addresses the root cause?

  1. Obtain sign-off on explicit acceptance criteria and exclusions
  2. Validate risky assumptions through narrow experiments
  3. Interview each role that owns a distinct outcome or risk
  4. Map the owner and authority of every required data source

Explanation

Correct Answer

C:

Use role-based discovery to capture goals, current workflows, authority boundaries, and failure impacts from relevant stakeholders; traceable requirements reveal whether important operational perspectives were omitted; verify the repair using coverage of approved requirements traced to affected stakeholder groups.

Incorrect Answers

A:

Informal meeting consensus may not survive staff changes or new deployment expectations; document in-scope outcomes, exclusions, metrics, responsibilities, and go-live gates for accountable approval; that control is assessed by signed acceptance criteria mapped to demonstrable tests and named owners; the incident instead requires coverage of approved requirements traced to affected stakeholder groups.

B:

Detailed architecture diagrams do not prove uncertain input quality or model capability; identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; that control is assessed by resolution rate of prioritized unknowns through measured prototypes; the incident instead requires coverage of approved requirements traced to affected stakeholder groups.

D:

A list of desired features cannot grant access or establish source truthfulness; create a source inventory with accountable owners, sensitivity, update rate, and approval dependencies before committing to the design; that control is assessed by percentage of required sources with approved owners, legal uses, and integration paths; the incident instead requires coverage of approved requirements traced to affected stakeholder groups.

Question 60

A large engineering project delegates code search, test investigation, and documentation to assistants with separate permission sets. Which acceptance outcome would show that delegated work completes without unnecessary privileged tools?

  1. Consistency of generated changes with repository build and test conventions across fresh sessions
  2. Denial of prohibited commands after attempts to override local and project configuration
  3. Secret-scanning results and blocked exfiltration on seeded repository credentials
  4. Subagent task completion with zero out-of-scope tool access

Explanation

Correct Answer

D:

Role-isolation tests prove that decomposition also restricts authority; measure subagent task completion with zero out-of-scope tool access at the release gate; give each delegated role explicit objectives, context limits, and tool permissions with clear return artifacts.

Incorrect Answers

A:

Reproducible onboarding shows whether team guidance is discoverable and current; put reviewed repository-specific guidance in a shared CLAUDE.md file with appropriate scope and code review; this tests consistency of generated changes with repository build and test conventions across fresh sessions; the actual release condition is subagent task completion with zero out-of-scope tool access.

B:

Policy precedence tests validate that enterprise controls remain effective; use organization-managed policy and supported permission configuration so enforced restrictions cannot be casually overridden by per-user convenience settings; this tests denial of prohibited commands after attempts to override local and project configuration; the actual release condition is subagent task completion with zero out-of-scope tool access.

C:

Source and runtime scans target the actual paths by which developer secrets can leak; use approved secret storage, scoped environment access, and redaction while avoiding credentials in repositories, prompts, or logs; this tests secret-scanning results and blocked exfiltration on seeded repository credentials; the actual release condition is subagent task completion with zero out-of-scope tool access.

Claude Certified Architect - Professional (CCAR-P) Practice Tests