Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 4

Question 30

A test corpus of only friendly documents gives a false sense of safety before hostile uploads are accepted. What design correction resolves the reported failure?

  1. Use deterministic code checks for objective output contracts
  2. Measure human annotator consistency before trusting labels
  3. Use expert rubrics for nuanced qualitative outputs
  4. Include hostile prompt-injection and no-answer probes

Explanation

Correct Answer

D:

Add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; red-team cases make trust-boundary performance observable; verify the repair using attack success rate and safe handling of intentionally unanswerable prompts.

Incorrect Answers

A:

Subjective model grading is unnecessary and less reliable for an objectively checkable parser contract; apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; that control is assessed by zero schema violations accepted by the downstream consumer across generated cases; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.

B:

More test volume does not fix unreliable or undefined labels; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; that control is assessed by inter-annotator agreement and unresolved-label rate after adjudication; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.

C:

Pure string comparison is unsuitable when many substantively correct phrasings exist; use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; that control is assessed by expert-rated factual fidelity and calibrated grader agreement on sampled explanations; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.

Question 31

A large commerce deployment produces more inference traces than the log pipeline can retain economically. Which intervention is most appropriate for this use case?

  1. Sample complete traces strategically under heavy traffic
  2. Separate retrieval, provider, and tool latency budgets
  3. Correlate model, retrieval, and tool spans in distributed traces
  4. Segment reliability metrics by tenant and workload

Explanation

Correct Answer

A:

Retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; risk-aware sampling preserves evidence for rare faults without uncontrolled storage growth; evaluate the design with capture rate of seeded critical failures at a bounded telemetry cost.

Incorrect Answers

B:

Instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; optimizing generation tokens cannot repair a slow connector that dominates the critical path; this alternative would measure stage-specific p95 deltas identifying injected bottlenecks; it does not establish capture rate of seeded critical failures at a bounded telemetry cost for the described use case.

C:

End-to-end traces with consistent request and span identifiers identify which service and step consumed time; a model-only latency chart cannot explain retrieval or downstream network bottlenecks; this alternative would measure trace completeness and correct bottleneck attribution on instrumented fault injections; it does not establish capture rate of seeded critical failures at a bounded telemetry cost for the described use case.

D:

Break down latency, errors, and tool success by permission-safe tenant and workload dimensions with cardinality controls; overall averages can hide concentrated failures without meaningful dimensional analysis; this alternative would measure detection time for an injected single-tenant degradation; it does not establish capture rate of seeded critical failures at a bounded telemetry cost for the described use case.

Question 32

A coding team needs to invoke reproducible local build and test commands in a sandboxed engineering environment. Which intervention is most appropriate for this use case?

  1. Use agent-to-agent exchange for independently owned specialist systems
  2. Use direct APIs for a narrow stable application integration
  3. Use a controlled CLI interface for developer automation
  4. Use MCP to expose reusable tools and resources across compatible clients

Explanation

Correct Answer

C:

A bounded CLI execution environment can reuse standard developer tools with explicit directory and command permissions; execution tests check both the developer outcome and the sandbox boundary; evaluate the design with reproducible build results with denied commands outside the approved allowlist.

Incorrect Answers

A:

An agent-to-agent contract can exchange bounded tasks and outputs while preserving local autonomy, identity, and authority of each specialist system; messaging between agents does not make their outputs trustworthy or authorize cross-system actions automatically; this alternative would measure successful authenticated task exchange with each agent retaining its own permission checks; it does not establish reproducible build results with denied commands outside the approved allowlist for the described use case.

B:

A direct typed API client is appropriate when the interface, arguments, and control flow are known and stable; direct API calls offer less automatic discovery when many heterogeneous tools must be used by several assistants; this alternative would measure integration error rate and p95 latency on the fixed status-check path; it does not establish reproducible build results with denied commands outside the approved allowlist for the described use case.

D:

A maintained MCP server can standardize discoverable tool and resource contracts while each client still enforces authentication and trust; mCP is an interface protocol, not a substitute for access policy or secure server implementation; this alternative would measure cross-client contract compatibility and unauthorized-action denial for the same MCP service; it does not establish reproducible build results with denied commands outside the approved allowlist for the described use case.

Question 33

A supplier-records assistant is about to gain write access through a controlled service interface. The test plan includes bad field names, unauthorized category values, and interrupted transactions. What evidence best shows safe input handling?

  1. Repeated and interrupted test runs yielding exactly one committed credit per approved case
  2. Invalid write payloads rejected before execution, with zero partial supplier mutations
  3. Fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition
  4. Percentage of ingested documents meeting schema checks before any Claude call

Explanation

Correct Answer

B:

Pre-write enforcement proves that the model cannot bypass the target system's integrity contract; measure invalid write payloads rejected before execution, with zero partial supplier mutations at the release gate; use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction.

Incorrect Answers

A:

Replay tests directly exercise the failure window that typically creates duplicate writes; persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; this tests repeated and interrupted test runs yielding exactly one committed credit per approved case; the actual release condition is invalid write payloads rejected before execution, with zero partial supplier mutations.

C:

End-to-end provenance lets reviewers investigate disputed outcomes without assuming the final text is self-evidencing; correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review; this tests fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition; the actual release condition is invalid write payloads rejected before execution, with zero partial supplier mutations.

D:

A schema-conformance gate reveals extraction failures before they can contaminate reasoning; an input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; this tests percentage of ingested documents meeting schema checks before any Claude call; the actual release condition is invalid write payloads rejected before execution, with zero partial supplier mutations.

Question 34

A safety console has a hard response deadline and uses optional descriptive enrichment. During delayed-service tests, which result shows that the core warning remains timely and accurately labeled?

  1. Job completion reliability and duplicate-submission rate during slow-system simulations
  2. Repeated-query latency with zero stale or cross-scope cache responses after updates
  3. Deadline compliance and correct indication of omitted enrichment
  4. Accuracy by difficulty segment and unit cost under p95 constraints
  5. End-to-end p95 improvement with all required sources still present

Explanation

Correct Answer

C:

Timed fault injection confirms graceful degradation without inventing evidence; measure deadline compliance and correct indication of omitted enrichment at the release gate; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget.

Incorrect Answers

A:

Durable job identity decouples user interaction time from background work duration; acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; this tests job completion reliability and duplicate-submission rate during slow-system simulations; the actual release condition is deadline compliance and correct indication of omitted enrichment.

B:

Freshness and authorization checks are essential alongside speed gains; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; this tests repeated-query latency with zero stale or cross-scope cache responses after updates; the actual release condition is deadline compliance and correct indication of omitted enrichment.

D:

Joint segmented reporting reveals whether expensive processing is spent where it helps; use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; this tests accuracy by difficulty segment and unit cost under p95 constraints; the actual release condition is deadline compliance and correct indication of omitted enrichment.

E:

Concurrency wins only if the final synthesis retains complete inputs; execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; this tests end-to-end p95 improvement with all required sources still present; the actual release condition is deadline compliance and correct indication of omitted enrichment.

Question 35

A mortgage processing organization is choosing between broad exploratory AI projects and a repetitive matching process with bounded exceptions. At a separate benefits office, managers want model recommendations but must preserve human authority over binding eligibility denials. Which TWO architecture decisions fit the respective needs? Choose TWO.

  1. Define which decisions remain with people
  2. Prioritize a repetitive step with bounded exceptions
  3. Measure the current manual task before choosing automation
  4. Translate vague accuracy demands into task-specific criteria
  5. Map available data and its permitted uses before solution selection

Explanation

Correct Answers

A:

Explicit decision-rights boundaries distinguish advisory generation from actions requiring a credentialed human or deterministic policy engine; an auditable decision trail demonstrates that AI outputs remained advisory where policy requires it; its corresponding workstream should be validated by share of binding decisions showing the required named approver and rationale.

B:

Select a high-frequency, well-described task with measurable work reduction and explicit escalation for ambiguous records; counting both throughput and escalations prevents a fast happy path from hiding a costly exception queue; its corresponding workstream should be validated by accepted cases per reviewer-hour with exception share tracked separately.

Incorrect Answers

C:

A baseline separates real business improvement from attractive demos; use completed cases, reviewer effort, errors, and exception handling as measurable comparators; process measurement establishes value, not the identity, data-permission, or runtime architecture of a Claude application; its natural acceptance measure is change in verified minutes and rework per adjudicated claim versus the documented baseline, which does not satisfy the separate workstreams being assessed.

D:

Convert vague requests into observable task outcomes, rubric-based quality dimensions, and failure thresholds that stakeholders can agree to test; a clear success rubric evaluates outcomes but cannot on its own implement retrieval access controls; its natural acceptance measure is agreement between domain reviewers on a rubric and pass rate against a labeled acceptance set, which does not satisfy the separate workstreams being assessed.

E:

Data-readiness and usage-rights mapping identifies legitimate inputs, owners, quality gaps, and prohibited transfers before committing to an implementation; a data inventory does not alone identify target SLOs or decide whether an orchestration loop is necessary; its natural acceptance measure is percentage of required fields with validated owners, quality checks, and approved processing purposes, which does not satisfy the separate workstreams being assessed.

Question 36

A deductions assistant has split eligibility checks from the client-facing narrative stage. The release test deliberately includes unsupported deductions. Which outcome would establish the prerequisite gate?

  1. Intermediate artifact validity and final score agreement with a policy-engine oracle
  2. Correct type selection on unseen evidence-versus-action pairs
  3. Label accuracy and token use versus a heavier example-based prompt
  4. Explanations generated only for records that passed the eligibility stage

Explanation

Correct Answer

D:

A prerequisite gate catches unvalidated reasoning before downstream text is produced; measure explanations generated only for records that passed the eligibility stage at the release gate; use a staged prompt chain where a verified intermediate classification is required before the next generation step.

Incorrect Answers

A:

Checkable milestones provide evidence of reasoning quality beyond rhetorical fluency; require explicit intermediate evidence and structured rule checks that can be independently validated; avoid treating verbose reasoning text as proof; this tests intermediate artifact validity and final score agreement with a policy-engine oracle; the actual release condition is explanations generated only for records that passed the eligibility stage.

B:

Contrastive tests verify the semantic boundary, not just the presence of a familiar template; show a paired positive and negative example with the decision rule that separates the formats; then test unseen cases; this tests correct type selection on unseen evidence-versus-action pairs; the actual release condition is explanations generated only for records that passed the eligibility stage.

C:

Controlled comparisons show whether added prompt complexity yields a genuine benefit; a concise task statement with precise labels is a defensible baseline when success is readily evaluated and examples add little value; this tests label accuracy and token use versus a heavier example-based prompt; the actual release condition is explanations generated only for records that passed the eligibility stage.

Question 37

A board sponsor needs to choose whether to fund a Claude rollout but receives a dense model-parameter presentation. Which choice most directly satisfies the documented goal?

  1. Keep a versioned decision and change history
  2. Present executive decisions through outcomes, cost, and risk
  3. Record alternatives and rationale in an architecture decision record
  4. Maintain a risk register with controls and accountable owners
  5. Demonstrate known limitations alongside successful workflows

Explanation

Correct Answer

B:

Summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; decision-focused review checks whether the audience can act on the information; evaluate the design with sponsor comprehension of expected outcomes, decision options, and risk ownership.

Incorrect Answers

A:

Maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; chat history and scattered slides rarely provide an authoritative current decision state; this alternative would measure correct retrieval of active decisions and their supersession chain during review; it does not establish sponsor comprehension of expected outcomes, decision options, and risk ownership for the described use case.

C:

Use an ADR recording context, considered options, decision, trade-offs, implications, and conditions for revisiting it; a final diagram alone often omits rejected alternatives and the reasoning behind the choice; this alternative would measure reconstructability of key design decisions after a simulated staff handoff; it does not establish sponsor comprehension of expected outcomes, decision options, and risk ownership for the described use case.

D:

Record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; a high-level statement that AI has risks does not create an actionable governance plan; this alternative would measure completion and ownership of high-severity mitigations before the release gate; it does not establish sponsor comprehension of expected outcomes, decision options, and risk ownership for the described use case.

E:

Use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; a demo focused solely on best cases can mask residual risks and create unrealistic expectations; this alternative would measure stakeholder identification of boundaries and fallback responsibilities after the demo; it does not establish sponsor comprehension of expected outcomes, decision options, and risk ownership for the described use case.

Question 38

A developer assistant reports a missing file, but fault injection shows the repository search operation sometimes times out. Which diagnostic outcome would establish that the team identifies the faulty component rather than compensating with a larger model?

  1. Correct identification of enforced deny source without allowing prohibited commands
  2. Bug reproduction on sanitized fixtures with no sensitive token disclosure
  3. Correct root-cause attribution under injected search timeouts
  4. Time to restore normal operation using an approved previous release
  5. Reproduction rate of sampled incidents from stored sanitized diagnostic metadata

Explanation

Correct Answer

C:

Controlled fault injection tests whether the team diagnoses the failing layer; measure correct root-cause attribution under injected search timeouts at the release gate; classify tool errors separately from model responses, validate return codes, and probe the failing integration before changing prompts.

Incorrect Answers

A:

Precedence tests prevent insecure workarounds that bypass governance intent; review organization, project, and local permission layers in the documented precedence order before weakening security controls; this tests correct identification of enforced deny source without allowing prohibited commands; the actual release condition is correct root-cause attribution under injected search timeouts.

B:

Safe replay demonstrates that privacy and diagnostic value can coexist; build redacted, access-controlled replay fixtures preserving relevant syntax and error conditions without unnecessary secrets or personal data; this tests bug reproduction on sanitized fixtures with no sensitive token disclosure; the actual release condition is correct root-cause attribution under injected search timeouts.

D:

A rollback drill verifies that recovery does not depend on the original developer being online; maintain versioned releases and documented rollback steps for hooks, prompts, model routes, and connector configuration; this tests time to restore normal operation using an approved previous release; the actual release condition is correct root-cause attribution under injected search timeouts.

E:

Reproducibility is a prerequisite for a trustworthy fix; collect privacy-safe traces, request IDs, environment details, and exact configuration revisions to establish a reproducible failure case; this tests reproduction rate of sampled incidents from stored sanitized diagnostic metadata; the actual release condition is correct root-cause attribution under injected search timeouts.

Question 39

A university personnel assistant must answer questions from a handbook whose authorized version changes each semester. Another HR team wants a fast, faithful summary of a transcript already supplied in the request, with no external operations. Which TWO patterns suit these different tasks? Choose TWO.

  1. Use a bounded tool-using agent for exploratory investigation
  2. Use a deterministic workflow for known ordered stages
  3. Ground dynamic policy answers with retrieval
  4. Use a single augmented-LLM call for a bounded text transformation
  5. Embed a small agent inside a controlled workflow

Explanation

Correct Answers

C:

Retrieval from a versioned, access-filtered source grounds answers in current documents and can expose the authoritative citation; version-aware citation checks show whether answers rely on authoritative current evidence; its corresponding workstream should be validated by correct citations to currently effective policy versions across update tests.

D:

For a fixed transformation, a prompt enriched with only needed context is simpler, cheaper, and easier to evaluate than an agentic loop; a bounded call should meet the content rubric without introducing an unnecessary action surface; its corresponding workstream should be validated by summary fidelity and p95 completion time without any unapproved tool invocation.

Incorrect Answers

A:

Bounded agentic planning chooses follow-up tools from observed evidence while enforcing time, budget, and tool permissions; agent autonomy adds cost and uncertainty to tasks whose steps are already predictable and linear; its natural acceptance measure is resolution of unseen diagnostic branches within a specified tool-call and time budget, which does not satisfy the separate workstreams being assessed.

B:

Explicit state transitions preserve required order, testing points, and deterministic branch conditions; Claude can supply judgment inside selected stages; a rigid sequence becomes cumbersome when the next action is genuinely unknown until exploratory results arrive; its natural acceptance measure is rate of recommendations backed by all required completed stage checks, which does not satisfy the separate workstreams being assessed.

E:

Place bounded exploratory reasoning inside one stage of a deterministic state machine; enforce approval and write permissions outside the agent; a fully autonomous controller is not needed when most transitions and side effects are governed by policy; its natural acceptance measure is disputed-case resolution with every refund still passing the identity and approval gates, which does not satisfy the separate workstreams being assessed.

Question 40

A staged extraction workflow is upgraded one component at a time while downstream integrations stay on their prior versions. Which measure shows that independent deployments preserve the expected contracts?

  1. Correct merged clause counts across repeated parallel execution races
  2. Contract test pass rate for independently upgraded stages and invalid payload rejection
  3. Stage-specific defect localization and repair without redoing previously validated stages
  4. Successful recovery after injected worker failures without duplicated writes or stages

Explanation

Correct Answer

B:

Versioned interfaces expose breaking changes before a multi-team deployment; measure contract test pass rate for independently upgraded stages and invalid payload rejection at the release gate; typed versioned contracts and compatibility tests let stages evolve while preserving their callers' assumptions.

Incorrect Answers

A:

Deterministic merge checks measure integrity rather than only wall-clock speed; isolate independent tasks, retain separate results, then join only when required dependencies have completed; this tests correct merged clause counts across repeated parallel execution races; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

C:

A traceable artifact boundary confines error analysis to the failed operation; separate stages with typed artifacts allow validation and targeted reruns at meaningful boundaries; this tests stage-specific defect localization and repair without redoing previously validated stages; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

D:

Replay testing verifies the precise recovery behavior the pipeline requires; durable checkpoints record validated intermediate artifacts and resume at safe boundaries with idempotent side effects; this tests successful recovery after injected worker failures without duplicated writes or stages; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

Claude Certified Architect - Professional (CCAR-P) Practice Tests