Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 54

Question 530

A payment-credit tool times out after committing a write. The caller retries without an operation key and double-posts the credit. Which state-management mechanism prevents this outcome?

  1. Capture reviewer corrections as labeled feedback
  2. Persist workflow state and make retries idempotent
  3. Link each output to source and processing trace IDs
  4. Use a queue and asynchronous workers for bursty workloads

Explanation

Correct Answer

B:

Persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; replay tests directly exercise the failure window that typically creates duplicate writes; verify the repair using repeated and interrupted test runs yielding exactly one committed credit per approved case.

Incorrect Answers

A:

A feedback dataset improves future evaluation; it does not guarantee that the current response is timely or properly permissioned; preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; that control is assessed by regression pass rate on previously corrected clause-omission cases; the incident instead requires repeated and interrupted test runs yielding exactly one committed credit per approved case.

C:

Traceability explains decisions after the fact; it is not a replacement for preventing unauthorized source access; correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review; that control is assessed by fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition; the incident instead requires repeated and interrupted test runs yielding exactly one committed credit per approved case.

D:

Queueing absorbs bursts but does not improve a weak answer rubric or justify a higher-privilege tool set; decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; that control is assessed by job completion rate and queue age during the peak intake window; the incident instead requires repeated and interrupted test runs yielding exactly one committed credit per approved case.

Question 531

An exact-string grader penalizes valid paraphrases and rewards templated but misleading wording. What is the most direct remediation?

  1. Preserve a separate holdout set for regression decisions
  2. Stratify a golden dataset by real workload segments
  3. Measure human annotator consistency before trusting labels
  4. Use expert rubrics for nuanced qualitative outputs

Explanation

Correct Answer

D:

Use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; a semantic rubric tests meaning while allowing legitimate variation in language; verify the repair using expert-rated factual fidelity and calibrated grader agreement on sampled explanations.

Incorrect Answers

A:

Reusing a public tuning set as the sole release gate invites overfitting rather than generalization; maintain untouched holdout cases and periodic refreshes representing real traffic, with clear version and leakage controls; that control is assessed by holdout quality stability across successive prompt versions; the incident instead requires expert-rated factual fidelity and calibrated grader agreement on sampled explanations.

B:

A random sample alone can underrepresent important low-frequency and high-consequence events; build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; that control is assessed by per-stratum pass rates with no uncovered critical scenario classes; the incident instead requires expert-rated factual fidelity and calibrated grader agreement on sampled explanations.

C:

More test volume does not fix unreliable or undefined labels; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; that control is assessed by inter-annotator agreement and unresolved-label rate after adjudication; the incident instead requires expert-rated factual fidelity and calibrated grader agreement on sampled explanations.

Question 532

A public-sector chatbot proposes to use records from multiple departments with different stewards. Which choice most directly satisfies the documented goal?

  1. Interview each role that owns a distinct outcome or risk
  2. Check risky assumptions through narrow experiments
  3. Capture external dependencies and acceptance risks explicitly
  4. Map the owner and authority of every required data source

Explanation

Correct Answer

D:

Create a source inventory with accountable owners, sensitivity, update rate, and approval dependencies before committing to the design; data ownership completeness measures feasibility before engineering spends effort; evaluate the design with percentage of required sources with approved owners, legal uses, and integration paths.

Incorrect Answers

A:

Use role-based discovery to capture goals, current workflows, authority boundaries, and failure impacts from relevant stakeholders; a sponsor's high-level vision does not fully describe operational controls or edge cases; this alternative would measure coverage of approved requirements traced to affected stakeholder groups; it does not establish percentage of required sources with approved owners, legal uses, and integration paths for the described use case.

B:

Identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; detailed architecture diagrams do not prove uncertain input quality or model capability; this alternative would measure resolution rate of prioritized unknowns through measured prototypes; it does not establish percentage of required sources with approved owners, legal uses, and integration paths for the described use case.

C:

Maintain a dependency register covering third-party SLAs, quotas, outage paths, and contractual conditions with mitigation owners; an AI model benchmark cannot predict every upstream system's production behavior; this alternative would measure verified readiness of high-impact external dependencies against test quotas and outage cases; it does not establish percentage of required sources with approved owners, legal uses, and integration paths for the described use case.

Question 533

A sourcing copilot consumes web pages containing instructions written by untrusted suppliers. Another research service runs collaborator-provided scripts that might access confidential files or external networks. Which TWO controls address document-level injection and execution isolation? Choose TWO.

  1. Enforce least privilege where tool actions execute
  2. Use controls at input, tool execution, and output boundaries
  3. Sandbox untrusted code and file operations
  4. Screen unsafe outputs in proportion to the use case
  5. Treat retrieved instructions as untrusted data

Explanation

Correct Answers

C:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; sandbox escapes and denial logs are more relevant than the assistant's stated intentions; its corresponding workstream should be validated by forbidden file and network access attempts blocked in execution tests.

E:

Keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; simulated hostile documents test the trust boundary directly; its corresponding workstream should be validated by attack success rate on adversarial web pages and zero unauthorized egress calls.

Incorrect Answers

A:

Enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; natural-language reminders do not remove capabilities exposed by the API or bearer token; its natural acceptance measure is zero payment writes when tested with the analytics identity and forged instructions, which does not satisfy the separate workstreams being assessed.

B:

Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; a single content filter cannot enforce account privileges or validate downstream business transactions; its natural acceptance measure is denial of unauthorized actions and harmful outputs across multi-stage red-team paths, which does not satisfy the separate workstreams being assessed.

D:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; an input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; its natural acceptance measure is harmful-completion detection on labeled outputs with false-positive monitoring, which does not satisfy the separate workstreams being assessed.

Question 534

A security report generator adopts shorter responses to save tokens, but the final artifact still requires every mandatory section. Which cost and completion measure would prevent false economies?

  1. Accepted resolution rate and cost per case across routed difficulty levels
  2. Supported-answer rate per input token on labeled audit questions
  3. Cost and successful throughput per nightly processing window
  4. All-in cost per accepted summary at a fixed mandatory-section completion rate

Explanation

Correct Answer

D:

Quality-constrained unit economics penalizes truncation and repeat work; measure all-in cost per accepted summary at a fixed mandatory-section completion rate at the release gate; optimize total cost per successful output with floor constraints on completeness, safety, and latency.

Incorrect Answers

A:

Stratified monitoring confirms economic gains do not displace expensive errors onto users; evaluate a smaller model on straightforward request strata and escalate only where measurable capability is needed; this tests accepted resolution rate and cost per case across routed difficulty levels; the actual release condition is all-in cost per accepted summary at a fixed mandatory-section completion rate.

B:

Evidence efficiency tests whether trimming preserves the information that matters; retrieve relevant excerpts and retain enough surrounding evidence for interpretation rather than indiscriminately filling the context window; this tests supported-answer rate per input token on labeled audit questions; the actual release condition is all-in cost per accepted summary at a fixed mandatory-section completion rate.

C:

Deadline-oriented throughput is the relevant performance objective for noninteractive work; use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense; this tests cost and successful throughput per nightly processing window; the actual release condition is all-in cost per accepted summary at a fixed mandatory-section completion rate.

Question 535

An indexed policy corpus has just received a new effective-date revision. A regression test asks questions whose correct citations recently changed. Which result would demonstrate that freshness, not prompt tweaks, fixes the fault?

  1. Correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations
  2. Correct 'unknown' handling and no false denials during injected API failures
  3. Recall of newly published policy passages and removal of stale hits
  4. Schema error rate across multiline, null, and unusual-field edge cases

Explanation

Correct Answer

C:

A retrieval-specific test isolates the component that changed; measure recall of newly published policy passages and removal of stale hits at the release gate; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning.

Incorrect Answers

A:

Controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; this tests correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the actual release condition is recall of newly published policy passages and removal of stale hits.

B:

Fault injection distinguishes tool availability from genuine absence of data; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; this tests correct 'unknown' handling and no false denials during injected API failures; the actual release condition is recall of newly published policy passages and removal of stale hits.

D:

Edge-specific parser tests identify contract failures separately from content comprehension; inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; this tests schema error rate across multiline, null, and unusual-field edge cases; the actual release condition is recall of newly published policy passages and removal of stale hits.

Question 536

A regulatory search system presents a narrow exception paragraph along with its defining parent section. Which comparison would demonstrate that parent expansion prevents misreading conditional language?

  1. Joint exact-identifier accuracy and semantic-symptom recall on mixed queries
  2. Recall on human-authored paraphrases absent from manual headings
  3. Conditional-rule accuracy with and without parent-document expansion
  4. Relevant-procedure rank and end-to-end answer support rate after reranking
  5. Exact-ID hit rate at the top retrieval ranks
  6. Exact aggregate agreement with a trusted query on authorized records

Explanation

Correct Answer

C:

Context reconstruction tests whether narrow hits are interpreted in their governing scope; measure conditional-rule accuracy with and without parent-document expansion at the release gate; use child-chunk retrieval for recall and fetch validated parent context or adjacent governing sections for interpretation.

Incorrect Answers

A:

The evaluation must verify both constraints at once rather than reporting only aggregate relevance; hybrid retrieval joins exact-term and semantic evidence, then applies metadata filtering and ranking; this tests joint exact-identifier accuracy and semantic-symptom recall on mixed queries; the actual release condition is conditional-rule accuracy with and without parent-document expansion.

B:

Paraphrase benchmark cases test the capability for which embeddings are chosen; vector retrieval can match conceptual similarity between user wording and document text, followed by evidence verification; this tests recall on human-authored paraphrases absent from manual headings; the actual release condition is conditional-rule accuracy with and without parent-document expansion.

D:

Rank-sensitive evaluation tests whether useful evidence reaches the final context window; apply a query-aware reranker after broad retrieval, respecting freshness and permissions, to prioritize evidence for synthesis; this tests relevant-procedure rank and end-to-end answer support rate after reranking; the actual release condition is conditional-rule accuracy with and without parent-document expansion.

E:

Identifier-focused tests reveal errors that a semantic relevance score may ignore; keyword or lexical matching is effective for codes, exact phrases, and terms whose characters carry the meaning; this tests exact-ID hit rate at the top retrieval ranks; the actual release condition is conditional-rule accuracy with and without parent-document expansion.

F:

A structured oracle detects missing rows and arithmetic mistakes; use a governed structured query interface with parameter checks and business semantics for exact aggregates; this tests exact aggregate agreement with a trusted query on authorized records; the actual release condition is conditional-rule accuracy with and without parent-document expansion.

Question 537

An assistant must react to approved document-publication events in a content-management system. Which intervention is most appropriate for this use case?

  1. Use MCP to expose reusable tools and resources across compatible clients
  2. Use direct APIs for a narrow stable application integration
  3. Use event notifications to drive state changes rather than constant polling
  4. Prefer managed server-side tools when appropriate to the platform

Explanation

Correct Answer

C:

An authenticated event or webhook integration can trigger bounded processing only when upstream state changes; event-driven tests verify timely updates without wasteful polling loops; evaluate the design with event-to-index update delay and duplicate-event idempotency under replay.

Incorrect Answers

A:

A maintained MCP server can standardize discoverable tool and resource contracts while each client still enforces authentication and trust; mCP is an interface protocol, not a substitute for access policy or secure server implementation; this alternative would measure cross-client contract compatibility and unauthorized-action denial for the same MCP service; it does not establish event-to-index update delay and duplicate-event idempotency under replay for the described use case.

B:

A direct typed API client is appropriate when the interface, arguments, and control flow are known and stable; direct API calls offer less automatic discovery when many heterogeneous tools must be used by several assistants; this alternative would measure integration error rate and p95 latency on the fixed status-check path; it does not establish event-to-index update delay and duplicate-event idempotency under replay for the described use case.

D:

A supported server-executed tool can offload execution to the provider while the application retains usage, privacy, and result-validation responsibilities; hosted tools do not automatically make external search content authoritative or safe from prompt injection; this alternative would measure search coverage, tool cost, and provenance checks with no application-owned execution endpoint; it does not establish event-to-index update delay and duplicate-event idempotency under replay for the described use case.

Question 538

A recommendation assistant spans search, model inference, and a business-data connector. During a simulated outage, which tracing result would demonstrate that operators can locate the actual failing stage?

  1. Measured safety precision/recall and legitimate-task completion before and after the change
  2. Quality delta between recent audited production samples and the launch baseline
  3. Percent of failures with a complete trace to the responsible stage
  4. Detection lag for injected per-request token inflation

Explanation

Correct Answer

C:

Trace completeness measures incident diagnosability rather than raw log volume; measure percent of failures with a complete trace to the responsible stage at the release gate; correlation and span IDs across components allow latency and failure attribution along the actual path.

Incorrect Answers

A:

Paired monitoring distinguishes beneficial safety controls from excessive refusals; monitor both harmful-output violations and false-positive blocks by version and user segment; this tests safety precision/recall and legitimate-task completion before and after the change; the actual release condition is percent of failures with a complete trace to the responsible stage.

B:

Live sampling reveals degradation that unchanged historical tests cannot see; periodically sample representative production outputs for labeling, segment analysis, and drift signals without retaining unnecessary personal data; this tests quality delta between recent audited production samples and the launch baseline; the actual release condition is percent of failures with a complete trace to the responsible stage.

D:

Unit-level telemetry catches runaway expense before invoices provide delayed evidence; monitor input/output tokens, cache reads, per-stage costs, and cost-per-task anomalies with accountable ownership; this tests detection lag for injected per-request token inflation; the actual release condition is percent of failures with a complete trace to the responsible stage.

Question 539

An organization has built one opaque prompt to scan legal records, extract facts, compare rules, and draft findings; defects are difficult to localize. A merger research service processes conflicting evidence across many independent filing sets. Which TWO decomposition approaches fit the needs? Choose TWO.

  1. Checkpoint long-running work for replay and recovery
  2. Parallelize only truly independent branches
  3. Partition research by source and reconcile conflicts explicitly
  4. Define input and output contracts for each subtask
  5. Divide work along independently verifiable artifacts

Explanation

Correct Answers

C:

Analyze source partitions separately, retain provenance, and reconcile contradictions through a dedicated synthesis stage; the final synthesis must expose disagreements rather than average them away; its corresponding workstream should be validated by conflict detection recall and citation coverage across independently processed source sets.

E:

Separate stages with typed artifacts allow validation and targeted reruns at meaningful boundaries; a traceable artifact boundary confines error analysis to the failed operation; its corresponding workstream should be validated by stage-specific defect localization and repair without redoing previously validated stages.

Incorrect Answers

A:

Durable checkpoints record validated intermediate artifacts and resume at safe boundaries with idempotent side effects; checkpointing reduces replay cost but cannot correct a defective classification rubric; its natural acceptance measure is successful recovery after injected worker failures without duplicated writes or stages, which does not satisfy the separate workstreams being assessed.

B:

Isolate independent tasks, retain separate results, then join only when required dependencies have completed; parallelizing dependent mutations without isolation can increase errors even if raw processing is faster; its natural acceptance measure is correct merged clause counts across repeated parallel execution races, which does not satisfy the separate workstreams being assessed.

D:

Typed versioned contracts and compatibility tests let stages evolve while preserving their callers' assumptions; contracts define allowed shapes and semantics but cannot ensure a source is truthful; its natural acceptance measure is contract test pass rate for independently upgraded stages and invalid payload rejection, which does not satisfy the separate workstreams being assessed.

Question 540

People reference obsolete decisions from a meeting note and approve incompatible changes. How should the design be corrected?

  1. Compare options on a shared multi-dimensional scorecard
  2. Demonstrate known limitations alongside successful workflows
  3. Present executive decisions through outcomes, cost, and risk
  4. Keep a versioned decision and change history

Explanation

Correct Answer

D:

Maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; version-aware retrieval prevents implementation against outdated agreements; verify the repair using correct retrieval of active decisions and their supersession chain during review.

Incorrect Answers

A:

Isolated benchmark rankings cannot settle a decision with several business constraints; present the same quality, latency, cost, risk, and maintainability measures for each candidate under agreed weighting; that control is assessed by stakeholder-approved selection using common scored evidence and disclosed trade-offs; the incident instead requires correct retrieval of active decisions and their supersession chain during review.

B:

A demo focused solely on best cases can mask residual risks and create unrealistic expectations; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; that control is assessed by stakeholder identification of boundaries and fallback responsibilities after the demo; the incident instead requires correct retrieval of active decisions and their supersession chain during review.

C:

A detailed API design is valuable to implementers but not sufficient for a funding decision; summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; that control is assessed by sponsor comprehension of expected outcomes, decision options, and risk ownership; the incident instead requires correct retrieval of active decisions and their supersession chain during review.

Claude Certified Architect - Professional (CCAR-P) Practice Tests