Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 25

Question 240

A catalog assistant receives many repeated public product questions while underlying SKUs change only at managed publish times. What is the most defensible next step?

  1. Use a safe limited answer when the deadline would otherwise be missed
  2. Cache safe, version-keyed retrieval responses
  3. Move slow operations into asynchronous jobs with progress states
  4. Route expensive reasoning to high-uncertainty cases
  5. Rerank a short candidate set for ambiguous retrieval queries

Explanation

Correct Answer

B:

Use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; freshness and authorization checks are essential alongside speed gains; evaluate the design with repeated-query latency with zero stale or cross-scope cache responses after updates.

Incorrect Answers

A:

Keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; this alternative would measure deadline compliance and correct indication of omitted enrichment; it does not establish repeated-query latency with zero stale or cross-scope cache responses after updates for the described use case.

C:

Acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; a longer synchronous timeout would still tie up request resources and expose users to unreliable waits; this alternative would measure job completion reliability and duplicate-submission rate during slow-system simulations; it does not establish repeated-query latency with zero stale or cross-scope cache responses after updates for the described use case.

D:

Use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; uniform deep reasoning does not add value to deterministic records and may increase response time; this alternative would measure accuracy by difficulty segment and unit cost under p95 constraints; it does not establish repeated-query latency with zero stale or cross-scope cache responses after updates for the described use case.

E:

Retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; taking every retrieved hit into context may raise token cost and distraction without raising answer quality; this alternative would measure grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths; it does not establish repeated-query latency with zero stale or cross-scope cache responses after updates for the described use case.

Question 241

A multi-tenant agent receives customer-supplied invoice IDs and forwards them to a backend with broad read authority. Which choice most directly satisfies the documented goal?

  1. Verify tenant ownership of resource identifiers
  2. Use delegated user identity for per-user data access
  3. Record the user, tenant, and action in audit trails
  4. Enforce authorization inside the tool backend

Explanation

Correct Answer

A:

Check the requested resource against the authenticated tenant and role at the data service rather than trusting user-provided identifiers; object-level access checks close the horizontal privilege gap that prompt filtering misses; evaluate the design with denial of cross-tenant identifier enumeration attempts in direct tool tests.

Incorrect Answers

B:

Carry an appropriate user-bound authorization context to downstream services and enforce record-level access per request; application-level authentication alone does not prove permission to read every record returned by a shared account; this alternative would measure zero cross-user CRM record disclosures in permission-matrix tests; it does not establish denial of cross-tenant identifier enumeration attempts in direct tool tests for the described use case.

C:

Propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records; logging only a model response cannot attribute which authenticated principal caused a backend change; this alternative would measure audit reconstruction of actor, authority, target, and result for sampled mutations; it does not establish denial of cross-tenant identifier enumeration attempts in direct tool tests for the described use case.

D:

Implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; prompt-only restrictions can be bypassed and are not an independently audited authorization control; this alternative would measure rejection of unauthorized tool calls even when sent outside the model workflow; it does not establish denial of cross-tenant identifier enumeration attempts in direct tool tests for the described use case.

Question 242

A product-information assistant is tested on discontinued plans and questions whose authoritative answer is absent. Which evidence shows that it avoids plausible but unsupported financial statements?

  1. Rate of user messages that fail to override defined task boundaries in regression tests
  2. Valid schema rate and zero unauthorized enum values reaching the consumer
  3. Out-of-scope refusal and handoff accuracy on a policy-boundary test set
  4. Unsupported-answer rate and appropriate abstention precision on no-answer cases

Explanation

Correct Answer

D:

An abstention test verifies whether uncertainty is handled safely without refusing answerable questions; measure unsupported-answer rate and appropriate abstention precision on no-answer cases at the release gate; tell the assistant when to identify missing evidence, ask for clarification, or escalate, and reinforce this with evidence validation.

Incorrect Answers

A:

Boundary-specific tests confirm that changing user content cannot rewrite the application contract; separate stable system-level policy from user content and tool results, retaining consistent privilege boundaries across requests; this tests rate of user messages that fail to override defined task boundaries in regression tests; the actual release condition is unsupported-answer rate and appropriate abstention precision on no-answer cases.

B:

Parser-level rejection tests verify the actual interface boundary; request a documented structured format and validate schema, allowed values, and business semantics before downstream use; this tests valid schema rate and zero unauthorized enum values reaching the consumer; the actual release condition is unsupported-answer rate and appropriate abstention precision on no-answer cases.

C:

Boundary tests demonstrate task scoping rather than only fluent responses on ordinary requests; define the assistant's permitted tasks, out-of-scope handling, and escalation behavior in stable application-level instructions, backed by independent access enforcement; this tests out-of-scope refusal and handoff accuracy on a policy-boundary test set; the actual release condition is unsupported-answer rate and appropriate abstention precision on no-answer cases.

Question 243

A user persuades an assistant to initiate a high-impact transaction that the service policy forbids. Which backend enforcement boundary must deny the unauthorized action regardless of the prompt?

  1. Verify tenant ownership of resource identifiers
  2. Record the user, tenant, and action in audit trails
  3. Issue short-lived scoped credentials for tools
  4. Enforce authorization inside the tool backend

Explanation

Correct Answer

D:

Implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; direct API tests distinguish true backend policy from model cooperation; verify the repair using rejection of unauthorized tool calls even when sent outside the model workflow.

Incorrect Answers

A:

Syntactically valid IDs and a logged-in session do not establish ownership of a resource; check the requested resource against the authenticated tenant and role at the data service rather than trusting user-provided identifiers; that control is assessed by denial of cross-tenant identifier enumeration attempts in direct tool tests; the incident instead requires rejection of unauthorized tool calls even when sent outside the model workflow.

B:

Logging only a model response cannot attribute which authenticated principal caused a backend change; propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records; that control is assessed by audit reconstruction of actor, authority, target, and result for sampled mutations; the incident instead requires rejection of unauthorized tool calls even when sent outside the model workflow.

C:

Encrypting a shared permanent key at rest does not mitigate broad privilege after the key is exposed; prefer tightly scoped, expiring credentials with secure rotation and audited issuance for the precise operation; that control is assessed by credential-expiry and scope-denial tests after simulated token compromise; the incident instead requires rejection of unauthorized tool calls even when sent outside the model workflow.

Question 244

A data-extraction service accepts documents with multiline comments, optional fields, and unusual punctuation. Which release result proves the output contract holds beyond typical samples?

  1. Correct 'unknown' handling and no false denials during injected API failures
  2. Correct labels on scenarios affected by the policy revision
  3. Schema error rate across multiline, null, and unusual-field edge cases
  4. Recall of newly published policy passages and removal of stale hits
  5. Supported-claim rate on source-checked summaries with seeded unsupported numbers
  6. Correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations

Explanation

Correct Answer

C:

Edge-specific parser tests identify contract failures separately from content comprehension; measure schema error rate across multiline, null, and unusual-field edge cases at the release gate; inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure.

Incorrect Answers

A:

Fault injection distinguishes tool availability from genuine absence of data; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; this tests correct 'unknown' handling and no false denials during injected API failures; the actual release condition is schema error rate across multiline, null, and unusual-field edge cases.

B:

A targeted regression set proves the prompt no longer anchors on obsolete examples; audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; this tests correct labels on scenarios affected by the policy revision; the actual release condition is schema error rate across multiline, null, and unusual-field edge cases.

D:

A retrieval-specific test isolates the component that changed; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; this tests recall of newly published policy passages and removal of stale hits; the actual release condition is schema error rate across multiline, null, and unusual-field edge cases.

E:

Claim-level verification targets hallucinations that ordinary coherence checks miss; compare material claims against cited evidence and flag unsupported statements for abstention or review; this tests supported-claim rate on source-checked summaries with seeded unsupported numbers; the actual release condition is schema error rate across multiline, null, and unusual-field edge cases.

F:

Controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; this tests correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the actual release condition is schema error rate across multiline, null, and unusual-field edge cases.

Question 245

A search assistant's p95 grows sharply after a new data connector is deployed. What should the solution architect recommend?

  1. Alert on unusual high-impact tool activity
  2. Separate retrieval, provider, and tool latency budgets
  3. Correlate model, retrieval, and tool spans in distributed traces
  4. Log structured tool outcomes without exposing sensitive payloads

Explanation

Correct Answer

B:

Instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; budget attribution points remediation at the stage that truly regressed; evaluate the design with stage-specific p95 deltas identifying injected bottlenecks.

Incorrect Answers

A:

Monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; general chat satisfaction metrics are too delayed and indirect to detect dangerous mutation spikes; this alternative would measure time to detect and stop a simulated unauthorized bulk-write burst; it does not establish stage-specific p95 deltas identifying injected bottlenecks for the described use case.

C:

End-to-end traces with consistent request and span identifiers identify which service and step consumed time; a model-only latency chart cannot explain retrieval or downstream network bottlenecks; this alternative would measure trace completeness and correct bottleneck attribution on instrumented fault injections; it does not establish stage-specific p95 deltas identifying injected bottlenecks for the described use case.

D:

Record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; full payload capture may help debugging but expands sensitive-data exposure and retention obligations; this alternative would measure error diagnosability with seeded sensitive fields absent from collected logs; it does not establish stage-specific p95 deltas identifying injected bottlenecks for the described use case.

Question 246

An audit team samples generated inspection decisions after several model and tool releases. It must be able to reconstruct the exact inputs and disposition of each sample. Which quality signal most directly proves traceability?

  1. Regression pass rate on previously corrected clause-omission cases
  2. Repeated and interrupted test runs yielding exactly one committed credit per approved case
  3. Fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition
  4. Job completion rate and queue age during the peak intake window
  5. Invalid write payloads rejected before execution, with zero partial supplier mutations
  6. Percentage of ingested documents meeting schema checks before any Claude call

Explanation

Correct Answer

C:

End-to-end provenance lets reviewers investigate disputed outcomes without assuming the final text is self-evidencing; measure fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition at the release gate; correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review.

Incorrect Answers

A:

Retesting concrete corrections detects whether a release reintroduces known business mistakes; preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; this tests regression pass rate on previously corrected clause-omission cases; the actual release condition is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

B:

Replay tests directly exercise the failure window that typically creates duplicate writes; persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; this tests repeated and interrupted test runs yielding exactly one committed credit per approved case; the actual release condition is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

D:

Measuring backlog and completion under load tests the exact burst tolerance the design promises; decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; this tests job completion rate and queue age during the peak intake window; the actual release condition is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

E:

Pre-write enforcement proves that the model cannot bypass the target system's integrity contract; use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; this tests invalid write payloads rejected before execution, with zero partial supplier mutations; the actual release condition is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

F:

A schema-conformance gate reveals extraction failures before they can contaminate reasoning; an input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; this tests percentage of ingested documents meeting schema checks before any Claude call; the actual release condition is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition.

Question 247

A repair manual query includes a precise serial family and a loosely described symptom. Which option addresses the immediate design need?

  1. Rerank candidate passages for question relevance
  2. Use lexical search for exact identifiers and literals
  3. Retrieve parent context after finding a specific chunk
  4. Combine lexical and semantic candidates when queries mix both
  5. Query structured data through authorized SQL or typed APIs

Explanation

Correct Answer

D:

Hybrid retrieval joins exact-term and semantic evidence, then applies metadata filtering and ranking; the evaluation must verify both constraints at once rather than reporting only aggregate relevance; evaluate the design with joint exact-identifier accuracy and semantic-symptom recall on mixed queries.

Incorrect Answers

A:

Apply a query-aware reranker after broad retrieval, respecting freshness and permissions, to prioritize evidence for synthesis; a bigger first-stage candidate set without selection can dilute context and increase cost; this alternative would measure relevant-procedure rank and end-to-end answer support rate after reranking; it does not establish joint exact-identifier accuracy and semantic-symptom recall on mixed queries for the described use case.

B:

Keyword or lexical matching is effective for codes, exact phrases, and terms whose characters carry the meaning; semantic similarity alone can rank conceptually related but operationally different identifiers; this alternative would measure exact-ID hit rate at the top retrieval ranks; it does not establish joint exact-identifier accuracy and semantic-symptom recall on mixed queries for the described use case.

C:

Use child-chunk retrieval for recall and fetch validated parent context or adjacent governing sections for interpretation; passing isolated small chunks may preserve relevance while losing the context needed for a correct conclusion; this alternative would measure conditional-rule accuracy with and without parent-document expansion; it does not establish joint exact-identifier accuracy and semantic-symptom recall on mixed queries for the described use case.

E:

Use a governed structured query interface with parameter checks and business semantics for exact aggregates; unstructured document retrieval cannot ensure completeness or mathematical correctness for database totals; this alternative would measure exact aggregate agreement with a trusted query on authorized records; it does not establish joint exact-identifier accuracy and semantic-symptom recall on mixed queries for the described use case.

Question 248

A development assistant loads task instructions only when the corresponding workflow is invoked. The team is comparing several unrelated coding tasks. Which evidence would show that selective instruction loading improves context efficiency safely?

  1. Cross-application prompt-version alignment and regression pass rate
  2. Task success with reduced irrelevant instruction tokens across mixed workflows
  3. Cache-read input tokens and end-to-end latency over repeated requests
  4. Net cost per accepted request across realistic interarrival times

Explanation

Correct Answer

B:

On-demand loading must retain quality while avoiding unnecessary context overhead; measure task success with reduced irrelevant instruction tokens across mixed workflows at the release gate; keep Skills discoverable and load task-relevant instructions or resources on demand, maintaining scope and permission constraints.

Incorrect Answers

A:

A shared module succeeds when consuming workflows inherit the same reviewed rule set; maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; this tests cross-application prompt-version alignment and regression pass rate; the actual release condition is task success with reduced irrelevant instruction tokens across mixed workflows.

C:

Cache-hit telemetry verifies the optimization rather than relying on estimated token savings; put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; this tests cache-read input tokens and end-to-end latency over repeated requests; the actual release condition is task success with reduced irrelevant instruction tokens across mixed workflows.

D:

Reuse-adjusted economics tests whether the caching decision pays off under actual traffic; assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; this tests net cost per accepted request across realistic interarrival times; the actual release condition is task success with reduced irrelevant instruction tokens across mixed workflows.

Question 249

Fixed-size splits separate an exception from the clause it modifies and the answer misstates applicability. What design correction resolves the reported failure?

  1. Chunk documents along meaningful structural boundaries
  2. Evaluate retrieval quality independently of generation
  3. Version embeddings and reindex when representation changes
  4. Preserve citation metadata through retrieval and synthesis
  5. Filter retrieval by access control before model exposure

Explanation

Correct Answer

A:

Chunk by headings and semantic sections with deliberate overlap where necessary while maintaining parent-document references; structure-aware test cases measure preservation of legally relevant context; verify the repair using retrieval of both governing clause and exception for labeled contract questions.

Incorrect Answers

B:

A stronger model cannot reliably cite evidence that retrieval never supplies; measure retrieval recall, ranking relevance, and evidence coverage on labeled query-document pairs before optimizing generation; that control is assessed by recall at k and evidence-coverage rate for known-answer queries; the incident instead requires retrieval of both governing clause and exception for labeled contract questions.

C:

Updating only the query encoder can corrupt retrieval ranking even if the documents are unchanged; record embedding model and preprocessing versions, re-embed consistently, and use controlled index migration; that control is assessed by retrieval recall before and after a complete embedding-index migration; the incident instead requires retrieval of both governing clause and exception for labeled contract questions.

D:

A similarity score is not a source identifier and cannot support a traceable citation; store source IDs, locations, dates, and stable links with chunks and propagate them to answer citation validation; that control is assessed by fraction of material claims linked to resolvable source locations; the incident instead requires retrieval of both governing clause and exception for labeled contract questions.

E:

A model instruction to ignore confidential items is not a data-access boundary once the text is supplied; apply document permissions to retrieval candidates before material enters model context or logs; that control is assessed by unauthorized chunk exposure rate under staff permission matrices, targeting zero; the incident instead requires retrieval of both governing clause and exception for labeled contract questions.

Question 250

A procurement assistant summarizes external vendor portals that contain arbitrary embedded prose. Which choice most directly satisfies the documented goal?

  1. Enforce least privilege where tool actions execute
  2. Screen unsafe outputs in proportion to the use case
  3. Treat retrieved instructions as untrusted data
  4. Sandbox untrusted code and file operations
  5. Apply controls at input, tool execution, and output boundaries

Explanation

Correct Answer

C:

Keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; simulated hostile documents test the trust boundary directly; evaluate the design with attack success rate on adversarial web pages and zero unauthorized egress calls.

Incorrect Answers

A:

Enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; natural-language reminders do not remove capabilities exposed by the API or bearer token; this alternative would measure zero payment writes when tested with the analytics identity and forged instructions; it does not establish attack success rate on adversarial web pages and zero unauthorized egress calls for the described use case.

B:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; an input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; this alternative would measure harmful-completion detection on labeled outputs with false-positive monitoring; it does not establish attack success rate on adversarial web pages and zero unauthorized egress calls for the described use case.

D:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; an instruction saying 'do not read secrets' cannot contain arbitrary code on an unrestricted host; this alternative would measure forbidden file and network access attempts blocked in execution tests; it does not establish attack success rate on adversarial web pages and zero unauthorized egress calls for the described use case.

E:

Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; a single content filter cannot enforce account privileges or validate downstream business transactions; this alternative would measure denial of unauthorized actions and harmful outputs across multi-stage red-team paths; it does not establish attack success rate on adversarial web pages and zero unauthorized egress calls for the described use case.

Claude Certified Architect - Professional (CCAR-P) Practice Tests