Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 50

Question 490

An analyst assistant should read reconciliation results but its connector also permits payment creation. What should the solution architect recommend?

  1. Allowlist generated fields and destinations before external calls
  2. Treat retrieved instructions as untrusted data
  3. Sandbox untrusted code and file operations
  4. Enforce least privilege where tool actions execute
  5. Screen unsafe outputs in proportion to the use case

Explanation

Correct Answer

D:

Enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; direct permission tests verify security independent of model obedience; evaluate the design with zero payment writes when tested with the analytics identity and forged instructions.

Incorrect Answers

A:

Validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; formatting the request as JSON does not make arbitrary field values safe to transmit; this alternative would measure zero unapproved recipients or fields accepted by the outbound service; it does not establish zero payment writes when tested with the analytics identity and forged instructions for the described use case.

B:

Keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; this alternative would measure attack success rate on adversarial web pages and zero unauthorized egress calls; it does not establish zero payment writes when tested with the analytics identity and forged instructions for the described use case.

C:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; an instruction saying 'do not read secrets' cannot contain arbitrary code on an unrestricted host; this alternative would measure forbidden file and network access attempts blocked in execution tests; it does not establish zero payment writes when tested with the analytics identity and forged instructions for the described use case.

E:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; an input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; this alternative would measure harmful-completion detection on labeled outputs with false-positive monitoring; it does not establish zero payment writes when tested with the analytics identity and forged instructions for the described use case.

Question 491

A project team finishes building an assistant but disbands before operational monitoring or budget ownership is assigned. Which approach best meets the stated requirement?

  1. Use bounded pilots with exit criteria before expansion
  2. Plan deprecation and rollback from the start
  3. Transfer durable ownership and operations before handoff
  4. Set an evidence gate before moving beyond discovery
  5. Run a recurring monitoring-to-improvement loop

Explanation

Correct Answer

C:

Establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; ownership evidence shows the service can be maintained after consultants leave; evaluate the design with named owner coverage and support-drill completion at handoff.

Incorrect Answers

A:

Define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; more launch enthusiasm does not replace production-like testing under load and edge conditions; this alternative would measure pilot gate pass rate with defined performance and safety thresholds; it does not establish named owner coverage and support-drill completion at handoff for the described use case.

B:

Inventory consumers, define migration dates and fallback, preserve data obligations, and test the rollback or retirement path; removing an old endpoint alone does not manage hidden dependencies or operational continuity; this alternative would measure successful retirement with zero unplanned dependent-system failures; it does not establish named owner coverage and support-drill completion at handoff for the described use case.

D:

Require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; a procurement timeline does not eliminate uncertainty about whether the problem is solvable; this alternative would measure closure of high-risk feasibility questions before design approval; it does not establish named owner coverage and support-drill completion at handoff for the described use case.

E:

Feed production samples and feedback into versioned evals, prioritize changes, and release improvements behind regression gates; a one-time launch test cannot guarantee reliability under changing input distributions; this alternative would measure cycle time from detected drift to validated improvement without safety regression; it does not establish named owner coverage and support-drill completion at handoff for the described use case.

Question 492

A retrieval pipeline sends access-restricted contracts into a reranker, then relies on a final prompt to hide them from the answer. At what boundary must the permission check occur?

  1. Chunk documents along meaningful structural boundaries
  2. Filter retrieval by access control before model exposure
  3. Refresh the index when source content changes
  4. Version embeddings and reindex when representation changes
  5. Preserve citation metadata through retrieval and synthesis

Explanation

Correct Answer

B:

Apply document permissions to retrieval candidates before material enters model context or logs; the security test must inspect retrieved inputs, not just final visible answers; verify the repair using unauthorized chunk exposure rate under staff permission matrices, targeting zero.

Incorrect Answers

A:

Arbitrary equal-length chunks may split dependencies despite fitting an embedding model's input limit; chunk by headings and semantic sections with deliberate overlap where necessary while maintaining parent-document references; that control is assessed by retrieval of both governing clause and exception for labeled contract questions; the incident instead requires unauthorized chunk exposure rate under staff permission matrices, targeting zero.

C:

A larger context window cannot compensate for documents missing from the index; build a monitored ingestion path for adds, updates, deletes, and effective dates, with freshness checks and removal of superseded chunks; that control is assessed by publication-to-search freshness lag and absence of withdrawn clauses in results; the incident instead requires unauthorized chunk exposure rate under staff permission matrices, targeting zero.

D:

Updating only the query encoder can corrupt retrieval ranking even if the documents are unchanged; record embedding model and preprocessing versions, re-embed consistently, and use controlled index migration; that control is assessed by retrieval recall before and after a complete embedding-index migration; the incident instead requires unauthorized chunk exposure rate under staff permission matrices, targeting zero.

E:

A similarity score is not a source identifier and cannot support a traceable citation; store source IDs, locations, dates, and stable links with chunks and propagate them to answer citation validation; that control is assessed by fraction of material claims linked to resolvable source locations; the incident instead requires unauthorized chunk exposure rate under staff permission matrices, targeting zero.

Question 493

A legal summarization assistant is being refined, but human changes disappear after a document is approved. What should the solution architect recommend?

  1. Persist workflow state and make retries idempotent
  2. Capture reviewer corrections as labeled feedback
  3. Validate model output against a business schema before side effects
  4. Use a queue and asynchronous workers for bursty workloads
  5. Normalize incoming documents into a validated input contract

Explanation

Correct Answer

B:

Preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; retesting concrete corrections detects whether a release reintroduces known business mistakes; evaluate the design with regression pass rate on previously corrected clause-omission cases.

Incorrect Answers

A:

Persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; durable-state control prevents duplicate effects but cannot decide whether a refund is substantively authorized; this alternative would measure repeated and interrupted test runs yielding exactly one committed credit per approved case; it does not establish regression pass rate on previously corrected clause-omission cases for the described use case.

C:

Use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; output parsing constrains generated data but does not by itself manage retry idempotency or transport durability; this alternative would measure invalid write payloads rejected before execution, with zero partial supplier mutations; it does not establish regression pass rate on previously corrected clause-omission cases for the described use case.

D:

Decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; queueing absorbs bursts but does not improve a weak answer rubric or justify a higher-privilege tool set; this alternative would measure job completion rate and queue age during the peak intake window; it does not establish regression pass rate on previously corrected clause-omission cases for the described use case.

E:

An input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; input normalization addresses upstream ambiguity, not the authorization needed before writing to a finance ledger; this alternative would measure percentage of ingested documents meeting schema checks before any Claude call; it does not establish regression pass rate on previously corrected clause-omission cases for the described use case.

Question 494

The model confidently guesses on unsupported cases while also escalating many straightforward cases. Which corrective change best addresses the root cause?

  1. Use task-correctness metrics tied to verified labels
  2. Track cost per verified successful task
  3. Measure rate and severity of safety policy violations
  4. Evaluate whether abstention and escalation are calibrated

Explanation

Correct Answer

D:

Measure selective risk: error rate on answered items alongside abstention precision, coverage, and escalation cost; coverage-risk curves expose the tradeoff between reliability and unnecessary handoffs; verify the repair using correct-answer rate at each supported coverage threshold including no-answer cases.

Incorrect Answers

A:

Positive conversational tone does not establish that a legally or financially meaningful label is correct; measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; that control is assessed by critical-class precision and recall on independently verified tax-code cases; the incident instead requires correct-answer rate at each supported coverage threshold including no-answer cases.

B:

Cost per token can improve while cost per correct business outcome deteriorates; relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; that control is assessed by fully loaded cost per correctly completed document; the incident instead requires correct-answer rate at each supported coverage threshold including no-answer cases.

C:

General task correctness does not measure policy-conformance failures on dangerous requests; define safety incident categories, risk-weighted violation rates, and appropriate exposure denominators for adversarial and real traffic; that control is assessed by severity-weighted policy violation incidence on a reviewed adversarial test set; the incident instead requires correct-answer rate at each supported coverage threshold including no-answer cases.

Question 495

An HR policy portal publishes new eligibility rules but the RAG index still contains last quarter's version. Which intervention is most appropriate for this use case?

  1. Preserve citation metadata through retrieval and synthesis
  2. Refresh the index when source content changes
  3. Version embeddings and reindex when representation changes
  4. Filter retrieval by access control before model exposure

Explanation

Correct Answer

B:

Build a monitored ingestion path for adds, updates, deletes, and effective dates, with freshness checks and removal of superseded chunks; update/delete tests prove the index tracks the authoritative corpus; evaluate the design with publication-to-search freshness lag and absence of withdrawn clauses in results.

Incorrect Answers

A:

Store source IDs, locations, dates, and stable links with chunks and propagate them to answer citation validation; a similarity score is not a source identifier and cannot support a traceable citation; this alternative would measure fraction of material claims linked to resolvable source locations; it does not establish publication-to-search freshness lag and absence of withdrawn clauses in results for the described use case.

C:

Record embedding model and preprocessing versions, re-embed consistently, and use controlled index migration; updating only the query encoder can corrupt retrieval ranking even if the documents are unchanged; this alternative would measure retrieval recall before and after a complete embedding-index migration; it does not establish publication-to-search freshness lag and absence of withdrawn clauses in results for the described use case.

D:

Apply document permissions to retrieval candidates before material enters model context or logs; a model instruction to ignore confidential items is not a data-access boundary once the text is supplied; this alternative would measure unauthorized chunk exposure rate under staff permission matrices, targeting zero; it does not establish publication-to-search freshness lag and absence of withdrawn clauses in results for the described use case.

Question 496

A fraud monitoring console must respond within a short deadline, but a secondary explanation service is intermittently slow. Which intervention is most appropriate for this use case?

  1. Rerank a short candidate set for ambiguous retrieval queries
  2. Cache safe, version-keyed retrieval responses
  3. Parallelize independent read-only tool calls
  4. Use a safe limited answer when the deadline would otherwise be missed
  5. Route expensive reasoning to high-uncertainty cases

Explanation

Correct Answer

D:

Keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; timed fault injection confirms graceful degradation without inventing evidence; evaluate the design with deadline compliance and correct indication of omitted enrichment.

Incorrect Answers

A:

Retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; taking every retrieved hit into context may raise token cost and distraction without raising answer quality; this alternative would measure grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths; it does not establish deadline compliance and correct indication of omitted enrichment for the described use case.

B:

Use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; an unversioned cache can improve speed while silently sacrificing freshness and access correctness; this alternative would measure repeated-query latency with zero stale or cross-scope cache responses after updates; it does not establish deadline compliance and correct indication of omitted enrichment for the described use case.

C:

Execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; parallel execution offers no benefit when one call requires another's output or modifies shared state; this alternative would measure end-to-end p95 improvement with all required sources still present; it does not establish deadline compliance and correct indication of omitted enrichment for the described use case.

E:

Use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; uniform deep reasoning does not add value to deterministic records and may increase response time; this alternative would measure accuracy by difficulty segment and unit cost under p95 constraints; it does not establish deadline compliance and correct indication of omitted enrichment for the described use case.

Question 497

An operational feedback program now converts reported assistant defects into tracked engineering work. Which measure would show that actionable reports move through ownership, verification, and closure promptly?

  1. Delivery of agreed business and risk indicators on schedule with decision-ready interpretation
  2. SLO compliance and prescribed actions during a simulated error-budget breach
  3. Time to appropriate escalation and fallback activation during drills
  4. Time from actionable feedback to triage, owner assignment, and verified fix

Explanation

Correct Answer

D:

Issue lifecycle metrics show whether feedback produces controlled improvement; measure time from actionable feedback to triage, owner assignment, and verified fix at the release gate; classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests.

Incorrect Answers

A:

Reporting completeness lets stakeholders govern the system using relevant evidence; agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; this tests delivery of agreed business and risk indicators on schedule with decision-ready interpretation; the actual release condition is time from actionable feedback to triage, owner assignment, and verified fix.

B:

Operational SLO tests connect observed service quality to response commitments; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; this tests sLO compliance and prescribed actions during a simulated error-budget breach; the actual release condition is time from actionable feedback to triage, owner assignment, and verified fix.

C:

Incident response tests verify that operational ownership works under pressure; define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; this tests time to appropriate escalation and fallback activation during drills; the actual release condition is time from actionable feedback to triage, owner assignment, and verified fix.

Question 498

The same contradictory clauses are available to Claude, but the answer repeatedly ignores a crucial dependency. Which corrective change best addresses the root cause?

  1. Test for model capability mismatch on complex reasoning
  2. Investigate retrieval recall before changing the language model
  3. Revalidate few-shot examples after policy changes
  4. Decouple external tool faults from model decisions

Explanation

Correct Answer

A:

Hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; verify the repair using correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations.

Incorrect Answers

B:

Model upgrades cannot recover information that never reaches the answer context; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; that control is assessed by recall of newly published policy passages and removal of stale hits; the incident instead requires correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations.

C:

Index re-embedding cannot resolve a stale rule embedded directly in the prompt template; audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; that control is assessed by correct labels on scenarios affected by the policy revision; the incident instead requires correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations.

D:

Prompt changes alone cannot restore a failing service or make an empty placeholder a valid business result; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; that control is assessed by correct 'unknown' handling and no false denials during injected API failures; the incident instead requires correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations.

Question 499

A support assistant has a manual continuation path for regional incidents. In a drill that makes automated decisions unreliable, which timing result would prove that escalation and fallback responsibilities are operational?

  1. Affected-user awareness and support-ticket volume following staged changes
  2. SLO compliance and prescribed actions during a simulated error-budget breach
  3. Time from actionable feedback to triage, owner assignment, and verified fix
  4. Time to appropriate escalation and fallback activation during drills

Explanation

Correct Answer

D:

Incident response tests verify that operational ownership works under pressure; measure time to appropriate escalation and fallback activation during drills at the release gate; define incident severity, on-call ownership, manual fallback authority, and customer communication triggers.

Incorrect Answers

A:

Communication quality is reflected in predictable adoption and fewer avoidable surprises; define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; this tests affected-user awareness and support-ticket volume following staged changes; the actual release condition is time to appropriate escalation and fallback activation during drills.

B:

Operational SLO tests connect observed service quality to response commitments; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; this tests sLO compliance and prescribed actions during a simulated error-budget breach; the actual release condition is time to appropriate escalation and fallback activation during drills.

C:

Issue lifecycle metrics show whether feedback produces controlled improvement; classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; this tests time from actionable feedback to triage, owner assignment, and verified fix; the actual release condition is time to appropriate escalation and fallback activation during drills.

Question 500

A service team receives scattered negative ratings that may signal retrieval, tone, or permission problems. Which choice most directly satisfies the documented goal?

  1. Define a documented escalation path for service incidents
  2. Route user feedback into a triaged, owned backlog
  3. Define SLOs and error budgets for agreed user outcomes
  4. Communicate release behavior changes before rollout
  5. Set observability and reporting expectations with stakeholders
  6. Clarify shared responsibility for the service boundary

Explanation

Correct Answer

B:

Classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; issue lifecycle metrics show whether feedback produces controlled improvement; evaluate the design with time from actionable feedback to triage, owner assignment, and verified fix.

Incorrect Answers

A:

Define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; generic 'contact support' guidance lacks accountable timing and action for critical failures; this alternative would measure time to appropriate escalation and fallback activation during drills; it does not establish time from actionable feedback to triage, owner assignment, and verified fix for the described use case.

C:

Establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; generic uptime claims can conceal slow or unusable answer experiences; this alternative would measure sLO compliance and prescribed actions during a simulated error-budget breach; it does not establish time from actionable feedback to triage, owner assignment, and verified fix for the described use case.

D:

Define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; quiet deployment may reduce coordination overhead but damages trust when workflows or permissions change; this alternative would measure affected-user awareness and support-ticket volume following staged changes; it does not establish time from actionable feedback to triage, owner assignment, and verified fix for the described use case.

E:

Agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; platform telemetry is necessary but not equivalent to business service reporting; this alternative would measure delivery of agreed business and risk indicators on schedule with decision-ready interpretation; it does not establish time from actionable feedback to triage, owner assignment, and verified fix for the described use case.

F:

Define provider, platform, application, security, and business-owner responsibilities in an explicit operating model; a vendor SLA typically does not assign responsibility for every custom integration and decision rule; this alternative would measure ownership coverage for failures across model, application, data, and tool layers; it does not establish time from actionable feedback to triage, owner assignment, and verified fix for the described use case.

Claude Certified Architect - Professional (CCAR-P) Practice Tests