Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 46

Question 450

The model treats a narrow exception as a blanket rule because the surrounding conditions were omitted. Which action best removes the underlying failure mode?

  1. Combine lexical and semantic candidates when queries mix both
  2. Adopt semantic vector search for paraphrased knowledge questions
  3. Query structured data through authorized SQL or typed APIs
  4. Rerank candidate passages for question relevance
  5. Retrieve parent context after finding a specific chunk

Explanation

Correct Answer

E:

Use child-chunk retrieval for recall and fetch validated parent context or adjacent governing sections for interpretation; context reconstruction tests whether narrow hits are interpreted in their governing scope; verify the repair using conditional-rule accuracy with and without parent-document expansion.

Incorrect Answers

A:

Either modality alone leaves a known class of mixed-format queries uncovered; hybrid retrieval joins exact-term and semantic evidence, then applies metadata filtering and ranking; that control is assessed by joint exact-identifier accuracy and semantic-symptom recall on mixed queries; the incident instead requires conditional-rule accuracy with and without parent-document expansion.

B:

Semantic search is not guaranteed to respect exact version strings, numerals, or access boundaries without additional controls; vector retrieval can match conceptual similarity between user wording and document text, followed by evidence verification; that control is assessed by recall on human-authored paraphrases absent from manual headings; the incident instead requires conditional-rule accuracy with and without parent-document expansion.

C:

Unstructured document retrieval cannot ensure completeness or mathematical correctness for database totals; use a governed structured query interface with parameter checks and business semantics for exact aggregates; that control is assessed by exact aggregate agreement with a trusted query on authorized records; the incident instead requires conditional-rule accuracy with and without parent-document expansion.

D:

A bigger first-stage candidate set without selection can dilute context and increase cost; apply a query-aware reranker after broad retrieval, respecting freshness and permissions, to prioritize evidence for synthesis; that control is assessed by relevant-procedure rank and end-to-end answer support rate after reranking; the incident instead requires conditional-rule accuracy with and without parent-document expansion.

Question 451

The assistant rewrites findings into recommendations even when the requested output is an evidence log. What design correction resolves the reported failure?

  1. Use contrastive examples when two outputs look similar
  2. Introduce few-shot examples for difficult boundary cases
  3. Chain prompts when one result is a prerequisite for another
  4. Break multi-step reasoning into checkable intermediate outputs

Explanation

Correct Answer

A:

Show a paired positive and negative example with the decision rule that separates the formats; then test unseen cases; contrastive tests verify the semantic boundary, not just the presence of a familiar template; verify the repair using correct type selection on unseen evidence-versus-action pairs.

Incorrect Answers

B:

Examples are not a substitute for a clean class definition or representative test labels; provide a few diverse, accurately labeled counterexamples that illustrate the exact class boundary, then verify against held-out cases; that control is assessed by recall on previously confused boundary examples outside the prompt set; the incident instead requires correct type selection on unseen evidence-versus-action pairs.

C:

Chaining every trivial substep increases latency without improving reliability for simple independent tasks; use a staged prompt chain where a verified intermediate classification is required before the next generation step; that control is assessed by explanations generated only for records that passed the eligibility stage; the incident instead requires correct type selection on unseen evidence-versus-action pairs.

D:

Requesting longer explanations without validation can make unsupported conclusions appear more persuasive; require explicit intermediate evidence and structured rule checks that can be independently validated; avoid treating verbose reasoning text as proof; that control is assessed by intermediate artifact validity and final score agreement with a policy-engine oracle; the incident instead requires correct type selection on unseen evidence-versus-action pairs.

Question 452

A service desk agent has three overlapping ticket-update connectors, including a retired endpoint with outdated status values. Which approach best meets the stated requirement?

  1. Remove duplicate and deprecated tools from the catalog
  2. Offer each agent only the tools required for its role
  3. Stage privileged capabilities only after explicit intent confirmation
  4. Audit tool manifests as versioned production configuration

Explanation

Correct Answer

A:

Consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; removing ambiguity should reduce routing variance without reducing supported tasks; evaluate the design with single correct connector selection and consistent ticket states across repeated requests.

Incorrect Answers

B:

Give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; monitoring a broad tool catalog does not remove the authority to invoke dangerous operations; this alternative would measure unauthorized tool availability and invocation count in hostile transcript tests; it does not establish single correct connector selection and consistent ticket states across repeated requests for the described use case.

C:

Reveal high-impact tools only within a confirmed, authorized transaction stage; maintain server-side gating independently of model choice; hiding a tool in ordinary prompts is a defense-in-depth measure, not sufficient if APIs permit unauthorized direct calls; this alternative would measure absence of privileged tool calls before the confirmed transaction stage; it does not establish single correct connector selection and consistent ticket states across repeated requests for the described use case.

D:

Maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; ad hoc tool registration lacks the inventory needed to catch capability expansion before deployment; this alternative would measure changes rejected when requested tool scopes exceed the approved task manifest; it does not establish single correct connector selection and consistent ticket states across repeated requests for the described use case.

Question 453

A commerce search cache is being tested across product withdrawals, entitlement changes, and repeated popular queries. Which outcome would prove its speed benefit does not introduce stale or cross-account answers?

  1. Job completion reliability and duplicate-submission rate during slow-system simulations
  2. Accuracy by difficulty segment and unit cost under p95 constraints
  3. Deadline compliance and correct indication of omitted enrichment
  4. Repeated-query latency with zero stale or cross-scope cache responses after updates

Explanation

Correct Answer

D:

Freshness and authorization checks are essential alongside speed gains; measure repeated-query latency with zero stale or cross-scope cache responses after updates at the release gate; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates.

Incorrect Answers

A:

Durable job identity decouples user interaction time from background work duration; acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; this tests job completion reliability and duplicate-submission rate during slow-system simulations; the actual release condition is repeated-query latency with zero stale or cross-scope cache responses after updates.

B:

Joint segmented reporting reveals whether expensive processing is spent where it helps; use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; this tests accuracy by difficulty segment and unit cost under p95 constraints; the actual release condition is repeated-query latency with zero stale or cross-scope cache responses after updates.

C:

Timed fault injection confirms graceful degradation without inventing evidence; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; this tests deadline compliance and correct indication of omitted enrichment; the actual release condition is repeated-query latency with zero stale or cross-scope cache responses after updates.

Question 454

A workflow assistant prepares outbound customer notifications but the delivery service now validates each recipient and data field. Which result would prove that generated requests cannot expand the approved disclosure scope?

  1. Attack success rate on adversarial web pages and zero unauthorized egress calls
  2. Zero payment writes when tested with the analytics identity and forged instructions
  3. Zero unapproved recipients or fields accepted by the outbound service
  4. Forbidden file and network access attempts blocked in execution tests
  5. Denial of unauthorized actions and harmful outputs across multi-stage red-team paths

Explanation

Correct Answer

C:

Contract-level checks demonstrate control over the actual data leaving the system; measure zero unapproved recipients or fields accepted by the outbound service at the release gate; validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract.

Incorrect Answers

A:

Simulated hostile documents test the trust boundary directly; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; this tests attack success rate on adversarial web pages and zero unauthorized egress calls; the actual release condition is zero unapproved recipients or fields accepted by the outbound service.

B:

Direct permission tests verify security independent of model obedience; enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; this tests zero payment writes when tested with the analytics identity and forged instructions; the actual release condition is zero unapproved recipients or fields accepted by the outbound service.

D:

Sandbox escapes and denial logs are more relevant than the assistant's stated intentions; isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; this tests forbidden file and network access attempts blocked in execution tests; the actual release condition is zero unapproved recipients or fields accepted by the outbound service.

E:

End-to-end adversarial cases assess the combined defenses and their failure isolation; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; this tests denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the actual release condition is zero unapproved recipients or fields accepted by the outbound service.

Question 455

An explanatory chat message is misinterpreted as permission to isolate a critical subnet. What design correction resolves the reported failure?

  1. Require separate approval for high-impact mutations
  2. Issue short-lived scoped credentials for tools
  3. Enforce authorization inside the tool backend
  4. Verify tenant ownership of resource identifiers
  5. Record the user, tenant, and action in audit trails

Explanation

Correct Answer

A:

Bind high-impact actions to a separately verified approval and narrowly scoped execution credential; approval-path tests ensure the precise mutation was authorized before execution; verify the repair using zero critical network changes without a recorded action-specific approval.

Incorrect Answers

B:

Encrypting a shared permanent key at rest does not mitigate broad privilege after the key is exposed; prefer tightly scoped, expiring credentials with secure rotation and audited issuance for the precise operation; that control is assessed by credential-expiry and scope-denial tests after simulated token compromise; the incident instead requires zero critical network changes without a recorded action-specific approval.

C:

Prompt-only restrictions can be bypassed and are not an independently audited authorization control; implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; that control is assessed by rejection of unauthorized tool calls even when sent outside the model workflow; the incident instead requires zero critical network changes without a recorded action-specific approval.

D:

Syntactically valid IDs and a logged-in session do not establish ownership of a resource; check the requested resource against the authenticated tenant and role at the data service rather than trusting user-provided identifiers; that control is assessed by denial of cross-tenant identifier enumeration attempts in direct tool tests; the incident instead requires zero critical network changes without a recorded action-specific approval.

E:

Logging only a model response cannot attribute which authenticated principal caused a backend change; propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records; that control is assessed by audit reconstruction of actor, authority, target, and result for sampled mutations; the incident instead requires zero critical network changes without a recorded action-specific approval.

Question 456

A credit-support assistant provides recommendations that people might mistake for binding eligibility decisions. What should the solution architect recommend?

  1. Check accessibility of generated explanations and interfaces
  2. Measure errors across relevant user subgroups
  3. Publish scope, limitations, and residual risks to stakeholders
  4. Design an override path proportionate to consequences
  5. Expose evidence and decision boundaries to affected users

Explanation

Correct Answer

E:

Distinguish evidence, uncertainty, and advisory status with accessible citations and a path to authoritative decisions; transparent decision tests measure whether users understand what the system did and did not decide; evaluate the design with user comprehension of advisory status and source-citation validity.

Incorrect Answers

A:

Include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; factual accuracy alone does not make the interface usable to everyone; this alternative would measure task completion by users of tested assistive technologies on representative flows; it does not establish user comprehension of advisory status and source-citation validity for the described use case.

B:

Evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; overall accuracy cannot establish comparable treatment for every affected population; this alternative would measure quality and escalation parity across predeclared language segments; it does not establish user comprehension of advisory status and source-citation validity for the described use case.

C:

Maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; a generic disclaimer cannot compensate for hidden high-risk use or unavailable oversight; this alternative would measure accuracy of user understanding of supported versus unsupported tasks; it does not establish user comprehension of advisory status and source-citation validity for the described use case.

D:

Preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; human review after an irreversible action may be too late to prevent harm; this alternative would measure successful intervention before seeded high-impact incorrect actions are committed; it does not establish user comprehension of advisory status and source-citation validity for the described use case.

Question 457

A reimbursement assistant can draft settlement amounts but must not disburse funds autonomously. Which intervention is most appropriate for this use case?

  1. Require named authorization before moving money
  2. Capture reviewer overrides as structured improvement signals
  3. Require stronger approval paths for exceptional high-impact actions
  4. Give reviewers only necessary sensitive information

Explanation

Correct Answer

A:

Separate recommendation from execution and require an authenticated human approval bound to the exact amount, payee, and transaction; payment simulation tests prove approvals are bound to the exact side effect; evaluate the design with zero disbursements without an auditable transaction-specific approver.

Incorrect Answers

B:

Record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign; counting approval clicks alone does not describe why humans disagreed with the AI; this alternative would measure recurrence rate of categorized, previously corrected error types; it does not establish zero disbursements without an auditable transaction-specific approver for the described use case.

C:

Apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations; a uniform low-friction approval does not reflect differing blast radius and operational risk; this alternative would measure critical-action dual-approval compliance and zero bypasses during staged tests; it does not establish zero disbursements without an auditable transaction-specific approver for the described use case.

D:

Minimize, redact, and scope information in review interfaces according to purpose and authorized reviewer role; human oversight is not permission to expand access to unrelated personal information; this alternative would measure unauthorized field exposure and successful review accuracy after data minimization; it does not establish zero disbursements without an auditable transaction-specific approver for the described use case.

Question 458

An account assistant has a legitimate write tool but should rarely perform bulk changes. What action best fits the business and technical boundary?

  1. Alert on unusual high-impact tool activity
  2. Segment reliability metrics by tenant and workload
  3. Separate retrieval, provider, and tool latency budgets
  4. Correlate model, retrieval, and tool spans in distributed traces

Explanation

Correct Answer

A:

Monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; action telemetry and response drills measure whether harmful changes can be contained promptly; evaluate the design with time to detect and stop a simulated unauthorized bulk-write burst.

Incorrect Answers

B:

Break down latency, errors, and tool success by permission-safe tenant and workload dimensions with cardinality controls; overall averages can hide concentrated failures without meaningful dimensional analysis; this alternative would measure detection time for an injected single-tenant degradation; it does not establish time to detect and stop a simulated unauthorized bulk-write burst for the described use case.

C:

Instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; optimizing generation tokens cannot repair a slow connector that dominates the critical path; this alternative would measure stage-specific p95 deltas identifying injected bottlenecks; it does not establish time to detect and stop a simulated unauthorized bulk-write burst for the described use case.

D:

End-to-end traces with consistent request and span identifiers identify which service and step consumed time; a model-only latency chart cannot explain retrieval or downstream network bottlenecks; this alternative would measure trace completeness and correct bottleneck attribution on instrumented fault injections; it does not establish time to detect and stop a simulated unauthorized bulk-write burst for the described use case.

Question 459

A loan-review rubric has subject-matter experts disagreeing about borderline case classifications. What is the most targeted architecture choice?

  1. Protect a separate holdout set for regression decisions
  2. Stratify a golden dataset by real workload segments
  3. Use expert rubrics for nuanced qualitative outputs
  4. Measure human annotator consistency before trusting labels

Explanation

Correct Answer

D:

Use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; label reliability is necessary before model score differences can be interpreted; evaluate the design with inter-annotator agreement and unresolved-label rate after adjudication.

Incorrect Answers

A:

Maintain untouched holdout cases and periodic refreshes representing real traffic, with clear version and leakage controls; reusing a public tuning set as the sole release gate invites overfitting rather than generalization; this alternative would measure holdout quality stability across successive prompt versions; it does not establish inter-annotator agreement and unresolved-label rate after adjudication for the described use case.

B:

Build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; a random sample alone can underrepresent important low-frequency and high-consequence events; this alternative would measure per-stratum pass rates with no uncovered critical scenario classes; it does not establish inter-annotator agreement and unresolved-label rate after adjudication for the described use case.

C:

Use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; pure string comparison is unsuitable when many substantively correct phrasings exist; this alternative would measure expert-rated factual fidelity and calibrated grader agreement on sampled explanations; it does not establish inter-annotator agreement and unresolved-label rate after adjudication for the described use case.

Question 460

A malformed generated field reaches a write endpoint and is rejected after an unintended partial update. Which change should the architect recommend first?

  1. Use a queue and asynchronous workers for bursty workloads
  2. Persist workflow state and make retries idempotent
  3. Link each output to source and processing trace IDs
  4. Validate model output against a business schema before side effects

Explanation

Correct Answer

D:

Use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; pre-write enforcement proves that the model cannot bypass the target system's integrity contract; verify the repair using invalid write payloads rejected before execution, with zero partial supplier mutations.

Incorrect Answers

A:

Queueing absorbs bursts but does not improve a weak answer rubric or justify a higher-privilege tool set; decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; that control is assessed by job completion rate and queue age during the peak intake window; the incident instead requires invalid write payloads rejected before execution, with zero partial supplier mutations.

B:

Durable-state control prevents duplicate effects but cannot decide whether a refund is substantively authorized; persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; that control is assessed by repeated and interrupted test runs yielding exactly one committed credit per approved case; the incident instead requires invalid write payloads rejected before execution, with zero partial supplier mutations.

C:

Traceability explains decisions after the fact; it is not a replacement for preventing unauthorized source access; correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review; that control is assessed by fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition; the incident instead requires invalid write payloads rejected before execution, with zero partial supplier mutations.

Claude Certified Architect - Professional (CCAR-P) Practice Tests