Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 8

Question 70

A benefits assistant supports several languages, but an audit suspects one language has disproportionate guidance errors. Which measurement would reveal whether quality and escalation are equitable across supported groups?

  1. Time to resolve substantiated challenges and proportion of cases with documented review
  2. Quality and escalation parity across predeclared language segments
  3. Task completion by users of tested assistive technologies on representative flows
  4. Accuracy of user understanding of supported versus unsupported tasks
  5. Successful intervention before seeded high-impact incorrect actions are committed

Explanation

Correct Answer

B:

Group-specific evaluation reveals concentrated harm masked by pooled scores; measure quality and escalation parity across predeclared language segments at the release gate; evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review.

Incorrect Answers

A:

Remedy effectiveness measures whether affected people can obtain meaningful reconsideration; provide a contest and correction process with accountable human ownership for significant impacts; this tests time to resolve substantiated challenges and proportion of cases with documented review; the actual release condition is quality and escalation parity across predeclared language segments.

C:

Direct accessibility testing measures practical use rather than formal content correctness only; include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; this tests task completion by users of tested assistive technologies on representative flows; the actual release condition is quality and escalation parity across predeclared language segments.

D:

Scenario tests reveal whether disclosures prevent inappropriate reliance; maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; this tests accuracy of user understanding of supported versus unsupported tasks; the actual release condition is quality and escalation parity across predeclared language segments.

E:

Pre-commit override drills test real controllability rather than retrospective documentation; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; this tests successful intervention before seeded high-impact incorrect actions are committed; the actual release condition is quality and escalation parity across predeclared language segments.

Question 71

A compliance copilot summarizes regulations for an operations team but may encounter ambiguous jurisdictional requirements. What is the most defensible next step?

  1. Give reviewers only necessary sensitive information
  2. Escalate binding legal interpretations to qualified reviewers
  3. Capture reviewer overrides as structured improvement signals
  4. Require stronger approval paths for exceptional high-impact actions

Explanation

Correct Answer

B:

Define escalation thresholds and route consequential interpretations to qualified legal or compliance owners with source context; a consequence-aware test checks whether unsupported authoritative decisions are avoided; evaluate the design with correct escalation of ambiguous legal cases with reviewer attribution.

Incorrect Answers

A:

Minimize, redact, and scope information in review interfaces according to purpose and authorized reviewer role; human oversight is not permission to expand access to unrelated personal information; this alternative would measure unauthorized field exposure and successful review accuracy after data minimization; it does not establish correct escalation of ambiguous legal cases with reviewer attribution for the described use case.

C:

Record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign; counting approval clicks alone does not describe why humans disagreed with the AI; this alternative would measure recurrence rate of categorized, previously corrected error types; it does not establish correct escalation of ambiguous legal cases with reviewer attribution for the described use case.

D:

Apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations; a uniform low-friction approval does not reflect differing blast radius and operational risk; this alternative would measure critical-action dual-approval compliance and zero bypasses during staged tests; it does not establish correct escalation of ambiguous legal cases with reviewer attribution for the described use case.

Question 72

A developer agent has dozens of API specification manuals and needs just one endpoint's error contract. Which choice most directly satisfies the documented goal?

  1. Fetch detailed schemas only for shortlisted actions
  2. Preserve stable prefixes while dynamically discovering tools
  3. Keep orchestrator intent and policy separate from tool inventories
  4. Retrieve detailed reference material only when requested

Explanation

Correct Answer

D:

Discover resource summaries first and fetch the specific authoritative documentation section at the point of need; targeted documentation fetches should improve focus without losing necessary details; evaluate the design with correct endpoint contract use with lower documentation-token overhead.

Incorrect Answers

A:

Keep concise discovery metadata initially and retrieve the precise argument schema when a tool becomes actionable; schema-on-demand still needs strict parameter validation at execution time; this alternative would measure schema-token consumption and invalid-argument rate after staged loading; it does not establish correct endpoint contract use with lower documentation-token overhead for the described use case.

B:

Isolate stable instructions from changing discovery payloads and measure cache behavior when tool inventories vary; progressive discovery can lose its token advantage if dynamic tool definitions repeatedly destroy cache reuse; this alternative would measure cache-read tokens and total context cost while changing selected tool subsets; it does not establish correct endpoint contract use with lower documentation-token overhead for the described use case.

C:

Separate goal interpretation and policy decisions from a downstream staged capability-discovery step; tool search does not supply missing user intent or approval; it is only a mechanism for selecting capabilities; this alternative would measure intent-classification accuracy before any tool disclosure or call; it does not establish correct endpoint contract use with lower documentation-token overhead for the described use case.

Question 73

Pure lexical ranking misses conceptual symptoms, while pure vector retrieval ignores the exact serial constraint. What design correction resolves the reported failure?

  1. Use lexical search for exact identifiers and literals
  2. Rerank candidate passages for question relevance
  3. Retrieve parent context after finding a specific chunk
  4. Combine lexical and semantic candidates when queries mix both

Explanation

Correct Answer

D:

Hybrid retrieval joins exact-term and semantic evidence, then applies metadata filtering and ranking; the evaluation must verify both constraints at once rather than reporting only aggregate relevance; verify the repair using joint exact-identifier accuracy and semantic-symptom recall on mixed queries.

Incorrect Answers

A:

Semantic similarity alone can rank conceptually related but operationally different identifiers; keyword or lexical matching is effective for codes, exact phrases, and terms whose characters carry the meaning; that control is assessed by exact-ID hit rate at the top retrieval ranks; the incident instead requires joint exact-identifier accuracy and semantic-symptom recall on mixed queries.

B:

A bigger first-stage candidate set without selection can dilute context and increase cost; apply a query-aware reranker after broad retrieval, respecting freshness and permissions, to prioritize evidence for synthesis; that control is assessed by relevant-procedure rank and end-to-end answer support rate after reranking; the incident instead requires joint exact-identifier accuracy and semantic-symptom recall on mixed queries.

C:

Passing isolated small chunks may preserve relevance while losing the context needed for a correct conclusion; use child-chunk retrieval for recall and fetch validated parent context or adjacent governing sections for interpretation; that control is assessed by conditional-rule accuracy with and without parent-document expansion; the incident instead requires joint exact-identifier accuracy and semantic-symptom recall on mixed queries.

Question 74

A deployment architecture must be revisited when data location rules or response-time requirements change. Which simulation would prove that the recorded assumptions and decision triggers remain useful to a successor architect?

  1. Consumer contract pass rate under timeouts, invalid arguments, and permission denials
  2. Successful completion of standard support drills by staff who did not build the system
  3. Ability to identify affected decisions after simulated policy and workload changes
  4. Completeness of sensitive-data flow paths identified during security review

Explanation

Correct Answer

C:

Assumption-aware ADRs support controlled adaptation rather than accidental drift; measure ability to identify affected decisions after simulated policy and workload changes at the release gate; record architectural context, alternatives, decision rationale, constraints, and explicit conditions for revisiting choices.

Incorrect Answers

A:

Contract tests validate the documented behavior under non-happy paths; define typed inputs, outputs, error semantics, idempotency, required scopes, and audit obligations for tool boundaries; this tests consumer contract pass rate under timeouts, invalid arguments, and permission denials; the actual release condition is ability to identify affected decisions after simulated policy and workload changes.

B:

Independent operator readiness is stronger evidence than architect-led demonstrations; provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; this tests successful completion of standard support drills by staff who did not build the system; the actual release condition is ability to identify affected decisions after simulated policy and workload changes.

D:

Architecture diagrams are useful when they expose where protection and authority change; a current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects; this tests completeness of sensitive-data flow paths identified during security review; the actual release condition is ability to identify affected decisions after simulated policy and workload changes.

Question 75

A policy assistant starts answering incorrectly immediately after a document-index refresh. Which choice most directly satisfies the documented goal?

  1. Test for model capability mismatch on complex reasoning
  2. Revalidate few-shot examples after policy changes
  3. Trace malformed outputs to prompt or schema boundaries
  4. Investigate retrieval recall before changing the language model

Explanation

Correct Answer

D:

Inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; a retrieval-specific test isolates the component that changed; evaluate the design with recall of newly published policy passages and removal of stale hits.

Incorrect Answers

A:

Hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; rebuilding retrieval is unlikely to help when required evidence was already consistently supplied; this alternative would measure correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; it does not establish recall of newly published policy passages and removal of stale hits for the described use case.

B:

Audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; index re-embedding cannot resolve a stale rule embedded directly in the prompt template; this alternative would measure correct labels on scenarios affected by the policy revision; it does not establish recall of newly published policy passages and removal of stale hits for the described use case.

C:

Inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; switching to a more expensive model may leave the consumer's missing validation problem intact; this alternative would measure schema error rate across multiline, null, and unusual-field edge cases; it does not establish recall of newly published policy passages and removal of stale hits for the described use case.

Question 76

A generated command tries to read production credential files from the execution host. Which corrective change best addresses the root cause?

  1. Allowlist generated fields and destinations before external calls
  2. Screen unsafe outputs in proportion to the use case
  3. Sandbox untrusted code and file operations
  4. Treat retrieved instructions as untrusted data

Explanation

Correct Answer

C:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; sandbox escapes and denial logs are more relevant than the assistant's stated intentions; verify the repair using forbidden file and network access attempts blocked in execution tests.

Incorrect Answers

A:

Formatting the request as JSON does not make arbitrary field values safe to transmit; validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; that control is assessed by zero unapproved recipients or fields accepted by the outbound service; the incident instead requires forbidden file and network access attempts blocked in execution tests.

B:

An input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; that control is assessed by harmful-completion detection on labeled outputs with false-positive monitoring; the incident instead requires forbidden file and network access attempts blocked in execution tests.

D:

Assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; that control is assessed by attack success rate on adversarial web pages and zero unauthorized egress calls; the incident instead requires forbidden file and network access attempts blocked in execution tests.

Question 77

A model API outage blocks even the deterministic approval steps that staff previously handled manually. Which intervention targets the defect rather than its symptoms?

  1. Measure productivity after correction and escalation
  2. Choose model and architecture tiers by case impact
  3. Use a control group to establish incremental business impact
  4. Design a degraded mode that protects the value proposition

Explanation

Correct Answer

D:

Separate critical deterministic operations from AI enhancement and provide explicit fallback and manual paths when model service is unavailable; fallback drills demonstrate that the business process is not hostage to one generative dependency; verify the repair using successful completion of critical workflows during injected model outages.

Incorrect Answers

A:

Counting drafts as completed outcomes overlooks downstream human effort and defect consequences; measure net time saved after review, appeals, and escalations, segmented by complexity, before claiming productivity; that control is assessed by end-to-end completed cases per paid work-hour with rework and appeal volumes; the incident instead requires successful completion of critical workflows during injected model outages.

B:

A uniformly premium model can raise cost without proportionate value on simple, low-risk tasks; route by validated task complexity and consequence, using stronger reasoning or human review where evidence justifies the incremental spend; that control is assessed by quality and total cost segmented by request risk tier under a fixed accuracy floor; the incident instead requires successful completion of critical workflows during injected model outages.

C:

Before-after counts without a credible comparison group may misattribute unrelated changes to the model; use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders; that control is assessed by incremental resolution lift compared with a comparable non-assisted cohort; the incident instead requires successful completion of critical workflows during injected model outages.

Question 78

Users apply generated content to regulated decisions outside the approved use case. What is the most direct remediation?

  1. Design an override path proportionate to consequences
  2. Measure errors across relevant user subgroups
  3. Expose evidence and decision boundaries to affected users
  4. Publish scope, limitations, and residual risks to stakeholders
  5. Provide a route to challenge significant recommendations

Explanation

Correct Answer

D:

Maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; scenario tests reveal whether disclosures prevent inappropriate reliance; verify the repair using accuracy of user understanding of supported versus unsupported tasks.

Incorrect Answers

A:

Human review after an irreversible action may be too late to prevent harm; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; that control is assessed by successful intervention before seeded high-impact incorrect actions are committed; the incident instead requires accuracy of user understanding of supported versus unsupported tasks.

B:

Overall accuracy cannot establish comparable treatment for every affected population; evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; that control is assessed by quality and escalation parity across predeclared language segments; the incident instead requires accuracy of user understanding of supported versus unsupported tasks.

C:

A polished explanation without provenance may create false confidence in unsupported reasoning; distinguish evidence, uncertainty, and advisory status with accessible citations and a path to authoritative decisions; that control is assessed by user comprehension of advisory status and source-citation validity; the incident instead requires accuracy of user understanding of supported versus unsupported tasks.

E:

A fairness dashboard alone cannot remedy individual incorrect records or decisions; provide a contest and correction process with accountable human ownership for significant impacts; that control is assessed by time to resolve substantiated challenges and proportion of cases with documented review; the incident instead requires accuracy of user understanding of supported versus unsupported tasks.

Question 79

Specialist reviewers annotate recurring mistakes in an insurance-assistance workflow. Before approving the next iteration, which trend would show that documented interventions are informing system improvement?

  1. Recurrence rate of categorized, previously corrected error types
  2. Zero disbursements without an auditable transaction-specific approver
  3. Defect-detection rate and reviewer workload under a risk-tiered sampling policy
  4. Critical-action dual-approval compliance and zero bypasses during staged tests
  5. Unauthorized field exposure and successful review accuracy after data minimization

Explanation

Correct Answer

A:

Structured override trends reveal whether feedback improves the system over time; measure recurrence rate of categorized, previously corrected error types at the release gate; record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign.

Incorrect Answers

B:

Payment simulation tests prove approvals are bound to the exact side effect; separate recommendation from execution and require an authenticated human approval bound to the exact amount, payee, and transaction; this tests zero disbursements without an auditable transaction-specific approver; the actual release condition is recurrence rate of categorized, previously corrected error types.

C:

Sampling effectiveness must balance residual errors and human capacity; combine automated checks with representative human sampling and exception-based review scaled to risk; this tests defect-detection rate and reviewer workload under a risk-tiered sampling policy; the actual release condition is recurrence rate of categorized, previously corrected error types.

D:

Approval testing must target the cases where one mistake affects many users; apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations; this tests critical-action dual-approval compliance and zero bypasses during staged tests; the actual release condition is recurrence rate of categorized, previously corrected error types.

E:

Review workflows should preserve decision quality while protecting privacy; minimize, redact, and scope information in review interfaces according to purpose and authorized reviewer role; this tests unauthorized field exposure and successful review accuracy after data minimization; the actual release condition is recurrence rate of categorized, previously corrected error types.

Question 80

Average latency looks healthy even though a long tail violates the customer-facing promise. What is the most direct remediation?

  1. Use a control group to establish incremental business impact
  2. Track cost per accepted business outcome
  3. Translate response-time expectations into a percentile SLO
  4. Choose model and architecture tiers by case impact

Explanation

Correct Answer

C:

Define a p95 or p99 target against real end-to-end requests, including retrieval and tools, rather than reporting an average model time; tail measurement detects the bad experiences that averages can obscure; verify the repair using end-to-end p95 latency and error rate against the agreed service objective.

Incorrect Answers

A:

Before-after counts without a credible comparison group may misattribute unrelated changes to the model; use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders; that control is assessed by incremental resolution lift compared with a comparable non-assisted cohort; the incident instead requires end-to-end p95 latency and error rate against the agreed service objective.

B:

Raw token prices do not capture productivity or failure-driven rework; combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; that control is assessed by total operational cost divided by approved documents meeting the quality rubric; the incident instead requires end-to-end p95 latency and error rate against the agreed service objective.

D:

A uniformly premium model can raise cost without proportionate value on simple, low-risk tasks; route by validated task complexity and consequence, using stronger reasoning or human review where evidence justifies the incremental spend; that control is assessed by quality and total cost segmented by request risk tier under a fixed accuracy floor; the incident instead requires end-to-end p95 latency and error rate against the agreed service objective.

Claude Certified Architect - Professional (CCAR-P) Practice Tests