Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 38

Question 370

Reviewers cannot isolate behavior changes and rollbacks become unsafe when one small contract test fails. Which action best removes the underlying failure mode?

  1. Run reproducible CI gates on assisted code
  2. Break large refactors into independently reviewable changes
  3. Package reusable engineering instructions as governed Skills
  4. Use test-first acceptance cases for generated changes

Explanation

Correct Answer

B:

Use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; small-batch evidence shows whether modernization remains controlled and reversible; verify the repair using review time and defect localization per incremental change set.

Incorrect Answers

A:

Local success can be environment-dependent and should not be the only release assurance; execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; that control is assessed by cI pass reproducibility on clean builds with locked dependencies; the incident instead requires review time and defect localization per incremental change set.

C:

Shared instructions improve consistency but do not replace enforced approval or automated tests; maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; that control is assessed by checklist completeness across independent sessions using the reviewed Skill version; the incident instead requires review time and defect localization per incremental change set.

D:

Merely reading generated code for plausibility does not replace tests of business invariants; define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; that control is assessed by passing unit and property tests including specified rounding boundaries; the incident instead requires review time and defect localization per incremental change set.

Question 371

Accuracy declines gradually while the team continues to cite its original acceptance score. Which change should the architect recommend first?

  1. Set an evidence gate before moving beyond discovery
  2. Use bounded pilots with exit criteria before expansion
  3. Run a recurring monitoring-to-improvement loop
  4. Transfer durable ownership and operations before handoff

Explanation

Correct Answer

C:

Feed production samples and feedback into versioned evals, prioritize changes, and release improvements behind regression gates; continuous assessment links real usage to controlled change; verify the repair using cycle time from detected drift to validated improvement without safety regression.

Incorrect Answers

A:

A procurement timeline does not eliminate uncertainty about whether the problem is solvable; require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; that control is assessed by closure of high-risk feasibility questions before design approval; the incident instead requires cycle time from detected drift to validated improvement without safety regression.

B:

More launch enthusiasm does not replace production-like testing under load and edge conditions; define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; that control is assessed by pilot gate pass rate with defined performance and safety thresholds; the incident instead requires cycle time from detected drift to validated improvement without safety regression.

D:

A completed repository is not proof of ongoing service operability; establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; that control is assessed by named owner coverage and support-drill completion at handoff; the incident instead requires cycle time from detected drift to validated improvement without safety regression.

Question 372

A document assistant uses low-cost tokens but needs repeated human rewriting for many outputs. What should the solution architect recommend?

  1. Design a degraded mode that protects the value proposition
  2. Track cost per accepted business outcome
  3. Use a control group to establish incremental business impact
  4. Choose model and architecture tiers by case impact

Explanation

Correct Answer

B:

Combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; unit cost tied to accepted results distinguishes economical quality from cheap failed attempts; evaluate the design with total operational cost divided by approved documents meeting the quality rubric.

Incorrect Answers

A:

Separate critical deterministic operations from AI enhancement and provide explicit fallback and manual paths when model service is unavailable; high answer accuracy in normal mode says little about process continuity during dependency failures; this alternative would measure successful completion of critical workflows during injected model outages; it does not establish total operational cost divided by approved documents meeting the quality rubric for the described use case.

C:

Use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders; before-after counts without a credible comparison group may misattribute unrelated changes to the model; this alternative would measure incremental resolution lift compared with a comparable non-assisted cohort; it does not establish total operational cost divided by approved documents meeting the quality rubric for the described use case.

D:

Route by validated task complexity and consequence, using stronger reasoning or human review where evidence justifies the incremental spend; a uniformly premium model can raise cost without proportionate value on simple, low-risk tasks; this alternative would measure quality and total cost segmented by request risk tier under a fixed accuracy floor; it does not establish total operational cost divided by approved documents meeting the quality rubric for the described use case.

Question 373

A malicious file instructs the model to submit a transfer through the overprivileged connector. How should the design be corrected?

  1. Apply controls at input, tool execution, and output boundaries
  2. Enforce least privilege where tool actions execute
  3. Screen unsafe outputs in proportion to the use case
  4. Allowlist generated fields and destinations before external calls

Explanation

Correct Answer

B:

Enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; direct permission tests verify security independent of model obedience; verify the repair using zero payment writes when tested with the analytics identity and forged instructions.

Incorrect Answers

A:

A single content filter cannot enforce account privileges or validate downstream business transactions; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; that control is assessed by denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the incident instead requires zero payment writes when tested with the analytics identity and forged instructions.

C:

An input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; that control is assessed by harmful-completion detection on labeled outputs with false-positive monitoring; the incident instead requires zero payment writes when tested with the analytics identity and forged instructions.

D:

Formatting the request as JSON does not make arbitrary field values safe to transmit; validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; that control is assessed by zero unapproved recipients or fields accepted by the outbound service; the incident instead requires zero payment writes when tested with the analytics identity and forged instructions.

Question 374

A multilingual benefits assistant has strong average accuracy but complaints cluster among speakers of one supported language. Which option addresses the immediate design need?

  1. Publish scope, limitations, and residual risks to stakeholders
  2. Provide a route to challenge significant recommendations
  3. Measure errors across relevant user subgroups
  4. Design an override path proportionate to consequences

Explanation

Correct Answer

C:

Evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; group-specific evaluation reveals concentrated harm masked by pooled scores; evaluate the design with quality and escalation parity across predeclared language segments.

Incorrect Answers

A:

Maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; a generic disclaimer cannot compensate for hidden high-risk use or unavailable oversight; this alternative would measure accuracy of user understanding of supported versus unsupported tasks; it does not establish quality and escalation parity across predeclared language segments for the described use case.

B:

Provide a contest and correction process with accountable human ownership for significant impacts; a fairness dashboard alone cannot remedy individual incorrect records or decisions; this alternative would measure time to resolve substantiated challenges and proportion of cases with documented review; it does not establish quality and escalation parity across predeclared language segments for the described use case.

D:

Preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; human review after an irreversible action may be too late to prevent harm; this alternative would measure successful intervention before seeded high-impact incorrect actions are committed; it does not establish quality and escalation parity across predeclared language segments for the described use case.

Question 375

A multi-agent investigation service is evaluated with tasks deliberately lacking a conclusive answer. Which observed behavior verifies that retry limits and escalation paths prevent runaway coordination?

  1. Measured marginal quality improvement per added specialist against its incremental cost and latency
  2. Completed reports with correctly scoped evidence and no cross-specialty data leakage
  3. p95 end-to-end latency and deterministic merge completeness under one-branch failures
  4. Zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record

Explanation

Correct Answer

D:

Stressing unresolved cases proves that the system can fail predictably and within budget; measure zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record at the release gate; enforce maximum depth, handoff count, timeouts, and explicit unresolved-case escalation in the orchestrator.

Incorrect Answers

B:

Routing correctness must include scope enforcement, not just readable final reports; a coordinator can dispatch scoped subagents and merge their typed outputs while restricting each specialist's context and tools; this tests completed reports with correctly scoped evidence and no cross-specialty data leakage; the actual release condition is zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record.

C:

This validates concurrency benefits without silently dropping a missing assessment; parallel fan-out followed by deterministic aggregation can shorten wall-clock time if each branch has its own contract and errors are handled; this tests p95 end-to-end latency and deterministic merge completeness under one-branch failures; the actual release condition is zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record.

A:

Measuring the contribution of each delegate reveals needless orchestration layers; introduce a specialist only when its independent expertise, tool scope, or parallelism beats a simpler call on representative evaluations; this tests marginal quality improvement per added specialist against its incremental cost and latency; the actual release condition is zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record.

Question 376

A canary rollout initially fails for one model-and-prompt version, but aggregate dashboards blend its results with the old version. Which observability change would reveal the regression early?

  1. Detect distribution and quality drift in ongoing samples
  2. Instrument end-to-end execution spans for causal debugging
  3. Track safety policy events across product updates
  4. Compare production metrics by model and prompt version

Explanation

Correct Answer

D:

Partition traces and scored outcomes by deployed configuration, using release identifiers to support rollback and diagnosis; release segmentation isolates changes before broad impact; verify the repair using detection of version-specific quality regressions in a staged rollout.

Incorrect Answers

A:

A static validation set cannot reflect every change in real input distribution over time; periodically sample representative production outputs for labeling, segment analysis, and drift signals without retaining unnecessary personal data; that control is assessed by quality delta between recent audited production samples and the launch baseline; the incident instead requires detection of version-specific quality regressions in a staged rollout.

B:

Model usage totals do not reveal which integration failed within a particular request; correlation and span IDs across components allow latency and failure attribution along the actual path; that control is assessed by percent of failures with a complete trace to the responsible stage; the incident instead requires detection of version-specific quality regressions in a staged rollout.

C:

A falling count of harmful completions can coexist with an unusable overrestrictive assistant; monitor both harmful-output violations and false-positive blocks by version and user segment; that control is assessed by safety precision/recall and legitimate-task completion before and after the change; the incident instead requires detection of version-specific quality regressions in a staged rollout.

Question 377

A network automation service proposes both read-only diagnostics and service-disrupting operations. Which staged-test result would demonstrate that consequential changes require appropriately stronger authorization?

  1. Recurrence rate of categorized, previously corrected error types
  2. Critical-action dual-approval compliance and zero bypasses during staged tests
  3. Zero disbursements without an auditable transaction-specific approver
  4. Defect-detection rate and reviewer workload under a risk-tiered sampling policy

Explanation

Correct Answer

B:

Approval testing must target the cases where one mistake affects many users; measure critical-action dual-approval compliance and zero bypasses during staged tests at the release gate; apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations.

Incorrect Answers

A:

Structured override trends reveal whether feedback improves the system over time; record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign; this tests recurrence rate of categorized, previously corrected error types; the actual release condition is critical-action dual-approval compliance and zero bypasses during staged tests.

C:

Payment simulation tests prove approvals are bound to the exact side effect; separate recommendation from execution and require an authenticated human approval bound to the exact amount, payee, and transaction; this tests zero disbursements without an auditable transaction-specific approver; the actual release condition is critical-action dual-approval compliance and zero bypasses during staged tests.

D:

Sampling effectiveness must balance residual errors and human capacity; combine automated checks with representative human sampling and exception-based review scaled to risk; this tests defect-detection rate and reviewer workload under a risk-tiered sampling policy; the actual release condition is critical-action dual-approval compliance and zero bypasses during staged tests.

Question 378

A multi-turn assistant benefits from prompt caching but its connector list changes on every request. What is the most defensible next step?

  1. Fetch detailed schemas only for shortlisted actions
  2. Use searchable tool discovery for large catalogs
  3. Preserve stable prefixes while dynamically discovering tools
  4. Keep orchestrator intent and policy separate from tool inventories

Explanation

Correct Answer

C:

Isolate stable instructions from changing discovery payloads and measure cache behavior when tool inventories vary; the combined test reveals interaction between discovery strategy and prompt-prefix economics; evaluate the design with cache-read tokens and total context cost while changing selected tool subsets.

Incorrect Answers

A:

Keep concise discovery metadata initially and retrieve the precise argument schema when a tool becomes actionable; schema-on-demand still needs strict parameter validation at execution time; this alternative would measure schema-token consumption and invalid-argument rate after staged loading; it does not establish cache-read tokens and total context cost while changing selected tool subsets for the described use case.

B:

Expose a lean search or discovery interface and load relevant detailed tools only after intent is established; up-front registration of every schema does not improve a request that needs only one domain-specific operation; this alternative would measure tool-choice accuracy and context-token savings on varied employee requests; it does not establish cache-read tokens and total context cost while changing selected tool subsets for the described use case.

D:

Separate goal interpretation and policy decisions from a downstream staged capability-discovery step; tool search does not supply missing user intent or approval; it is only a mechanism for selecting capabilities; this alternative would measure intent-classification accuracy before any tool disclosure or call; it does not establish cache-read tokens and total context cost while changing selected tool subsets for the described use case.

Question 379

A cross-functional AI team has issued several revisions of its tool permissions and prompt rules. Which exercise would establish that reviewers can identify the current approved decision and why it replaced earlier choices?

  1. Stakeholder identification of boundaries and fallback responsibilities after the demo
  2. Completion and ownership of high-severity mitigations before the release gate
  3. Reconstructability of key design decisions after a simulated staff handoff
  4. Correct retrieval of active decisions and their supersession chain during review

Explanation

Correct Answer

D:

Version-aware retrieval prevents implementation against outdated agreements; measure correct retrieval of active decisions and their supersession chain during review at the release gate; maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance.

Incorrect Answers

A:

Understanding checks reveal whether decision-makers grasp operational limits; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; this tests stakeholder identification of boundaries and fallback responsibilities after the demo; the actual release condition is correct retrieval of active decisions and their supersession chain during review.

B:

A control register makes risk decisions reviewable rather than rhetorical; record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; this tests completion and ownership of high-severity mitigations before the release gate; the actual release condition is correct retrieval of active decisions and their supersession chain during review.

C:

Decision records preserve rationale beyond a particular architect's memory; use an ADR recording context, considered options, decision, trade-offs, implications, and conditions for revisiting it; this tests reconstructability of key design decisions after a simulated staff handoff; the actual release condition is correct retrieval of active decisions and their supersession chain during review.

Question 380

A research platform executes collaborator-supplied scripts in a constrained runtime. Penetration tests attempt to reach restricted files and network services. Which outcome would establish that the execution sandbox is effective?

  1. Measured attack success rate on adversarial web pages and zero unauthorized egress calls
  2. Forbidden file and network access attempts blocked in execution tests
  3. Denial of unauthorized actions and harmful outputs across multi-stage red-team paths
  4. Zero payment writes when tested with the analytics identity and forged instructions

Explanation

Correct Answer

B:

Sandbox escapes and denial logs are more relevant than the assistant's stated intentions; measure forbidden file and network access attempts blocked in execution tests at the release gate; isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials.

Incorrect Answers

A:

Simulated hostile documents test the trust boundary directly; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; this tests attack success rate on adversarial web pages and zero unauthorized egress calls; the actual release condition is forbidden file and network access attempts blocked in execution tests.

C:

End-to-end adversarial cases assess the combined defenses and their failure isolation; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; this tests denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the actual release condition is forbidden file and network access attempts blocked in execution tests.

D:

Direct permission tests verify security independent of model obedience; enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; this tests zero payment writes when tested with the analytics identity and forged instructions; the actual release condition is forbidden file and network access attempts blocked in execution tests.

Claude Certified Architect - Professional (CCAR-P) Practice Tests