Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 5

Question 40

A staged extraction workflow is upgraded one component at a time while downstream integrations stay on their prior versions. Which measure shows that independent deployments preserve the expected contracts?

  1. Correct merged clause counts across repeated parallel execution races
  2. Contract test pass rate for independently upgraded stages and invalid payload rejection
  3. Stage-specific defect localization and repair without redoing previously validated stages
  4. Successful recovery after injected worker failures without duplicated writes or stages

Explanation

Correct Answer

B:

Versioned interfaces expose breaking changes before a multi-team deployment; measure contract test pass rate for independently upgraded stages and invalid payload rejection at the release gate; typed versioned contracts and compatibility tests let stages evolve while preserving their callers' assumptions.

Incorrect Answers

A:

Deterministic merge checks measure integrity rather than only wall-clock speed; isolate independent tasks, retain separate results, then join only when required dependencies have completed; this tests correct merged clause counts across repeated parallel execution races; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

C:

A traceable artifact boundary confines error analysis to the failed operation; separate stages with typed artifacts allow validation and targeted reruns at meaningful boundaries; this tests stage-specific defect localization and repair without redoing previously validated stages; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

D:

Replay testing verifies the precise recovery behavior the pipeline requires; durable checkpoints record validated intermediate artifacts and resume at safe boundaries with idempotent side effects; this tests successful recovery after injected worker failures without duplicated writes or stages; the actual release condition is contract test pass rate for independently upgraded stages and invalid payload rejection.

Question 41

A fraud tool receives risk scores and approval-like labels from external data providers. Auditors deliberately label untrusted fields as authoritative. Which test outcome would demonstrate that external evidence cannot grant internal permission?

  1. Proportion of numeric claims supported by independently inspected references
  2. Unauthorized tool-action success rate over injected emails and documents
  3. Zero external-result fields promoted to internal approvals in test cases
  4. Blocked unauthorized destinations and absence of confidential fields in egress logs

Explanation

Correct Answer

C:

Provenance-sensitive checks stop evidence laundering through connector outputs; measure zero external-result fields promoted to internal approvals in test cases at the release gate; preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals.

Incorrect Answers

A:

Grounding tests detect confident fabrication at the statement level; require support from retrieved source passages and provide an explicit unknown path for unsupported claims; this tests proportion of numeric claims supported by independently inspected references; the actual release condition is zero external-result fields promoted to internal approvals in test cases.

B:

A realistic attack corpus exercises both model behavior and execution controls; model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; this tests unauthorized tool-action success rate over injected emails and documents; the actual release condition is zero external-result fields promoted to internal approvals in test cases.

D:

Network- and tool-level evidence catches leakage not visible in user-facing text; restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; this tests blocked unauthorized destinations and absence of confidential fields in egress logs; the actual release condition is zero external-result fields promoted to internal approvals in test cases.

Question 42

During a dry run, an exploratory planning step calls a production write tool before approval. What is the most direct remediation?

  1. Separate planning proposals from privileged execution
  2. Partition research by source and reconcile conflicts explicitly
  3. Parallelize only truly independent branches
  4. Divide work along independently verifiable artifacts
  5. Checkpoint long-running work for replay and recovery

Explanation

Correct Answer

A:

Use a planning artifact that is reviewed and handed to a distinct authorized executor; never treat proposed tool calls as permission to mutate state; this tests enforcement at the action boundary rather than relying on prose instructions; verify the repair using unauthorized mutations during plan-only tests, with a target of zero.

Incorrect Answers

B:

Simple partitioning can preserve conflicting claims unless the merge step includes a reconciliation rule; analyze source partitions separately, retain provenance, and reconcile contradictions through a dedicated synthesis stage; that control is assessed by conflict detection recall and citation coverage across independently processed source sets; the incident instead requires unauthorized mutations during plan-only tests, with a target of zero.

C:

Parallelizing dependent mutations without isolation can increase errors even if raw processing is faster; isolate independent tasks, retain separate results, then join only when required dependencies have completed; that control is assessed by correct merged clause counts across repeated parallel execution races; the incident instead requires unauthorized mutations during plan-only tests, with a target of zero.

D:

Artifact decomposition controls work structure but does not set the organization-wide release SLA; separate stages with typed artifacts allow validation and targeted reruns at meaningful boundaries; that control is assessed by stage-specific defect localization and repair without redoing previously validated stages; the incident instead requires unauthorized mutations during plan-only tests, with a target of zero.

E:

Checkpointing reduces replay cost but cannot correct a defective classification rubric; durable checkpoints record validated intermediate artifacts and resume at safe boundaries with idempotent side effects; that control is assessed by successful recovery after injected worker failures without duplicated writes or stages; the incident instead requires unauthorized mutations during plan-only tests, with a target of zero.

Question 43

A help-desk bot sends ordinary account lookups to the most expensive reasoning tier. Their accuracy does not improve, while p95 latency and spend rise. What should the model-routing policy change?

  1. Parallelize independent read-only tool calls
  2. Route expensive reasoning to high-uncertainty cases
  3. Cache safe, version-keyed retrieval responses
  4. Use a safe limited answer when the deadline would otherwise be missed
  5. Move slow operations into asynchronous jobs with progress states

Explanation

Correct Answer

B:

Use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; joint segmented reporting reveals whether expensive processing is spent where it helps; verify the repair using accuracy by difficulty segment and unit cost under p95 constraints.

Incorrect Answers

A:

Parallel execution offers no benefit when one call requires another's output or modifies shared state; execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; that control is assessed by end-to-end p95 improvement with all required sources still present; the incident instead requires accuracy by difficulty segment and unit cost under p95 constraints.

C:

An unversioned cache can improve speed while silently sacrificing freshness and access correctness; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; that control is assessed by repeated-query latency with zero stale or cross-scope cache responses after updates; the incident instead requires accuracy by difficulty segment and unit cost under p95 constraints.

D:

Silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; that control is assessed by deadline compliance and correct indication of omitted enrichment; the incident instead requires accuracy by difficulty segment and unit cost under p95 constraints.

E:

A longer synchronous timeout would still tie up request resources and expose users to unreliable waits; acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; that control is assessed by job completion reliability and duplicate-submission rate during slow-system simulations; the incident instead requires accuracy by difficulty segment and unit cost under p95 constraints.

Question 44

A finance backend relies on generated code for monetary calculations with rounding and negative adjustments. Which regression result would justify merging the change instead of trusting the happy-path demonstration?

  1. Detection of seeded authorization regressions during peer review and CI
  2. Passing unit and property tests including specified rounding boundaries
  3. Checklist completeness across independent sessions using the reviewed Skill version
  4. Review time and defect localization per incremental change set

Explanation

Correct Answer

B:

Test-first evidence anchors generated code in independently defined behavior; measure passing unit and property tests including specified rounding boundaries at the release gate; define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate.

Incorrect Answers

A:

Diff-focused review finds changes that ordinary output fluency cannot reveal; require focused code review of the actual diff, threat-relevant tests, and approval before merging sensitive changes; this tests detection of seeded authorization regressions during peer review and CI; the actual release condition is passing unit and property tests including specified rounding boundaries.

C:

Reuse tests verify that essential steps are consistently available; maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; this tests checklist completeness across independent sessions using the reviewed Skill version; the actual release condition is passing unit and property tests including specified rounding boundaries.

D:

Small-batch evidence shows whether modernization remains controlled and reversible; use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; this tests review time and defect localization per incremental change set; the actual release condition is passing unit and property tests including specified rounding boundaries.

Question 45

A service-level review spans model hosting, business data, application middleware, and external tools. Which evidence would show that each class of failure has an accountable operator rather than an assumed vendor owner?

  1. Affected-user awareness and support-ticket volume following staged changes
  2. Delivery of agreed business and risk indicators on schedule with decision-ready interpretation
  3. SLO compliance and prescribed actions during a simulated error-budget breach
  4. Ownership coverage for failures across model, application, data, and tool layers

Explanation

Correct Answer

D:

Responsibility tests prevent gaps between contractual expectations and implemented controls; measure ownership coverage for failures across model, application, data, and tool layers at the release gate; define provider, platform, application, security, and business-owner responsibilities in an explicit operating model.

Incorrect Answers

A:

Communication quality is reflected in predictable adoption and fewer avoidable surprises; define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; this tests affected-user awareness and support-ticket volume following staged changes; the actual release condition is ownership coverage for failures across model, application, data, and tool layers.

B:

Reporting completeness lets stakeholders govern the system using relevant evidence; agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; this tests delivery of agreed business and risk indicators on schedule with decision-ready interpretation; the actual release condition is ownership coverage for failures across model, application, data, and tool layers.

C:

Operational SLO tests connect observed service quality to response commitments; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; this tests sLO compliance and prescribed actions during a simulated error-budget breach; the actual release condition is ownership coverage for failures across model, application, data, and tool layers.

Question 46

An enterprise support assistant handles frequent requests using a common manual. The team has reorganized stable prompt content to improve caching. Which combined result would justify the optimization without harming answer quality?

  1. All-in cost per accepted summary at a fixed mandatory-section completion rate
  2. p95 response time with all three source checks present under load
  3. Cost and successful throughput per nightly processing window
  4. Accepted resolution rate and cost per case across routed difficulty levels
  5. Cache-read rate, p95 latency, and cost per accepted answer under repeated queries

Explanation

Correct Answer

E:

Joint economics and quality checks demonstrate that reuse helps without sacrificing correctness; measure cache-read rate, p95 latency, and cost per accepted answer under repeated queries at the release gate; keep a stable eligible prompt prefix and measure cache-read tokens to reduce repeated processing.

Incorrect Answers

A:

Quality-constrained unit economics penalizes truncation and repeat work; optimize total cost per successful output with floor constraints on completeness, safety, and latency; this tests all-in cost per accepted summary at a fixed mandatory-section completion rate; the actual release condition is cache-read rate, p95 latency, and cost per accepted answer under repeated queries.

B:

The acceptance signal guards against a fast but incomplete response; parallel retrieval with per-call budgets reduces wall time while preserving authorization and complete aggregation; this tests p95 response time with all three source checks present under load; the actual release condition is cache-read rate, p95 latency, and cost per accepted answer under repeated queries.

C:

Deadline-oriented throughput is the relevant performance objective for noninteractive work; use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense; this tests cost and successful throughput per nightly processing window; the actual release condition is cache-read rate, p95 latency, and cost per accepted answer under repeated queries.

D:

Stratified monitoring confirms economic gains do not displace expensive errors onto users; evaluate a smaller model on straightforward request strata and escalate only where measurable capability is needed; this tests accepted resolution rate and cost per case across routed difficulty levels; the actual release condition is cache-read rate, p95 latency, and cost per accepted answer under repeated queries.

Question 47

Product dashboards show accurate normal answers while harmful edge responses go unreported. Which corrective change best addresses the root cause?

  1. Track cost per verified successful task
  2. Use task-correctness metrics tied to verified labels
  3. Measure rate and severity of safety policy violations
  4. Test whether the tool layer denies unauthorized requests

Explanation

Correct Answer

C:

Define safety incident categories, risk-weighted violation rates, and appropriate exposure denominators for adversarial and real traffic; risk-weighted scoring detects rare critical violations that a single average hides; verify the repair using severity-weighted policy violation incidence on a reviewed adversarial test set.

Incorrect Answers

A:

Cost per token can improve while cost per correct business outcome deteriorates; relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; that control is assessed by fully loaded cost per correctly completed document; the incident instead requires severity-weighted policy violation incidence on a reviewed adversarial test set.

B:

Positive conversational tone does not establish that a legally or financially meaningful label is correct; measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; that control is assessed by critical-class precision and recall on independently verified tax-code cases; the incident instead requires severity-weighted policy violation incidence on a reviewed adversarial test set.

D:

Model refusal scores can be high even when the underlying API is vulnerable; treat backend authorization denial rate and cross-scope access tests as primary security metrics, not merely refusal wording; that control is assessed by unauthorized API requests rejected under role, tenant, and resource attack matrices; the incident instead requires severity-weighted policy violation incidence on a reviewed adversarial test set.

Question 48

A multi-customer platform has acceptable global error rates but one enterprise tenant sees repeated timeouts. Which intervention is most appropriate for this use case?

  1. Segment reliability metrics by tenant and workload
  2. Correlate model, retrieval, and tool spans in distributed traces
  3. Sample complete traces strategically under heavy traffic
  4. Log structured tool outcomes without exposing sensitive payloads
  5. Separate retrieval, provider, and tool latency budgets

Explanation

Correct Answer

A:

Break down latency, errors, and tool success by permission-safe tenant and workload dimensions with cardinality controls; segmented monitoring finds localized issues that global summary statistics miss; evaluate the design with detection time for an injected single-tenant degradation.

Incorrect Answers

B:

End-to-end traces with consistent request and span identifiers identify which service and step consumed time; a model-only latency chart cannot explain retrieval or downstream network bottlenecks; this alternative would measure trace completeness and correct bottleneck attribution on instrumented fault injections; it does not establish detection time for an injected single-tenant degradation for the described use case.

C:

Retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; keeping every raw trace is costly and may violate retention principles without improving detection proportionally; this alternative would measure capture rate of seeded critical failures at a bounded telemetry cost; it does not establish detection time for an injected single-tenant degradation for the described use case.

D:

Record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; full payload capture may help debugging but expands sensitive-data exposure and retention obligations; this alternative would measure error diagnosability with seeded sensitive fields absent from collected logs; it does not establish detection time for an injected single-tenant degradation for the described use case.

E:

Instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; optimizing generation tokens cannot repair a slow connector that dominates the critical path; this alternative would measure stage-specific p95 deltas identifying injected bottlenecks; it does not establish detection time for an injected single-tenant degradation for the described use case.

Question 49

An order-adjustment workflow is tested by replaying the same approved operation after delayed acknowledgments and worker restarts. Which outcome confirms that recovery cannot cause an extra financial adjustment?

  1. Invalid write payloads rejected before execution, with zero partial supplier mutations
  2. Regression pass rate on previously corrected clause-omission cases
  3. Job completion rate and queue age during the peak intake window
  4. Repeated and interrupted test runs yielding exactly one committed credit per approved case

Explanation

Correct Answer

D:

Replay tests directly exercise the failure window that typically creates duplicate writes; measure repeated and interrupted test runs yielding exactly one committed credit per approved case at the release gate; persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays.

Incorrect Answers

A:

Pre-write enforcement proves that the model cannot bypass the target system's integrity contract; use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; this tests invalid write payloads rejected before execution, with zero partial supplier mutations; the actual release condition is repeated and interrupted test runs yielding exactly one committed credit per approved case.

B:

Retesting concrete corrections detects whether a release reintroduces known business mistakes; preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; this tests regression pass rate on previously corrected clause-omission cases; the actual release condition is repeated and interrupted test runs yielding exactly one committed credit per approved case.

C:

Measuring backlog and completion under load tests the exact burst tolerance the design promises; decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; this tests job completion rate and queue age during the peak intake window; the actual release condition is repeated and interrupted test runs yielding exactly one committed credit per approved case.

Question 50

A credit operations team is assembling a labeled benchmark from reviewers who initially disagree on ambiguous classifications. Which dataset-quality measure would show that the reference answers have been adjudicated consistently?

  1. Attack success rate and safe handling of intentionally unanswerable prompts
  2. Expert-rated factual fidelity and calibrated grader agreement on sampled explanations
  3. Inter-annotator agreement and unresolved-label rate after adjudication
  4. Zero schema violations accepted by the downstream consumer across generated cases
  5. Per-stratum pass rates with no uncovered critical scenario classes

Explanation

Correct Answer

C:

Label reliability is necessary before model score differences can be interpreted; measure inter-annotator agreement and unresolved-label rate after adjudication at the release gate; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth.

Incorrect Answers

A:

Red-team cases make trust-boundary performance observable; add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; this tests attack success rate and safe handling of intentionally unanswerable prompts; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

B:

A semantic rubric tests meaning while allowing legitimate variation in language; use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; this tests expert-rated factual fidelity and calibrated grader agreement on sampled explanations; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

D:

Executable contract tests deliver reproducible feedback at scale; apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; this tests zero schema violations accepted by the downstream consumer across generated cases; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

E:

Stratified coverage prevents abundant easy cases from hiding unsupported workflows; build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; this tests per-stratum pass rates with no uncovered critical scenario classes; the actual release condition is inter-annotator agreement and unresolved-label rate after adjudication.

Claude Certified Architect - Professional (CCAR-P) Practice Tests