Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 32

Question 310

An answer regression cannot be reproduced because its tool schema and prompt revision are unknown. Which corrective change best addresses the root cause?

  1. Prepare operator onboarding and support handoff materials
  2. Keep an end-to-end data-flow and trust-boundary diagram
  3. Version prompts, model routes, tools, and evaluation artifacts together
  4. Publish an incident runbook with containment and rollback steps

Explanation

Correct Answer

C:

Maintain a release manifest linking model configuration, prompt, tool contracts, data versions, and evaluation results; versioned provenance enables defensible debugging and rollback; verify the repair using reproducibility of a sampled historical answer using its full manifest.

Incorrect Answers

A:

Source code alone rarely communicates the operational decisions and permission steps operators need; provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; that control is assessed by successful completion of standard support drills by staff who did not build the system; the incident instead requires reproducibility of a sampled historical answer using its full manifest.

B:

A feature list does not reveal crossing of sensitive trust boundaries; a current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects; that control is assessed by completeness of sensitive-data flow paths identified during security review; the incident instead requires reproducibility of a sampled historical answer using its full manifest.

D:

A high-level diagram alone is not an executable incident response procedure; maintain runbooks for detection, containment, safe fallback, rollback, and communication with named incident roles; that control is assessed by successful containment time in a simulated stale-index incident; the incident instead requires reproducibility of a sampled historical answer using its full manifest.

Question 311

A workflow assistant constructs customer notification requests that contain destination addresses and data fields. What action best fits the business and technical boundary?

  1. Screen unsafe outputs in proportion to the use case
  2. Sandbox untrusted code and file operations
  3. Allowlist generated fields and destinations before external calls
  4. Apply controls at input, tool execution, and output boundaries

Explanation

Correct Answer

C:

Validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; contract-level checks demonstrate control over the actual data leaving the system; evaluate the design with zero unapproved recipients or fields accepted by the outbound service.

Incorrect Answers

A:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; an input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; this alternative would measure harmful-completion detection on labeled outputs with false-positive monitoring; it does not establish zero unapproved recipients or fields accepted by the outbound service for the described use case.

B:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; an instruction saying 'do not read secrets' cannot contain arbitrary code on an unrestricted host; this alternative would measure forbidden file and network access attempts blocked in execution tests; it does not establish zero unapproved recipients or fields accepted by the outbound service for the described use case.

D:

Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; a single content filter cannot enforce account privileges or validate downstream business transactions; this alternative would measure denial of unauthorized actions and harmful outputs across multi-stage red-team paths; it does not establish zero unapproved recipients or fields accepted by the outbound service for the described use case.

Question 312

Engineering dashboards display raw token usage but lack cost-per-approved-case and safety trend reports. What is the most direct remediation?

  1. Communicate release behavior changes before rollout
  2. Clarify shared responsibility for the service boundary
  3. Establish a documented escalation path for service incidents
  4. Set observability and reporting expectations with stakeholders

Explanation

Correct Answer

D:

Agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; reporting completeness lets stakeholders govern the system using relevant evidence; verify the repair using delivery of agreed business and risk indicators on schedule with decision-ready interpretation.

Incorrect Answers

A:

Quiet deployment may reduce coordination overhead but damages trust when workflows or permissions change; define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; that control is assessed by affected-user awareness and support-ticket volume following staged changes; the incident instead requires delivery of agreed business and risk indicators on schedule with decision-ready interpretation.

B:

A vendor SLA typically does not assign responsibility for every custom integration and decision rule; define provider, platform, application, security, and business-owner responsibilities in an explicit operating model; that control is assessed by ownership coverage for failures across model, application, data, and tool layers; the incident instead requires delivery of agreed business and risk indicators on schedule with decision-ready interpretation.

C:

Generic 'contact support' guidance lacks accountable timing and action for critical failures; define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; that control is assessed by time to appropriate escalation and fallback activation during drills; the incident instead requires delivery of agreed business and risk indicators on schedule with decision-ready interpretation.

Question 313

A structured invoice extraction endpoint works on ordinary documents but fails when customers include multiline notes. Which choice most directly satisfies the documented goal?

  1. Trace malformed outputs to prompt or schema boundaries
  2. Verify whether claims are supported by supplied sources
  3. Revalidate few-shot examples after policy changes
  4. Investigate retrieval recall before changing the language model

Explanation

Correct Answer

A:

Inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; edge-specific parser tests identify contract failures separately from content comprehension; evaluate the design with schema error rate across multiline, null, and unusual-field edge cases.

Incorrect Answers

B:

Compare material claims against cited evidence and flag unsupported statements for abstention or review; a well-formatted citation is not proof that the cited text supports a particular assertion; this alternative would measure supported-claim rate on source-checked summaries with seeded unsupported numbers; it does not establish schema error rate across multiline, null, and unusual-field edge cases for the described use case.

C:

Audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; index re-embedding cannot resolve a stale rule embedded directly in the prompt template; this alternative would measure correct labels on scenarios affected by the policy revision; it does not establish schema error rate across multiline, null, and unusual-field edge cases for the described use case.

D:

Inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; model upgrades cannot recover information that never reaches the answer context; this alternative would measure recall of newly published policy passages and removal of stale hits; it does not establish schema error rate across multiline, null, and unusual-field edge cases for the described use case.

Question 314

An internal approvals assistant is useful only if the organization can continue routine work during provider or retrieval outages. What action best fits the business and technical boundary?

  1. Track cost per accepted business outcome
  2. Translate response-time expectations into a percentile SLO
  3. Use a control group to establish incremental business impact
  4. Design a degraded mode that protects the value proposition

Explanation

Correct Answer

D:

Separate critical deterministic operations from AI enhancement and provide explicit fallback and manual paths when model service is unavailable; fallback drills demonstrate that the business process is not hostage to one generative dependency; evaluate the design with successful completion of critical workflows during injected model outages.

Incorrect Answers

A:

Combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; raw token prices do not capture productivity or failure-driven rework; this alternative would measure total operational cost divided by approved documents meeting the quality rubric; it does not establish successful completion of critical workflows during injected model outages for the described use case.

B:

Define a p95 or p99 target against real end-to-end requests, including retrieval and tools, rather than reporting an average model time; a percentile SLO does not alone reveal whether the answers are factually correct; this alternative would measure end-to-end p95 latency and error rate against the agreed service objective; it does not establish successful completion of critical workflows during injected model outages for the described use case.

C:

Use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders; before-after counts without a credible comparison group may misattribute unrelated changes to the model; this alternative would measure incremental resolution lift compared with a comparable non-assisted cohort; it does not establish successful completion of critical workflows during injected model outages for the described use case.

Question 315

A document classification pipeline reduces token spend by aggressively shortening prompts. What should the solution architect recommend?

  1. Assess whether abstention and escalation are calibrated
  2. Track cost per verified successful task
  3. Use task-correctness metrics tied to verified labels
  4. Test whether the tool layer denies unauthorized requests

Explanation

Correct Answer

B:

Relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; this metric includes rework rather than rewarding low-quality cheap drafts; evaluate the design with fully loaded cost per correctly completed document.

Incorrect Answers

A:

Measure selective risk: error rate on answered items alongside abstention precision, coverage, and escalation cost; high refusal frequency alone can appear safe while making the assistant operationally useless; this alternative would measure correct-answer rate at each supported coverage threshold including no-answer cases; it does not establish fully loaded cost per correctly completed document for the described use case.

C:

Measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; positive conversational tone does not establish that a legally or financially meaningful label is correct; this alternative would measure critical-class precision and recall on independently verified tax-code cases; it does not establish fully loaded cost per correctly completed document for the described use case.

D:

Treat backend authorization denial rate and cross-scope access tests as primary security metrics, not merely refusal wording; model refusal scores can be high even when the underlying API is vulnerable; this alternative would measure unauthorized API requests rejected under role, tenant, and resource attack matrices; it does not establish fully loaded cost per correctly completed document for the described use case.

Question 316

A research assistant can execute analysis scripts supplied by collaborators from outside the organization. What is the most targeted architecture choice?

  1. Sandbox untrusted code and file operations
  2. Treat retrieved instructions as untrusted data
  3. Allowlist generated fields and destinations before external calls
  4. Screen unsafe outputs in proportion to the use case
  5. Apply controls at input, tool execution, and output boundaries

Explanation

Correct Answer

A:

Isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; sandbox escapes and denial logs are more relevant than the assistant's stated intentions; evaluate the design with forbidden file and network access attempts blocked in execution tests.

Incorrect Answers

B:

Keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; this alternative would measure attack success rate on adversarial web pages and zero unauthorized egress calls; it does not establish forbidden file and network access attempts blocked in execution tests for the described use case.

C:

Validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; formatting the request as JSON does not make arbitrary field values safe to transmit; this alternative would measure zero unapproved recipients or fields accepted by the outbound service; it does not establish forbidden file and network access attempts blocked in execution tests for the described use case.

D:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; an input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; this alternative would measure harmful-completion detection on labeled outputs with false-positive monitoring; it does not establish forbidden file and network access attempts blocked in execution tests for the described use case.

E:

Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; a single content filter cannot enforce account privileges or validate downstream business transactions; this alternative would measure denial of unauthorized actions and harmful outputs across multi-stage red-team paths; it does not establish forbidden file and network access attempts blocked in execution tests for the described use case.

Question 317

A legacy agent interface has several downstream consumers. The program plans a managed deprecation with compatibility and notification stages. Which test result would prove it can retire the endpoint safely?

  1. Cycle time from detected drift to validated improvement without safety regression
  2. Resolution of high-severity architecture risks before implementation approval
  3. Successful retirement with zero unplanned dependent-system failures
  4. Named owner coverage and support-drill completion at handoff
  5. Pilot gate pass rate with defined performance and safety thresholds

Explanation

Correct Answer

C:

Dependency-aware migration tests verify that lifecycle closure is complete; measure successful retirement with zero unplanned dependent-system failures at the release gate; inventory consumers, define migration dates and fallback, preserve data obligations, and test the rollback or retirement path.

Incorrect Answers

A:

Continuous assessment links real usage to controlled change; feed production samples and feedback into versioned evals, prioritize changes, and release improvements behind regression gates; this tests cycle time from detected drift to validated improvement without safety regression; the actual release condition is successful retirement with zero unplanned dependent-system failures.

B:

Early controls reduce late redesign and exposure; incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens; this tests resolution of high-severity architecture risks before implementation approval; the actual release condition is successful retirement with zero unplanned dependent-system failures.

D:

Ownership evidence shows the service can be maintained after consultants leave; establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; this tests named owner coverage and support-drill completion at handoff; the actual release condition is successful retirement with zero unplanned dependent-system failures.

E:

A controlled pilot turns rollout into a measurable decision; define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; this tests pilot gate pass rate with defined performance and safety thresholds; the actual release condition is successful retirement with zero unplanned dependent-system failures.

Question 318

A code assistant must run a mandatory policy or validation check before selected tool operations or commits. Which intervention is most appropriate for this use case?

  1. Use specialized subagents with bounded tools and instructions
  2. Keep credentials out of prompts and versioned configuration
  3. Use deterministic hooks to enforce required developer gates
  4. Use managed configuration to constrain dangerous permissions
  5. Scope MCP connections by project and user identity

Explanation

Correct Answer

C:

Configure supported hooks for deterministic checks at appropriate lifecycle points, with clear failure behavior and maintenance ownership; gate testing verifies that a real program, not a conversational promise, enforces the rule; evaluate the design with hook execution and blocked noncompliant operations under simulated bypass attempts.

Incorrect Answers

A:

Give each delegated role explicit objectives, context limits, and tool permissions with clear return artifacts; delegation without role scoping merely multiplies the number of powerful agents; this alternative would measure subagent task completion with zero out-of-scope tool access; it does not establish hook execution and blocked noncompliant operations under simulated bypass attempts for the described use case.

B:

Use approved secret storage, scoped environment access, and redaction while avoiding credentials in repositories, prompts, or logs; masking output after committing a secret cannot undo credential exposure or grant least privilege; this alternative would measure secret-scanning results and blocked exfiltration on seeded repository credentials; it does not establish hook execution and blocked noncompliant operations under simulated bypass attempts for the described use case.

D:

Use organization-managed policy and supported permission configuration so enforced restrictions cannot be casually overridden by per-user convenience settings; personal preference files are not a trustworthy substitute for centrally governed high-risk permissions; this alternative would measure denial of prohibited commands after attempts to override local and project configuration; it does not establish hook execution and blocked noncompliant operations under simulated bypass attempts for the described use case.

E:

Choose appropriate configuration scope and per-user or service authorization for MCP servers, avoiding shared cross-project secrets; a working connection does not prove the assistant is restricted to the user's intended project resources; this alternative would measure access denial for cross-project MCP resources under user identity tests; it does not establish hook execution and blocked noncompliant operations under simulated bypass attempts for the described use case.

Question 319

A reporting-only service identity is reviewed after the account team splits its read and update APIs. Which contract-test result would prove that an analyst cannot modify financial records through Claude?

  1. Changes rejected when requested tool scopes exceed the approved task manifest
  2. Absence of privileged tool calls before the confirmed transaction stage
  3. Single correct connector selection and consistent ticket states across repeated requests
  4. Zero mutation permissions for report-only identities across tool contract checks

Explanation

Correct Answer

D:

Permission tests reveal whether supposedly analytical agents can cause side effects; measure zero mutation permissions for report-only identities across tool contract checks at the release gate; expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals.

Incorrect Answers

A:

Policy-based manifest validation detects excessive authority before runtime exposure; maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; this tests changes rejected when requested tool scopes exceed the approved task manifest; the actual release condition is zero mutation permissions for report-only identities across tool contract checks.

B:

Stage-bound tests prove that tool exposure follows authorization and intent; reveal high-impact tools only within a confirmed, authorized transaction stage; maintain server-side gating independently of model choice; this tests absence of privileged tool calls before the confirmed transaction stage; the actual release condition is zero mutation permissions for report-only identities across tool contract checks.

C:

Removing ambiguity should reduce routing variance without reducing supported tasks; consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; this tests single correct connector selection and consistent ticket states across repeated requests; the actual release condition is zero mutation permissions for report-only identities across tool contract checks.

Question 320

Support staff have no clear threshold or owner for switching to manual handling. Which corrective change best addresses the root cause?

  1. Set observability and reporting expectations with stakeholders
  2. Establish a documented escalation path for service incidents
  3. Route user feedback into a triaged, owned backlog
  4. Define SLOs and error budgets for agreed user outcomes

Explanation

Correct Answer

B:

Define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; incident response tests verify that operational ownership works under pressure; verify the repair using time to appropriate escalation and fallback activation during drills.

Incorrect Answers

A:

Platform telemetry is necessary but not equivalent to business service reporting; agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; that control is assessed by delivery of agreed business and risk indicators on schedule with decision-ready interpretation; the incident instead requires time to appropriate escalation and fallback activation during drills.

C:

Unstructured rating totals cannot by themselves direct a safe corrective action; classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; that control is assessed by time from actionable feedback to triage, owner assignment, and verified fix; the incident instead requires time to appropriate escalation and fallback activation during drills.

D:

Generic uptime claims can conceal slow or unusable answer experiences; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; that control is assessed by sLO compliance and prescribed actions during a simulated error-budget breach; the incident instead requires time to appropriate escalation and fallback activation during drills.

Claude Certified Architect - Professional (CCAR-P) Practice Tests