Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 39

Question 380

A research platform executes collaborator-supplied scripts in a constrained runtime. Penetration tests attempt to reach restricted files and network services. Which outcome would establish that the execution sandbox is effective?

  1. Measured attack success rate on adversarial web pages and zero unauthorized egress calls
  2. Forbidden file and network access attempts blocked in execution tests
  3. Denial of unauthorized actions and harmful outputs across multi-stage red-team paths
  4. Zero payment writes when tested with the analytics identity and forged instructions

Explanation

Correct Answer

B:

Sandbox escapes and denial logs are more relevant than the assistant's stated intentions; measure forbidden file and network access attempts blocked in execution tests at the release gate; isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials.

Incorrect Answers

A:

Simulated hostile documents test the trust boundary directly; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; this tests attack success rate on adversarial web pages and zero unauthorized egress calls; the actual release condition is forbidden file and network access attempts blocked in execution tests.

C:

End-to-end adversarial cases assess the combined defenses and their failure isolation; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; this tests denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the actual release condition is forbidden file and network access attempts blocked in execution tests.

D:

Direct permission tests verify security independent of model obedience; enforce role and scope in the backend execution service using credentials that cannot perform the forbidden action; this tests zero payment writes when tested with the analytics identity and forged instructions; the actual release condition is forbidden file and network access attempts blocked in execution tests.

Question 381

A consumer-facing assistant handles abusive and manipulative prompts in addition to normal support issues. Which strategy should be prioritized for this requirement?

  1. Test whether the tool layer denies unauthorized requests
  2. Measure rate and severity of safety policy violations
  3. Use tail-latency indicators for interactive promises
  4. Use task-correctness metrics tied to verified labels
  5. Track cost per verified successful task

Explanation

Correct Answer

B:

Define safety incident categories, risk-weighted violation rates, and appropriate exposure denominators for adversarial and real traffic; risk-weighted scoring detects rare critical violations that a single average hides; evaluate the design with severity-weighted policy violation incidence on a reviewed adversarial test set.

Incorrect Answers

A:

Treat backend authorization denial rate and cross-scope access tests as primary security metrics, not merely refusal wording; model refusal scores can be high even when the underlying API is vulnerable; this alternative would measure unauthorized API requests rejected under role, tenant, and resource attack matrices; it does not establish severity-weighted policy violation incidence on a reviewed adversarial test set for the described use case.

C:

Report p95 or p99 end-to-end latency over representative traffic rather than relying only on averages or raw model runtime; mean latency underweights rare long waits and omits retrieval and external-tool delays; this alternative would measure p95 time from user submission to fully validated answer under load; it does not establish severity-weighted policy violation incidence on a reviewed adversarial test set for the described use case.

D:

Measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; positive conversational tone does not establish that a legally or financially meaningful label is correct; this alternative would measure critical-class precision and recall on independently verified tax-code cases; it does not establish severity-weighted policy violation incidence on a reviewed adversarial test set for the described use case.

E:

Relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; cost per token can improve while cost per correct business outcome deteriorates; this alternative would measure fully loaded cost per correctly completed document; it does not establish severity-weighted policy violation incidence on a reviewed adversarial test set for the described use case.

Question 382

A multinational organization wants to deploy employee profiling across several jurisdictions. The launch board requires proof that privacy analysis occurred before live processing. Which governance artifact or measure should it review?

  1. Documented authorization-boundary mapping for every service handling regulated data
  2. Deletion-request completion across all inventoried repositories within policy deadlines
  3. Verified geographic processing paths for all sensitive request components
  4. Approved contracting, access controls, and PHI data-flow verification for the selected service path
  5. Completion of required privacy assessment and recorded resolution of high risks before go-live

Explanation

Correct Answer

E:

Compliance readiness must be demonstrated upstream of irreversible data processing; measure completion of required privacy assessment and recorded resolution of high risks before go-live at the release gate; involve privacy counsel to assess applicable lawful bases, transparency, rights, and whether a data-protection impact assessment is required before release.

Incorrect Answers

A:

Boundary-specific checks prevent assuming certification transfers between unrelated services; verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection; this tests documented authorization-boundary mapping for every service handling regulated data; the actual release condition is completion of required privacy assessment and recorded resolution of high risks before go-live.

B:

End-to-end lifecycle tests verify more than the source system's delete operation; map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion; this tests deletion-request completion across all inventoried repositories within policy deadlines; the actual release condition is completion of required privacy assessment and recorded resolution of high risks before go-live.

C:

A full data-flow audit finds secondary exports that primary-storage checks miss; trace storage, inference, retrieval, logs, backups, and subprocessors against contractual and legal location requirements; this tests verified geographic processing paths for all sensitive request components; the actual release condition is completion of required privacy assessment and recorded resolution of high risks before go-live.

D:

Compliance verification must match the actual product configuration and agreement; confirm the specific service and contract support relevant HIPAA obligations, including any required business associate agreement and safeguards; this tests approved contracting, access controls, and PHI data-flow verification for the selected service path; the actual release condition is completion of required privacy assessment and recorded resolution of high risks before go-live.

Question 383

Staff treat a generated interpretation as an authoritative legal decision for a disputed case. What design correction resolves the reported failure?

  1. Escalate binding legal interpretations to qualified reviewers
  2. Give reviewers only necessary sensitive information
  3. Collect reviewer overrides as structured improvement signals
  4. Require stronger approval paths for exceptional high-impact actions

Explanation

Correct Answer

A:

Define escalation thresholds and route consequential interpretations to qualified legal or compliance owners with source context; a consequence-aware test checks whether unsupported authoritative decisions are avoided; verify the repair using correct escalation of ambiguous legal cases with reviewer attribution.

Incorrect Answers

B:

Human oversight is not permission to expand access to unrelated personal information; minimize, redact, and scope information in review interfaces according to purpose and authorized reviewer role; that control is assessed by unauthorized field exposure and successful review accuracy after data minimization; the incident instead requires correct escalation of ambiguous legal cases with reviewer attribution.

C:

Counting approval clicks alone does not describe why humans disagreed with the AI; record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign; that control is assessed by recurrence rate of categorized, previously corrected error types; the incident instead requires correct escalation of ambiguous legal cases with reviewer attribution.

D:

A uniform low-friction approval does not reflect differing blast radius and operational risk; apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations; that control is assessed by critical-action dual-approval compliance and zero bypasses during staged tests; the incident instead requires correct escalation of ambiguous legal cases with reviewer attribution.

Question 384

A proposed agent rewrite changes how billing corrections would be submitted to production tools. What action best fits the business and technical boundary?

  1. Predefine success criteria and sample-size assumptions
  2. Use shadow evaluation before enabling new side effects
  3. Require an explicit rollback trigger for harmful regressions
  4. Randomize consistently at the user or account level

Explanation

Correct Answer

B:

Replay or mirror representative inputs through a non-mutating shadow path, compare decisions, then gate any live actions separately; non-mutating evidence reduces risk before enabling operational authority; evaluate the design with shadow-policy agreement and zero write side effects across replayed billing cases.

Incorrect Answers

A:

Define primary metrics, minimal meaningful effect, sample-size method, and stopping rules before examining results; repeatedly peeking and stopping at a favorable fluctuation inflates false-positive decisions; this alternative would measure pre-registered experiment compliance and interval estimates at the planned sample size; it does not establish shadow-policy agreement and zero write side effects across replayed billing cases for the described use case.

C:

Establish automated and human-owned rollback triggers tied to critical error and unauthorized-action indicators; monitoring without an action threshold can document harm while allowing it to continue; this alternative would measure time from triggered guardrail breach to verified rollback; it does not establish shadow-policy agreement and zero write side effects across replayed billing cases for the described use case.

D:

Assign a stable randomized unit aligned to the behavior being measured and maintain consistent treatment during the experiment; request-by-request randomization can be invalid when outcomes depend on ongoing user interactions; this alternative would measure balanced treatment cohorts and clean user-level exposure histories; it does not establish shadow-policy agreement and zero write side effects across replayed billing cases for the described use case.

Question 385

A high-volume inference platform samples telemetry to control observability costs. Rare fault scenarios are seeded during a load run. Which test result would demonstrate the sampling policy still captures critical evidence?

  1. Error diagnosability with seeded sensitive fields absent from collected logs
  2. Capture rate of seeded critical failures at a bounded telemetry cost
  3. Stage-specific p95 deltas identifying injected bottlenecks
  4. Time to detect and stop a simulated unauthorized bulk-write burst

Explanation

Correct Answer

B:

Risk-aware sampling preserves evidence for rare faults without uncontrolled storage growth; measure capture rate of seeded critical failures at a bounded telemetry cost at the release gate; retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls.

Incorrect Answers

A:

Redaction and observability must be validated together, not traded off blindly; record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; this tests error diagnosability with seeded sensitive fields absent from collected logs; the actual release condition is capture rate of seeded critical failures at a bounded telemetry cost.

C:

Budget attribution points remediation at the stage that truly regressed; instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; this tests stage-specific p95 deltas identifying injected bottlenecks; the actual release condition is capture rate of seeded critical failures at a bounded telemetry cost.

D:

Action telemetry and response drills measure whether harmful changes can be contained promptly; monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; this tests time to detect and stop a simulated unauthorized bulk-write burst; the actual release condition is capture rate of seeded critical failures at a bounded telemetry cost.

Question 386

After a connector release, normal transaction volume remains flat but prompts become much larger. Which detection measure would validate a cost-anomaly alert before a billing surprise?

  1. Safety precision/recall and legitimate-task completion before and after the change
  2. Detection lag for injected per-request token inflation
  3. Quality delta between recent audited production samples and the launch baseline
  4. Detection of version-specific quality regressions in a staged rollout

Explanation

Correct Answer

B:

Unit-level telemetry catches runaway expense before invoices provide delayed evidence; measure detection lag for injected per-request token inflation at the release gate; monitor input/output tokens, cache reads, per-stage costs, and cost-per-task anomalies with accountable ownership.

Incorrect Answers

A:

Paired monitoring distinguishes beneficial safety controls from excessive refusals; monitor both harmful-output violations and false-positive blocks by version and user segment; this tests safety precision/recall and legitimate-task completion before and after the change; the actual release condition is detection lag for injected per-request token inflation.

C:

Live sampling reveals degradation that unchanged historical tests cannot see; periodically sample representative production outputs for labeling, segment analysis, and drift signals without retaining unnecessary personal data; this tests quality delta between recent audited production samples and the launch baseline; the actual release condition is detection lag for injected per-request token inflation.

D:

Release segmentation isolates changes before broad impact; partition traces and scored outcomes by deployed configuration, using release identifiers to support rollback and diagnosis; this tests detection of version-specific quality regressions in a staged rollout; the actual release condition is detection lag for injected per-request token inflation.

Question 387

A portfolio assistant requests a report assembled from several external systems and the job can take minutes. Which option addresses the immediate design need?

  1. Return a clearly marked partial answer rather than miss the deadline
  2. Move slow operations into asynchronous jobs with progress states
  3. Parallelize independent read-only tool calls
  4. Rerank a short candidate set for ambiguous retrieval queries

Explanation

Correct Answer

B:

Acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; durable job identity decouples user interaction time from background work duration; evaluate the design with job completion reliability and duplicate-submission rate during slow-system simulations.

Incorrect Answers

C:

Execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; parallel execution offers no benefit when one call requires another's output or modifies shared state; this alternative would measure end-to-end p95 improvement with all required sources still present; it does not establish job completion reliability and duplicate-submission rate during slow-system simulations for the described use case.

D:

Retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; taking every retrieved hit into context may raise token cost and distraction without raising answer quality; this alternative would measure grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths; it does not establish job completion reliability and duplicate-submission rate during slow-system simulations for the described use case.

A:

Keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; this alternative would measure deadline compliance and correct indication of omitted enrichment; it does not establish job completion reliability and duplicate-submission rate during slow-system simulations for the described use case.

Question 388

Executives cannot connect technical investment to financial impact, service quality, or residual risk. What design correction resolves the reported failure?

  1. Maintain a risk register with controls and accountable owners
  2. Present executive decisions through outcomes, cost, and risk
  3. Keep a versioned decision and change history
  4. Demonstrate known limitations alongside successful workflows

Explanation

Correct Answer

B:

Summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; decision-focused review checks whether the audience can act on the information; verify the repair using sponsor comprehension of expected outcomes, decision options, and risk ownership.

Incorrect Answers

A:

A high-level statement that AI has risks does not create an actionable governance plan; record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; that control is assessed by completion and ownership of high-severity mitigations before the release gate; the incident instead requires sponsor comprehension of expected outcomes, decision options, and risk ownership.

C:

Chat history and scattered slides rarely provide an authoritative current decision state; maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; that control is assessed by correct retrieval of active decisions and their supersession chain during review; the incident instead requires sponsor comprehension of expected outcomes, decision options, and risk ownership.

D:

A demo focused solely on best cases can mask residual risks and create unrealistic expectations; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; that control is assessed by stakeholder identification of boundaries and fallback responsibilities after the demo; the incident instead requires sponsor comprehension of expected outcomes, decision options, and risk ownership.

Question 389

A Claude agent has several similarly named lookup tools and picks the wrong tenant-filtered search. What action best fits the business and technical boundary?

  1. Write unambiguous tool descriptions and input schemas
  2. Remove duplicate and deprecated tools from the catalog
  3. Audit tool manifests as versioned production configuration
  4. Offer each agent only the tools required for its role

Explanation

Correct Answer

A:

Define task-specific descriptions, typed arguments, required fields, and clear side-effect semantics so tool selection is meaningful; confusion tests exercise the ambiguity that caused misrouting; evaluate the design with correct tool-selection rate for ambiguous intents and invalid-argument rejection.

Incorrect Answers

B:

Consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; more tool variants do not add genuine capability when they expose the same action with conflicting contracts; this alternative would measure single correct connector selection and consistent ticket states across repeated requests; it does not establish correct tool-selection rate for ambiguous intents and invalid-argument rejection for the described use case.

C:

Maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; ad hoc tool registration lacks the inventory needed to catch capability expansion before deployment; this alternative would measure changes rejected when requested tool scopes exceed the approved task manifest; it does not establish correct tool-selection rate for ambiguous intents and invalid-argument rejection for the described use case.

D:

Give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; monitoring a broad tool catalog does not remove the authority to invoke dangerous operations; this alternative would measure unauthorized tool availability and invocation count in hostile transcript tests; it does not establish correct tool-selection rate for ambiguous intents and invalid-argument rejection for the described use case.

Question 390

Human sign-off remains present on paper, but audits show reviewers never inspect contradictory source details. How should the design be corrected?

  1. Validate tool-result provenance and semantics
  2. Counter automation bias with visible uncertainty and review
  3. Constrain outbound channels to prevent data exfiltration
  4. Threat-model prompt injection across lower-trust inputs

Explanation

Correct Answer

B:

Design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button; the review test measures meaningful human oversight, not click-through compliance; verify the repair using reviewer error interception on seeded incorrect recommendations.

Incorrect Answers

A:

A successful HTTP status does not mean returned data has decision authority; preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; that control is assessed by zero external-result fields promoted to internal approvals in test cases; the incident instead requires reviewer error interception on seeded incorrect recommendations.

C:

Redaction of the final chat answer does not prevent a tool from transmitting data elsewhere; restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; that control is assessed by blocked unauthorized destinations and absence of confidential fields in egress logs; the incident instead requires reviewer error interception on seeded incorrect recommendations.

D:

Stronger wording in the system prompt is not a complete defense against an authorized dangerous tool; model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; that control is assessed by unauthorized tool-action success rate over injected emails and documents; the incident instead requires reviewer error interception on seeded incorrect recommendations.

Claude Certified Architect - Professional (CCAR-P) Practice Tests