Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 23

Question 220

A customer-service agent uses a write-capable connector with a documented role policy. What is the most defensible next step?

  1. Quantify rate and severity of safety policy violations
  2. Test whether the tool layer denies unauthorized requests
  3. Evaluate whether abstention and escalation are calibrated
  4. Use task-correctness metrics tied to verified labels

Explanation

Correct Answer

B:

Treat backend authorization denial rate and cross-scope access tests as primary security metrics, not merely refusal wording; direct penetration-style checks assess the enforceable security boundary; evaluate the design with unauthorized API requests rejected under role, tenant, and resource attack matrices.

Incorrect Answers

A:

Define safety incident categories, risk-weighted violation rates, and appropriate exposure denominators for adversarial and real traffic; general task correctness does not measure policy-conformance failures on dangerous requests; this alternative would measure severity-weighted policy violation incidence on a reviewed adversarial test set; it does not establish unauthorized API requests rejected under role, tenant, and resource attack matrices for the described use case.

C:

Measure selective risk: error rate on answered items alongside abstention precision, coverage, and escalation cost; high refusal frequency alone can appear safe while making the assistant operationally useless; this alternative would measure correct-answer rate at each supported coverage threshold including no-answer cases; it does not establish unauthorized API requests rejected under role, tenant, and resource attack matrices for the described use case.

D:

Measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; positive conversational tone does not establish that a legally or financially meaningful label is correct; this alternative would measure critical-class precision and recall on independently verified tax-code cases; it does not establish unauthorized API requests rejected under role, tenant, and resource attack matrices for the described use case.

Question 221

The team checks only the incoming prompt and misses unsafe specificity generated in the final answer. Which change should the architect recommend first?

  1. Screen unsafe outputs in proportion to the use case
  2. Apply controls at input, tool execution, and output boundaries
  3. Treat retrieved instructions as untrusted data
  4. Allowlist generated fields and destinations before external calls

Explanation

Correct Answer

A:

Evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; output-layer tests show whether the actual delivered material meets policy constraints; verify the repair using harmful-completion detection on labeled outputs with false-positive monitoring.

Incorrect Answers

B:

A single content filter cannot enforce account privileges or validate downstream business transactions; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; that control is assessed by denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the incident instead requires harmful-completion detection on labeled outputs with false-positive monitoring.

C:

Assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; that control is assessed by attack success rate on adversarial web pages and zero unauthorized egress calls; the incident instead requires harmful-completion detection on labeled outputs with false-positive monitoring.

D:

Formatting the request as JSON does not make arbitrary field values safe to transmit; validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; that control is assessed by zero unapproved recipients or fields accepted by the outbound service; the incident instead requires harmful-completion detection on labeled outputs with false-positive monitoring.

Question 222

A field service app must interpret photographed equipment labels alongside written maintenance notes. Which strategy should be prioritized for this requirement?

  1. Gate model-version updates on regression tests
  2. Adopt confidence-aware routing with a bounded fallback
  3. Benchmark a smaller model for routine classification
  4. Select a model path compatible with required modalities
  5. Use stronger reasoning capacity for genuinely complex decisions

Explanation

Correct Answer

D:

Confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; a multimodal acceptance set measures the capability absent from text-only trials; evaluate the design with correct equipment identification and recommendation accuracy on labeled photographs.

Incorrect Answers

A:

Treat model replacements as versioned releases with task-specific eval gates, contract tests, canaries, and rollback capability; newer model branding alone cannot ensure identical tool-use or output behavior in an existing workflow; this alternative would measure no contract regressions and acceptable quality/latency deltas on the proposed release; it does not establish correct equipment identification and recommendation accuracy on labeled photographs for the described use case.

B:

Combine an evaluated uncertainty trigger with escalation to a stronger model or reviewer, using explicit limits and post-route quality measurement; blindly using one model tier on all cases neither controls the hardest error modes nor optimizes cost; this alternative would measure high-risk escalation recall and total cost under the required error ceiling; it does not establish correct equipment identification and recommendation accuracy on labeled photographs for the described use case.

C:

Compare an efficient Claude model against labeled classification cases and select the lowest-cost model that meets the defined accuracy floor; a lightweight route should not be assumed suitable for complex multi-step decisions without a representative evaluation; this alternative would measure macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic; it does not establish correct equipment identification and recommendation accuracy on labeled photographs for the described use case.

E:

Escalate tasks with demonstrated reasoning difficulty to a capable model after measuring gains on cases involving cross-document dependencies; high reasoning capacity is wasteful if the task only extracts a known field or matches a stable label; this alternative would measure correct resolution of cross-exhibit conflicts at the required latency and review cost; it does not establish correct equipment identification and recommendation accuracy on labeled photographs for the described use case.

Question 223

A support assistant's contract refers only to 'good availability' while customers report unacceptable response delays. What action best fits the business and technical boundary?

  1. Establish a documented escalation path for service incidents
  2. Route user feedback into a triaged, owned backlog
  3. Define SLOs and error budgets for agreed user outcomes
  4. Communicate release behavior changes before rollout

Explanation

Correct Answer

C:

Establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; operational SLO tests connect observed service quality to response commitments; evaluate the design with sLO compliance and prescribed actions during a simulated error-budget breach.

Incorrect Answers

A:

Define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; generic 'contact support' guidance lacks accountable timing and action for critical failures; this alternative would measure time to appropriate escalation and fallback activation during drills; it does not establish sLO compliance and prescribed actions during a simulated error-budget breach for the described use case.

B:

Classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; unstructured rating totals cannot by themselves direct a safe corrective action; this alternative would measure time from actionable feedback to triage, owner assignment, and verified fix; it does not establish sLO compliance and prescribed actions during a simulated error-budget breach for the described use case.

D:

Define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; quiet deployment may reduce coordination overhead but damages trust when workflows or permissions change; this alternative would measure affected-user awareness and support-ticket volume following staged changes; it does not establish sLO compliance and prescribed actions during a simulated error-budget breach for the described use case.

Question 224

A scientific evidence assistant must support material claims with resolvable publication locations, not just a document title. Which result would establish citation-level traceability?

  1. Publication-to-search freshness lag and absence of withdrawn clauses in results
  2. Recall at k and evidence-coverage rate for known-answer queries
  3. Unauthorized chunk exposure rate under staff permission matrices, targeting zero
  4. Fraction of material claims linked to resolvable source locations

Explanation

Correct Answer

D:

Citation resolution testing verifies that evidence can actually be audited; measure fraction of material claims linked to resolvable source locations at the release gate; store source IDs, locations, dates, and stable links with chunks and propagate them to answer citation validation.

Incorrect Answers

A:

Update/delete tests prove the index tracks the authoritative corpus; build a monitored ingestion path for adds, updates, deletes, and effective dates, with freshness checks and removal of superseded chunks; this tests publication-to-search freshness lag and absence of withdrawn clauses in results; the actual release condition is fraction of material claims linked to resolvable source locations.

B:

Separating retrieval metrics identifies which pipeline stage needs repair; measure retrieval recall, ranking relevance, and evidence coverage on labeled query-document pairs before optimizing generation; this tests recall at k and evidence-coverage rate for known-answer queries; the actual release condition is fraction of material claims linked to resolvable source locations.

C:

The security test must inspect retrieved inputs, not just final visible answers; apply document permissions to retrieval candidates before material enters model context or logs; this tests unauthorized chunk exposure rate under staff permission matrices, targeting zero; the actual release condition is fraction of material claims linked to resolvable source locations.

Question 225

Operations have error snippets but cannot reproduce the issue because prompt and connector versions were not captured. How should the design be corrected?

  1. Inspect permission precedence when a tool is unexpectedly denied
  2. Roll back to a known-good tool and prompt release
  3. Reproduce incidents using correlated logs and configuration versions
  4. Produce incident learning actions with accountable follow-up

Explanation

Correct Answer

C:

Collect privacy-safe traces, request IDs, environment details, and exact configuration revisions to establish a reproducible failure case; reproducibility is a prerequisite for a trustworthy fix; verify the repair using reproduction rate of sampled incidents from stored sanitized diagnostic metadata.

Incorrect Answers

A:

Editing a user preference cannot override a stronger centrally managed deny rule; review organization, project, and local permission layers in the documented precedence order before weakening security controls; that control is assessed by correct identification of enforced deny source without allowing prohibited commands; the incident instead requires reproduction rate of sampled incidents from stored sanitized diagnostic metadata.

B:

Repeated untracked hotfixes can compound an outage and obscure the initial trigger; maintain versioned releases and documented rollback steps for hooks, prompts, model routes, and connector configuration; that control is assessed by time to restore normal operation using an approved previous release; the incident instead requires reproduction rate of sampled incidents from stored sanitized diagnostic metadata.

D:

Restarting the service restores availability but does not reduce the recurrence probability; run blameless root-cause analysis, link corrective actions to owners, and add regression or operational safeguards before closing the incident; that control is assessed by recurrence rate of the diagnosed failure and completion of preventive actions; the incident instead requires reproduction rate of sampled incidents from stored sanitized diagnostic metadata.

Question 226

A federal information workflow must operate inside a specified security authorization boundary. Which documentation result would establish that every component handling controlled records is within an appropriately evaluated service path?

  1. Deletion-request completion across all inventoried repositories within policy deadlines
  2. Completion of required privacy assessment and recorded resolution of high risks before go-live
  3. Verified geographic processing paths for all sensitive request components
  4. Documented authorization-boundary mapping for every service handling regulated data

Explanation

Correct Answer

D:

Boundary-specific checks prevent assuming certification transfers between unrelated services; measure documented authorization-boundary mapping for every service handling regulated data at the release gate; verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection.

Incorrect Answers

A:

End-to-end lifecycle tests verify more than the source system's delete operation; map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion; this tests deletion-request completion across all inventoried repositories within policy deadlines; the actual release condition is documented authorization-boundary mapping for every service handling regulated data.

B:

Compliance readiness must be demonstrated upstream of irreversible data processing; involve privacy counsel to assess applicable lawful bases, transparency, rights, and whether a data-protection impact assessment is required before release; this tests completion of required privacy assessment and recorded resolution of high risks before go-live; the actual release condition is documented authorization-boundary mapping for every service handling regulated data.

C:

A full data-flow audit finds secondary exports that primary-storage checks miss; trace storage, inference, retrieval, logs, backups, and subprocessors against contractual and legal location requirements; this tests verified geographic processing paths for all sensitive request components; the actual release condition is documented authorization-boundary mapping for every service handling regulated data.

Question 227

A regulated support service is introducing stricter approvals in stages. Which adoption signal would demonstrate that affected employees were informed and that predictable behavior changes were not mistaken for outages?

  1. Time from actionable feedback to triage, owner assignment, and verified fix
  2. Delivery of agreed business and risk indicators on schedule with decision-ready interpretation
  3. SLO compliance and prescribed actions during a simulated error-budget breach
  4. Affected-user awareness and support-ticket volume following staged changes

Explanation

Correct Answer

D:

Communication quality is reflected in predictable adoption and fewer avoidable surprises; measure affected-user awareness and support-ticket volume following staged changes at the release gate; define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact.

Incorrect Answers

A:

Issue lifecycle metrics show whether feedback produces controlled improvement; classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; this tests time from actionable feedback to triage, owner assignment, and verified fix; the actual release condition is affected-user awareness and support-ticket volume following staged changes.

B:

Reporting completeness lets stakeholders govern the system using relevant evidence; agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; this tests delivery of agreed business and risk indicators on schedule with decision-ready interpretation; the actual release condition is affected-user awareness and support-ticket volume following staged changes.

C:

Operational SLO tests connect observed service quality to response commitments; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; this tests sLO compliance and prescribed actions during a simulated error-budget breach; the actual release condition is affected-user awareness and support-ticket volume following staged changes.

Question 228

An application team enabled caching but sees no reduction in actual billable work. Which strategy should be prioritized for this requirement?

  1. Instrument cache effectiveness with token usage counters
  2. Modularize shared task instructions for consistent reuse
  3. Load specialized Skills only when the task needs them
  4. Keep dynamic fields out of cached prefix content

Explanation

Correct Answer

A:

Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; cache-specific counters distinguish a configuration issue from ordinary inference variance; evaluate the design with sustained cache-read token share under repeated workload replay.

Incorrect Answers

B:

Maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; this alternative would measure cross-application prompt-version alignment and regression pass rate; it does not establish sustained cache-read token share under repeated workload replay for the described use case.

C:

Keep Skills discoverable and load task-relevant instructions or resources on demand, maintaining scope and permission constraints; skills provide reusable guidance, not a separate security boundary or a substitute for approved tool permissions; this alternative would measure task success with reduced irrelevant instruction tokens across mixed workflows; it does not establish sustained cache-read token share under repeated workload replay for the described use case.

D:

Move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; a cache breakpoint cannot compensate for changes to earlier content in the hashed prefix; this alternative would measure prefix reuse ratio when request metadata changes across calls; it does not establish sustained cache-read token share under repeated workload replay for the described use case.

Question 229

The assistant reports permission errors only in the centrally managed enterprise environment. What is the most direct remediation?

  1. Produce incident learning actions with accountable follow-up
  2. Distinguish model selection faults from tool execution failures
  3. Replay failures safely with sanitized inputs
  4. Inspect permission precedence when a tool is unexpectedly denied

Explanation

Correct Answer

D:

Review organization, project, and local permission layers in the documented precedence order before weakening security controls; precedence tests prevent insecure workarounds that bypass governance intent; verify the repair using correct identification of enforced deny source without allowing prohibited commands.

Incorrect Answers

A:

Restarting the service restores availability but does not reduce the recurrence probability; run blameless root-cause analysis, link corrective actions to owners, and add regression or operational safeguards before closing the incident; that control is assessed by recurrence rate of the diagnosed failure and completion of preventive actions; the incident instead requires correct identification of enforced deny source without allowing prohibited commands.

B:

A more capable model cannot read data from a tool that never returns successfully; classify tool errors separately from model responses, validate return codes, and probe the failing integration before changing prompts; that control is assessed by correct root-cause attribution under injected search timeouts; the incident instead requires correct identification of enforced deny source without allowing prohibited commands.

C:

Raw transcript reuse can expose confidential inputs and violate data-handling policies; build redacted, access-controlled replay fixtures preserving relevant syntax and error conditions without unnecessary secrets or personal data; that control is assessed by bug reproduction on sanitized fixtures with no sensitive token disclosure; the incident instead requires correct identification of enforced deny source without allowing prohibited commands.

Question 230

A platform team allows independent squads to add tools to a shared Claude deployment without review. What is the most defensible next step?

  1. Audit tool manifests as versioned production configuration
  2. Separate read-only retrieval from mutating tools
  3. Write unambiguous tool descriptions and input schemas
  4. Stage privileged capabilities only after explicit intent confirmation

Explanation

Correct Answer

A:

Maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; policy-based manifest validation detects excessive authority before runtime exposure; evaluate the design with changes rejected when requested tool scopes exceed the approved task manifest.

Incorrect Answers

B:

Expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; a generic read-write interface offers more authority than the reporting workflow requires; this alternative would measure zero mutation permissions for report-only identities across tool contract checks; it does not establish changes rejected when requested tool scopes exceed the approved task manifest for the described use case.

C:

Define task-specific descriptions, typed arguments, required fields, and clear side-effect semantics so tool selection is meaningful; a richer schema improves selection but does not override a backend that ignores caller authorization; this alternative would measure correct tool-selection rate for ambiguous intents and invalid-argument rejection; it does not establish changes rejected when requested tool scopes exceed the approved task manifest for the described use case.

D:

Reveal high-impact tools only within a confirmed, authorized transaction stage; maintain server-side gating independently of model choice; hiding a tool in ordinary prompts is a defense-in-depth measure, not sufficient if APIs permit unauthorized direct calls; this alternative would measure absence of privileged tool calls before the confirmed transaction stage; it does not establish changes rejected when requested tool scopes exceed the approved task manifest for the described use case.

Claude Certified Architect - Professional (CCAR-P) Practice Tests