Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 12

Question 110

A retrieval connector is suspected of increasing end-user latency. Before scaling an instrumentation change, which evidence would identify the slow stage rather than attributing the entire delay to the model?

  1. Capture rate of seeded critical failures at a bounded telemetry cost
  2. Trace completeness and correct bottleneck attribution on instrumented fault injections
  3. Stage-specific p95 deltas identifying injected bottlenecks
  4. Time to detect and stop a simulated unauthorized bulk-write burst

Explanation

Correct Answer

C:

Budget attribution points remediation at the stage that truly regressed; measure stage-specific p95 deltas identifying injected bottlenecks at the release gate; instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration.

Incorrect Answers

A:

Risk-aware sampling preserves evidence for rare faults without uncontrolled storage growth; retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; this tests capture rate of seeded critical failures at a bounded telemetry cost; the actual release condition is stage-specific p95 deltas identifying injected bottlenecks.

B:

Distributed spans expose ownership boundaries across the whole execution path; end-to-end traces with consistent request and span identifiers identify which service and step consumed time; this tests trace completeness and correct bottleneck attribution on instrumented fault injections; the actual release condition is stage-specific p95 deltas identifying injected bottlenecks.

D:

Action telemetry and response drills measure whether harmful changes can be contained promptly; monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; this tests time to detect and stop a simulated unauthorized bulk-write burst; the actual release condition is stage-specific p95 deltas identifying injected bottlenecks.

Question 111

An enterprise must finish large numbers of document summaries overnight. The architecture team is testing a batch lane and retry budget. Which result would establish that it meets the next-day throughput goal economically?

  1. Cost and successful throughput per nightly processing window
  2. All-in cost per accepted summary at a fixed mandatory-section completion rate
  3. Supported-answer rate per input token on labeled audit questions
  4. Cache-read rate, p95 latency, and cost per accepted answer under repeated queries

Explanation

Correct Answer

A:

Deadline-oriented throughput is the relevant performance objective for noninteractive work; measure cost and successful throughput per nightly processing window at the release gate; use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense.

Incorrect Answers

B:

Quality-constrained unit economics penalizes truncation and repeat work; optimize total cost per successful output with floor constraints on completeness, safety, and latency; this tests all-in cost per accepted summary at a fixed mandatory-section completion rate; the actual release condition is cost and successful throughput per nightly processing window.

C:

Evidence efficiency tests whether trimming preserves the information that matters; retrieve relevant excerpts and retain enough surrounding evidence for interpretation rather than indiscriminately filling the context window; this tests supported-answer rate per input token on labeled audit questions; the actual release condition is cost and successful throughput per nightly processing window.

D:

Joint economics and quality checks demonstrate that reuse helps without sacrificing correctness; keep a stable eligible prompt prefix and measure cache-read tokens to reduce repeated processing; this tests cache-read rate, p95 latency, and cost per accepted answer under repeated queries; the actual release condition is cost and successful throughput per nightly processing window.

Question 112

A customer assistant consults three unrelated read APIs before composing its answer. Which strategy should be prioritized for this requirement?

  1. Schedule non-urgent work through bounded batch processing
  2. Route straightforward requests to a validated economical model
  3. Parallelize independent retrieval calls on the critical path
  4. Reuse expensive stable prefixes through prompt caching

Explanation

Correct Answer

C:

Parallel retrieval with per-call budgets reduces wall time while preserving authorization and complete aggregation; the acceptance signal guards against a fast but incomplete response; evaluate the design with p95 response time with all three source checks present under load.

Incorrect Answers

A:

Use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense; batch pipelines are inappropriate where a person needs a subsecond answer for an active decision; this alternative would measure cost and successful throughput per nightly processing window; it does not establish p95 response time with all three source checks present under load for the described use case.

B:

Evaluate a smaller model on straightforward request strata and escalate only where measurable capability is needed; using the cheapest model universally could impair complex or safety-critical cases; this alternative would measure accepted resolution rate and cost per case across routed difficulty levels; it does not establish p95 response time with all three source checks present under load for the described use case.

D:

Keep a stable eligible prompt prefix and measure cache-read tokens to reduce repeated processing; selecting a smaller model alone may reduce price but does not address wasted recomputation of invariant content; this alternative would measure cache-read rate, p95 latency, and cost per accepted answer under repeated queries; it does not establish p95 response time with all three source checks present under load for the described use case.

Question 113

An audit team needs thousands of report summaries by the next working day, not interactive replies. Which option addresses the immediate design need?

  1. Parallelize independent retrieval calls on the critical path
  2. Schedule non-urgent work through bounded batch processing
  3. Reduce context using retrieval and relevance filtering
  4. Reuse expensive stable prefixes through prompt caching

Explanation

Correct Answer

B:

Use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense; deadline-oriented throughput is the relevant performance objective for noninteractive work; evaluate the design with cost and successful throughput per nightly processing window.

Incorrect Answers

A:

Parallel retrieval with per-call budgets reduces wall time while preserving authorization and complete aggregation; parallelism adds little if operations depend on each other or contend for the same rate-limited service; this alternative would measure p95 response time with all three source checks present under load; it does not establish cost and successful throughput per nightly processing window for the described use case.

C:

Retrieve relevant excerpts and retain enough surrounding evidence for interpretation rather than indiscriminately filling the context window; token truncation at arbitrary character positions can remove governing exceptions and worsen accuracy; this alternative would measure supported-answer rate per input token on labeled audit questions; it does not establish cost and successful throughput per nightly processing window for the described use case.

D:

Keep a stable eligible prompt prefix and measure cache-read tokens to reduce repeated processing; selecting a smaller model alone may reduce price but does not address wasted recomputation of invariant content; this alternative would measure cache-read rate, p95 latency, and cost per accepted answer under repeated queries; it does not establish cost and successful throughput per nightly processing window for the described use case.

Question 114

A fraud assistant ingests responses from third-party enrichment services with varying trust levels. Which strategy should be prioritized for this requirement?

  1. Counter automation bias with visible uncertainty and review
  2. Validate tool-result provenance and semantics
  3. Cap agent steps and resource expenditure
  4. Threat-model prompt injection across lower-trust inputs
  5. Constrain outbound channels to prevent data exfiltration
  6. Use claim verification to mitigate confident fabrication

Explanation

Correct Answer

B:

Preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; provenance-sensitive checks stop evidence laundering through connector outputs; evaluate the design with zero external-result fields promoted to internal approvals in test cases.

Incorrect Answers

A:

Design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button; a nominal human checkpoint is weak when the interface encourages blind confirmation; this alternative would measure reviewer error interception on seeded incorrect recommendations; it does not establish zero external-result fields promoted to internal approvals in test cases for the described use case.

C:

Set per-task step, token, cost, and elapsed-time budgets with a clear abort and escalation state; asking the model to 'be efficient' does not guarantee bounded autonomous execution; this alternative would measure maximum resource usage and orderly termination on deliberately unsolvable tasks; it does not establish zero external-result fields promoted to internal approvals in test cases for the described use case.

D:

Model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; stronger wording in the system prompt is not a complete defense against an authorized dangerous tool; this alternative would measure unauthorized tool-action success rate over injected emails and documents; it does not establish zero external-result fields promoted to internal approvals in test cases for the described use case.

E:

Restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; redaction of the final chat answer does not prevent a tool from transmitting data elsewhere; this alternative would measure blocked unauthorized destinations and absence of confidential fields in egress logs; it does not establish zero external-result fields promoted to internal approvals in test cases for the described use case.

F:

Require support from retrieved source passages and provide an explicit unknown path for unsupported claims; fluency and citation formatting cannot establish factual support on their own; this alternative would measure proportion of numeric claims supported by independently inspected references; it does not establish zero external-result fields promoted to internal approvals in test cases for the described use case.

Question 115

Treatment contamination makes one prompt appear better because prior conversations used the other strategy. Which intervention targets the defect rather than its symptoms?

  1. Randomize consistently at the user or account level
  2. Release canaries with guardrail metrics and rollback conditions
  3. Predefine success criteria and sample-size assumptions
  4. Inspect segment-level effects rather than only global lift

Explanation

Correct Answer

A:

Assign a stable randomized unit aligned to the behavior being measured and maintain consistent treatment during the experiment; stable assignments preserve causal interpretation for repeated sessions; verify the repair using balanced treatment cohorts and clean user-level exposure histories.

Incorrect Answers

B:

A single offline score cannot show all production interactions or blast radius; deploy to a bounded cohort while observing quality, errors, and tail latency, with predefined stop and rollback thresholds; that control is assessed by canary threshold compliance and time to revert a seeded regression; the incident instead requires balanced treatment cohorts and clean user-level exposure histories.

C:

Repeatedly peeking and stopping at a favorable fluctuation inflates false-positive decisions; define primary metrics, minimal meaningful effect, sample-size method, and stopping rules before examining results; that control is assessed by pre-registered experiment compliance and interval estimates at the planned sample size; the incident instead requires balanced treatment cohorts and clean user-level exposure histories.

D:

A positive overall average does not guarantee every relevant user group benefits; compare treatment effects across predeclared critical segments and require safeguards where harm would be unacceptable; that control is assessed by resolution and safety deltas by channel and accessibility segment; the incident instead requires balanced treatment cohorts and clean user-level exposure histories.

Question 116

An unrelated action tool is invoked during a summary request after a malicious sentence appears in a transcript. How should the design be corrected?

  1. Write unambiguous tool descriptions and input schemas
  2. Separate read-only retrieval from mutating tools
  3. Audit tool manifests as versioned production configuration
  4. Offer each agent only the tools required for its role

Explanation

Correct Answer

D:

Give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; capability inventory and adversarial execution tests verify that unnecessary actions are unreachable; verify the repair using unauthorized tool availability and invocation count in hostile transcript tests.

Incorrect Answers

A:

A richer schema improves selection but does not override a backend that ignores caller authorization; define task-specific descriptions, typed arguments, required fields, and clear side-effect semantics so tool selection is meaningful; that control is assessed by correct tool-selection rate for ambiguous intents and invalid-argument rejection; the incident instead requires unauthorized tool availability and invocation count in hostile transcript tests.

B:

A generic read-write interface offers more authority than the reporting workflow requires; expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; that control is assessed by zero mutation permissions for report-only identities across tool contract checks; the incident instead requires unauthorized tool availability and invocation count in hostile transcript tests.

C:

Ad hoc tool registration lacks the inventory needed to catch capability expansion before deployment; maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; that control is assessed by changes rejected when requested tool scopes exceed the approved task manifest; the incident instead requires unauthorized tool availability and invocation count in hostile transcript tests.

Question 117

A returns operation has fixed identity checks but requires open-ended investigation only for disputed delivery records. What action best fits the business and technical boundary?

  1. Ground dynamic policy answers with retrieval
  2. Embed a small agent inside a controlled workflow
  3. Use a single augmented-LLM call for a bounded text transformation
  4. Use a bounded tool-using agent for exploratory investigation

Explanation

Correct Answer

B:

Place bounded exploratory reasoning inside one stage of a deterministic state machine; enforce approval and write permissions outside the agent; this tests flexible investigation and deterministic control at the same time; evaluate the design with disputed-case resolution with every refund still passing the identity and approval gates.

Incorrect Answers

A:

Retrieval from a versioned, access-filtered source grounds answers in current documents and can expose the authoritative citation; retrieval does not create new authoritative policy; it depends on freshness, correct permissions, and source quality; this alternative would measure correct citations to currently effective policy versions across update tests; it does not establish disputed-case resolution with every refund still passing the identity and approval gates for the described use case.

C:

For a fixed transformation, a prompt enriched with only needed context is simpler, cheaper, and easier to evaluate than an agentic loop; a direct call is poor for tasks that require unknown sequences of external discovery or system writes; this alternative would measure summary fidelity and p95 completion time without any unapproved tool invocation; it does not establish disputed-case resolution with every refund still passing the identity and approval gates for the described use case.

D:

Bounded agentic planning chooses follow-up tools from observed evidence while enforcing time, budget, and tool permissions; agent autonomy adds cost and uncertainty to tasks whose steps are already predictable and linear; this alternative would measure resolution of unseen diagnostic branches within a specified tool-call and time budget; it does not establish disputed-case resolution with every refund still passing the identity and approval gates for the described use case.

Question 118

An applicant receives an adverse automated recommendation, discovers incorrect source information, and cannot request a correction. What recourse capability must the organization provide?

  1. Provide a route to challenge significant recommendations
  2. Quantify errors across relevant user subgroups
  3. Publish scope, limitations, and residual risks to stakeholders
  4. Expose evidence and decision boundaries to affected users
  5. Design an override path proportionate to consequences

Explanation

Correct Answer

A:

Provide a contest and correction process with accountable human ownership for significant impacts; remedy effectiveness measures whether affected people can obtain meaningful reconsideration; verify the repair using time to resolve substantiated challenges and proportion of cases with documented review.

Incorrect Answers

B:

Overall accuracy cannot establish comparable treatment for every affected population; evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; that control is assessed by quality and escalation parity across predeclared language segments; the incident instead requires time to resolve substantiated challenges and proportion of cases with documented review.

C:

A generic disclaimer cannot compensate for hidden high-risk use or unavailable oversight; maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; that control is assessed by accuracy of user understanding of supported versus unsupported tasks; the incident instead requires time to resolve substantiated challenges and proportion of cases with documented review.

D:

A polished explanation without provenance may create false confidence in unsupported reasoning; distinguish evidence, uncertainty, and advisory status with accessible citations and a path to authoritative decisions; that control is assessed by user comprehension of advisory status and source-citation validity; the incident instead requires time to resolve substantiated challenges and proportion of cases with documented review.

E:

Human review after an irreversible action may be too late to prevent harm; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; that control is assessed by successful intervention before seeded high-impact incorrect actions are committed; the incident instead requires time to resolve substantiated challenges and proportion of cases with documented review.

Question 119

An audit reporting tool must produce distinct structured records for observed evidence and future corrective actions. Which result on unseen mixed samples would prove it applies the right instruction to each record?

  1. Label accuracy and token use versus a heavier example-based prompt
  2. Recall on previously confused boundary examples outside the prompt set
  3. Intermediate artifact validity and final score agreement with a policy-engine oracle
  4. Correct type selection on unseen evidence-versus-action pairs

Explanation

Correct Answer

D:

Contrastive tests verify the semantic boundary, not just the presence of a familiar template; measure correct type selection on unseen evidence-versus-action pairs at the release gate; show a paired positive and negative example with the decision rule that separates the formats; then test unseen cases.

Incorrect Answers

A:

Controlled comparisons show whether added prompt complexity yields a genuine benefit; a concise task statement with precise labels is a defensible baseline when success is readily evaluated and examples add little value; this tests label accuracy and token use versus a heavier example-based prompt; the actual release condition is correct type selection on unseen evidence-versus-action pairs.

B:

Held-out boundary performance shows whether examples teach the distinction rather than merely echoing training examples; provide a few diverse, accurately labeled counterexamples that illustrate the exact class boundary, then verify against held-out cases; this tests recall on previously confused boundary examples outside the prompt set; the actual release condition is correct type selection on unseen evidence-versus-action pairs.

C:

Checkable milestones provide evidence of reasoning quality beyond rhetorical fluency; require explicit intermediate evidence and structured rule checks that can be independently validated; avoid treating verbose reasoning text as proof; this tests intermediate artifact validity and final score agreement with a policy-engine oracle; the actual release condition is correct type selection on unseen evidence-versus-action pairs.

Question 120

An agent has concise names for many tools but full schemas contain lengthy examples and edge-case descriptions. What should the solution architect recommend?

  1. Retrieve detailed reference material only when requested
  2. Preserve stable prefixes while dynamically discovering tools
  3. Fetch detailed schemas only for shortlisted actions
  4. Load capability groups after intent classification

Explanation

Correct Answer

C:

Keep concise discovery metadata initially and retrieve the precise argument schema when a tool becomes actionable; the test balances context savings against input-contract reliability; evaluate the design with schema-token consumption and invalid-argument rate after staged loading.

Incorrect Answers

A:

Discover resource summaries first and fetch the specific authoritative documentation section at the point of need; reference discovery without source-version checks can still supply outdated interface information; this alternative would measure correct endpoint contract use with lower documentation-token overhead; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.

B:

Isolate stable instructions from changing discovery payloads and measure cache behavior when tool inventories vary; progressive discovery can lose its token advantage if dynamic tool definitions repeatedly destroy cache reuse; this alternative would measure cache-read tokens and total context cost while changing selected tool subsets; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.

D:

Select a bounded task-specific capability group after intent routing with conservative fallback for ambiguous requests; capability-group routing cannot replace policy enforcement once a sensitive tool is selected; this alternative would measure irrelevant tool exposure per intent with no loss of required task completion; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.

Claude Certified Architect - Professional (CCAR-P) Practice Tests