Question 120
An agent has concise names for many tools but full schemas contain lengthy examples and edge-case descriptions. What should the solution architect recommend?
- Retrieve detailed reference material only when requested
- Preserve stable prefixes while dynamically discovering tools
- Fetch detailed schemas only for shortlisted actions
- Load capability groups after intent classification
Explanation
Correct Answer
C:Keep concise discovery metadata initially and retrieve the precise argument schema when a tool becomes actionable; the test balances context savings against input-contract reliability; evaluate the design with schema-token consumption and invalid-argument rate after staged loading.
Incorrect Answers
A:Discover resource summaries first and fetch the specific authoritative documentation section at the point of need; reference discovery without source-version checks can still supply outdated interface information; this alternative would measure correct endpoint contract use with lower documentation-token overhead; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.
B:Isolate stable instructions from changing discovery payloads and measure cache behavior when tool inventories vary; progressive discovery can lose its token advantage if dynamic tool definitions repeatedly destroy cache reuse; this alternative would measure cache-read tokens and total context cost while changing selected tool subsets; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.
D:Select a bounded task-specific capability group after intent routing with conservative fallback for ambiguous requests; capability-group routing cannot replace policy enforcement once a sensitive tool is selected; this alternative would measure irrelevant tool exposure per intent with no loss of required task completion; it does not establish schema-token consumption and invalid-argument rate after staged loading for the described use case.
Question 121
A coding agent has dozens of detailed operating playbooks, but most sessions need only a small subset. Which design decision directly addresses this constraint?
- Compare cache lifetime with actual reuse patterns
- Keep dynamic fields out of cached prefix content
- Load specialized Skills only when the task needs them
- Modularize shared task instructions for consistent reuse
Explanation
Correct Answer
C:Keep Skills discoverable and load task-relevant instructions or resources on demand, maintaining scope and permission constraints; on-demand loading must retain quality while avoiding unnecessary context overhead; evaluate the design with task success with reduced irrelevant instruction tokens across mixed workflows.
Incorrect Answers
A:Assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; a long cache lifetime is not automatically economical when there is little repeated work; this alternative would measure net cost per accepted request across realistic interarrival times; it does not establish task success with reduced irrelevant instruction tokens across mixed workflows for the described use case.
B:Move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; a cache breakpoint cannot compensate for changes to earlier content in the hashed prefix; this alternative would measure prefix reuse ratio when request metadata changes across calls; it does not establish task success with reduced irrelevant instruction tokens across mixed workflows for the described use case.
D:Maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; this alternative would measure cross-application prompt-version alignment and regression pass rate; it does not establish task success with reduced irrelevant instruction tokens across mixed workflows for the described use case.
Question 122
A team is publishing guidance about what an AI sales assistant can and cannot do. Before launch, which user-testing result would establish that the boundaries are understood rather than just written in policy?
- Successful intervention before seeded high-impact incorrect actions are committed
- Accuracy of user understanding of supported versus unsupported tasks
- Task completion by users of tested assistive technologies on representative flows
- Time to resolve substantiated challenges and proportion of cases with documented review
Explanation
Correct Answer
B:Scenario tests reveal whether disclosures prevent inappropriate reliance; measure accuracy of user understanding of supported versus unsupported tasks at the release gate; maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements.
Incorrect Answers
A:Pre-commit override drills test real controllability rather than retrospective documentation; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; this tests successful intervention before seeded high-impact incorrect actions are committed; the actual release condition is accuracy of user understanding of supported versus unsupported tasks.
C:Direct accessibility testing measures practical use rather than formal content correctness only; include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; this tests task completion by users of tested assistive technologies on representative flows; the actual release condition is accuracy of user understanding of supported versus unsupported tasks.
D:Remedy effectiveness measures whether affected people can obtain meaningful reconsideration; provide a contest and correction process with accountable human ownership for significant impacts; this tests time to resolve substantiated challenges and proportion of cases with documented review; the actual release condition is accuracy of user understanding of supported versus unsupported tasks.
Question 123
Users cannot tell which statements are sourced facts and which are model-generated inferences. Which corrective change best addresses the root cause?
- Design an override path proportionate to consequences
- Measure errors across relevant user subgroups
- Publish scope, limitations, and residual risks to stakeholders
- Test accessibility of generated explanations and interfaces
- Expose evidence and decision boundaries to affected users
Explanation
Correct Answer
E:Distinguish evidence, uncertainty, and advisory status with accessible citations and a path to authoritative decisions; transparent decision tests measure whether users understand what the system did and did not decide; verify the repair using user comprehension of advisory status and source-citation validity.
Incorrect Answers
A:Human review after an irreversible action may be too late to prevent harm; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; that control is assessed by successful intervention before seeded high-impact incorrect actions are committed; the incident instead requires user comprehension of advisory status and source-citation validity.
B:Overall accuracy cannot establish comparable treatment for every affected population; evaluate suitable subgroup error rates and access outcomes, with privacy-aware sampling and appropriate fairness review; that control is assessed by quality and escalation parity across predeclared language segments; the incident instead requires user comprehension of advisory status and source-citation validity.
C:A generic disclaimer cannot compensate for hidden high-risk use or unavailable oversight; maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; that control is assessed by accuracy of user understanding of supported versus unsupported tasks; the incident instead requires user comprehension of advisory status and source-citation validity.
D:Factual accuracy alone does not make the interface usable to everyone; include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; that control is assessed by task completion by users of tested assistive technologies on representative flows; the incident instead requires user comprehension of advisory status and source-citation validity.
Question 124
The final report stops mid-recommendation after excessive context has consumed the request budget. Which change should the architect recommend first?
- Allocate input and output token budgets per workflow step
- Persist operational decisions outside transient chat history
- Remove obsolete tool output while preserving current evidence
- Retrieve only relevant material instead of full archives
Explanation
Correct Answer
A:Reserve output capacity and bound each stage's retrieved context and response length according to the deliverable structure; budgeted completion tests directly reveal truncation at maximum expected load; verify the repair using complete mandatory report sections under the worst supported input size.
Incorrect Answers
B:Long context alone cannot provide transactional durability or a trustworthy authorization trail; store approvals and workflow state in an authoritative durable system; treat chat history as context, not a transaction ledger; that control is assessed by correct recovery of approvals and pending actions after forced session reset; the incident instead requires complete mandatory report sections under the worst supported input size.
C:Pruning without version markers can erase the evidence needed to explain the incident's evolution; prune or summarize superseded tool data, preserve timestamps and current authoritative observations, and fetch details only when needed; that control is assessed by current-state citation accuracy across log-update sequences; the incident instead requires complete mandatory report sections under the worst supported input size.
D:Broad context stuffing can degrade relevance and cost even when a large context window is available; apply query-driven retrieval and evidence selection before prompting Claude, retaining citations and permission filters; that control is assessed by answer support and retrieved-token volume on a sampled policy-question set; the incident instead requires complete mandatory report sections under the worst supported input size.
Question 125
Reviewers manually inspect schema validity and overlook intermittent invalid enumerations. How should the design be corrected?
- Protect a separate holdout set for regression decisions
- Use deterministic code checks for objective output contracts
- Use expert rubrics for nuanced qualitative outputs
- Include hostile prompt-injection and no-answer probes
- Stratify a golden dataset by real workload segments
- Measure human annotator consistency before trusting labels
Explanation
Correct Answer
B:Apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; executable contract tests deliver reproducible feedback at scale; verify the repair using zero schema violations accepted by the downstream consumer across generated cases.
Incorrect Answers
A:Reusing a public tuning set as the sole release gate invites overfitting rather than generalization; maintain untouched holdout cases and periodic refreshes representing real traffic, with clear version and leakage controls; that control is assessed by holdout quality stability across successive prompt versions; the incident instead requires zero schema violations accepted by the downstream consumer across generated cases.
C:Pure string comparison is unsuitable when many substantively correct phrasings exist; use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; that control is assessed by expert-rated factual fidelity and calibrated grader agreement on sampled explanations; the incident instead requires zero schema violations accepted by the downstream consumer across generated cases.
D:Testing normal prompts alone cannot estimate resilience to malicious or unsatisfiable requests; add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; that control is assessed by attack success rate and safe handling of intentionally unanswerable prompts; the incident instead requires zero schema violations accepted by the downstream consumer across generated cases.
E:A random sample alone can underrepresent important low-frequency and high-consequence events; build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; that control is assessed by per-stratum pass rates with no uncovered critical scenario classes; the incident instead requires zero schema violations accepted by the downstream consumer across generated cases.
F:More test volume does not fix unreliable or undefined labels; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; that control is assessed by inter-annotator agreement and unresolved-label rate after adjudication; the incident instead requires zero schema violations accepted by the downstream consumer across generated cases.
Question 126
A single user request triggers hundreds of repeated tool calls and consumes the daily budget. What design correction resolves the reported failure?
- Threat-model prompt injection across lower-trust inputs
- Counter automation bias with visible uncertainty and review
- Constrain outbound channels to prevent data exfiltration
- Validate tool-result provenance and semantics
- Cap agent steps and resource expenditure
Explanation
Correct Answer
E:Set per-task step, token, cost, and elapsed-time budgets with a clear abort and escalation state; budget stress tests show whether runaway loops fail safely; verify the repair using maximum resource usage and orderly termination on deliberately unsolvable tasks.
Incorrect Answers
A:Stronger wording in the system prompt is not a complete defense against an authorized dangerous tool; model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; that control is assessed by unauthorized tool-action success rate over injected emails and documents; the incident instead requires maximum resource usage and orderly termination on deliberately unsolvable tasks.
B:A nominal human checkpoint is weak when the interface encourages blind confirmation; design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button; that control is assessed by reviewer error interception on seeded incorrect recommendations; the incident instead requires maximum resource usage and orderly termination on deliberately unsolvable tasks.
C:Redaction of the final chat answer does not prevent a tool from transmitting data elsewhere; restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; that control is assessed by blocked unauthorized destinations and absence of confidential fields in egress logs; the incident instead requires maximum resource usage and orderly termination on deliberately unsolvable tasks.
D:A successful HTTP status does not mean returned data has decision authority; preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; that control is assessed by zero external-result fields promoted to internal approvals in test cases; the incident instead requires maximum resource usage and orderly termination on deliberately unsolvable tasks.
Question 127
A cross-team Claude deployment undergoes several prompt, tool, and policy revisions during integration. Which strategy should be prioritized for this requirement?
- Maintain a risk register with controls and accountable owners
- Keep a versioned decision and change history
- Compare options on a shared multi-dimensional scorecard
- Present executive decisions through outcomes, cost, and risk
- Demonstrate known limitations alongside successful workflows
Explanation
Correct Answer
B:Maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; version-aware retrieval prevents implementation against outdated agreements; evaluate the design with correct retrieval of active decisions and their supersession chain during review.
Incorrect Answers
A:Record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; a high-level statement that AI has risks does not create an actionable governance plan; this alternative would measure completion and ownership of high-severity mitigations before the release gate; it does not establish correct retrieval of active decisions and their supersession chain during review for the described use case.
C:Present the same quality, latency, cost, risk, and maintainability measures for each candidate under agreed weighting; isolated benchmark rankings cannot settle a decision with several business constraints; this alternative would measure stakeholder-approved selection using common scored evidence and disclosed trade-offs; it does not establish correct retrieval of active decisions and their supersession chain during review for the described use case.
D:Summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; a detailed API design is valuable to implementers but not sufficient for a funding decision; this alternative would measure sponsor comprehension of expected outcomes, decision options, and risk ownership; it does not establish correct retrieval of active decisions and their supersession chain during review for the described use case.
E:Use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; a demo focused solely on best cases can mask residual risks and create unrealistic expectations; this alternative would measure stakeholder identification of boundaries and fallback responsibilities after the demo; it does not establish correct retrieval of active decisions and their supersession chain during review for the described use case.
Question 128
A public health information service adds checks on generated responses, not just incoming questions. The evaluation team balances catching harmful advice against unnecessarily blocking safe education. Which measurement fits that task?
- Harmful-completion detection on labeled outputs with false-positive monitoring
- Denial of unauthorized actions and harmful outputs across multi-stage red-team paths
- Attack success rate on adversarial web pages and zero unauthorized egress calls
- Forbidden file and network access attempts blocked in execution tests
Explanation
Correct Answer
A:Output-layer tests show whether the actual delivered material meets policy constraints; measure harmful-completion detection on labeled outputs with false-positive monitoring at the release gate; evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims.
Incorrect Answers
B:End-to-end adversarial cases assess the combined defenses and their failure isolation; layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; this tests denial of unauthorized actions and harmful outputs across multi-stage red-team paths; the actual release condition is harmful-completion detection on labeled outputs with false-positive monitoring.
C:Simulated hostile documents test the trust boundary directly; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; this tests attack success rate on adversarial web pages and zero unauthorized egress calls; the actual release condition is harmful-completion detection on labeled outputs with false-positive monitoring.
D:Sandbox escapes and denial logs are more relevant than the assistant's stated intentions; isolate execution with restricted filesystem, network, credentials, resources, and observable policy denials; this tests forbidden file and network access attempts blocked in execution tests; the actual release condition is harmful-completion detection on labeled outputs with false-positive monitoring.
Question 129
A program commits its full implementation budget before checking a vendor integration; only later does it discover that the required endpoint is inaccessible. Which discovery control would have prevented the commitment?
- Set an evidence gate before moving beyond discovery
- Plan deprecation and rollback from the start
- Transfer durable ownership and operations before handoff
- Incorporate security and governance checks during design
Explanation
Correct Answer
A:Require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; stage gates prevent costly commitment to untested assumptions; verify the repair using closure of high-risk feasibility questions before design approval.
Incorrect Answers
B:Removing an old endpoint alone does not manage hidden dependencies or operational continuity; inventory consumers, define migration dates and fallback, preserve data obligations, and test the rollback or retirement path; that control is assessed by successful retirement with zero unplanned dependent-system failures; the incident instead requires closure of high-risk feasibility questions before design approval.
C:A completed repository is not proof of ongoing service operability; establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; that control is assessed by named owner coverage and support-drill completion at handoff; the incident instead requires closure of high-risk feasibility questions before design approval.
D:Post-build security sign-off is costly and may not undo prior unauthorized processing; incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens; that control is assessed by resolution of high-severity architecture risks before implementation approval; the incident instead requires closure of high-risk feasibility questions before design approval.
Question 130
A company processes a heavy morning batch of supplier certificates, with individual analysis taking much longer than its web request timeout. Which intervention is most appropriate for this use case?
- Validate model output against a business schema before side effects
- Use a queue and asynchronous workers for bursty workloads
- Capture reviewer corrections as labeled feedback
- Persist workflow state and make retries idempotent
Explanation
Correct Answer
B:Decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; measuring backlog and completion under load tests the exact burst tolerance the design promises; evaluate the design with job completion rate and queue age during the peak intake window.
Incorrect Answers
A:Use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; output parsing constrains generated data but does not by itself manage retry idempotency or transport durability; this alternative would measure invalid write payloads rejected before execution, with zero partial supplier mutations; it does not establish job completion rate and queue age during the peak intake window for the described use case.
C:Preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; a feedback dataset improves future evaluation; it does not guarantee that the current response is timely or properly permissioned; this alternative would measure regression pass rate on previously corrected clause-omission cases; it does not establish job completion rate and queue age during the peak intake window for the described use case.
D:Persist state transitions and use idempotency keys at side-effect boundaries to avoid duplicate transactions across replays; durable-state control prevents duplicate effects but cannot decide whether a refund is substantively authorized; this alternative would measure repeated and interrupted test runs yielding exactly one committed credit per approved case; it does not establish job completion rate and queue age during the peak intake window for the described use case.