Question 400
A user bypasses a prompt-only restriction and submits an unauthorized tool request. How should the design be corrected?
- Handle retrieved instructions as untrusted data
- Allowlist generated fields and destinations before external calls
- Screen unsafe outputs in proportion to the use case
- Apply controls at input, tool execution, and output boundaries
Explanation
Correct Answer
D:Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; end-to-end adversarial cases assess the combined defenses and their failure isolation; verify the repair using denial of unauthorized actions and harmful outputs across multi-stage red-team paths.
Incorrect Answers
A:Assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; that control is assessed by attack success rate on adversarial web pages and zero unauthorized egress calls; the incident instead requires denial of unauthorized actions and harmful outputs across multi-stage red-team paths.
B:Formatting the request as JSON does not make arbitrary field values safe to transmit; validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; that control is assessed by zero unapproved recipients or fields accepted by the outbound service; the incident instead requires denial of unauthorized actions and harmful outputs across multi-stage red-team paths.
C:An input-only filter cannot detect every unsafe completion, especially when harm arises from the generated detail; evaluate output safety against a documented risk rubric and use appropriate escalation or refusal for high-risk claims; that control is assessed by harmful-completion detection on labeled outputs with false-positive monitoring; the incident instead requires denial of unauthorized actions and harmful outputs across multi-stage red-team paths.
Question 401
A science advisory system must cite the exact study, page, and publication version supporting each claim. Which design decision directly addresses this constraint?
- Refresh the index when source content changes
- Version embeddings and reindex when representation changes
- Preserve citation metadata through retrieval and synthesis
- Chunk documents along meaningful structural boundaries
- Evaluate retrieval quality independently of generation
Explanation
Correct Answer
C:Store source IDs, locations, dates, and stable links with chunks and propagate them to answer citation validation; citation resolution testing verifies that evidence can actually be audited; evaluate the design with fraction of material claims linked to resolvable source locations.
Incorrect Answers
A:Build a monitored ingestion path for adds, updates, deletes, and effective dates, with freshness checks and removal of superseded chunks; a larger context window cannot compensate for documents missing from the index; this alternative would measure publication-to-search freshness lag and absence of withdrawn clauses in results; it does not establish fraction of material claims linked to resolvable source locations for the described use case.
B:Record embedding model and preprocessing versions, re-embed consistently, and use controlled index migration; updating only the query encoder can corrupt retrieval ranking even if the documents are unchanged; this alternative would measure retrieval recall before and after a complete embedding-index migration; it does not establish fraction of material claims linked to resolvable source locations for the described use case.
D:Chunk by headings and semantic sections with deliberate overlap where necessary while maintaining parent-document references; arbitrary equal-length chunks may split dependencies despite fitting an embedding model's input limit; this alternative would measure retrieval of both governing clause and exception for labeled contract questions; it does not establish fraction of material claims linked to resolvable source locations for the described use case.
E:Measure retrieval recall, ranking relevance, and evidence coverage on labeled query-document pairs before optimizing generation; a stronger model cannot reliably cite evidence that retrieval never supplies; this alternative would measure recall at k and evidence-coverage rate for known-answer queries; it does not establish fraction of material claims linked to resolvable source locations for the described use case.
Question 402
During a security review, nobody can identify every component that receives employee data or the point at which a privileged identity is used. What architecture documentation is missing?
- Prepare operator onboarding and support handoff materials
- Maintain an end-to-end data-flow and trust-boundary diagram
- Document decisions with assumptions and reversal triggers
- Publish an incident runbook with containment and rollback steps
- Version prompts, model routes, tools, and evaluation artifacts together
Explanation
Correct Answer
B:A current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects; architecture diagrams are useful when they expose where protection and authority change; verify the repair using completeness of sensitive-data flow paths identified during security review.
Incorrect Answers
A:Source code alone rarely communicates the operational decisions and permission steps operators need; provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; that control is assessed by successful completion of standard support drills by staff who did not build the system; the incident instead requires completeness of sensitive-data flow paths identified during security review.
C:Configuration exports show what exists but rarely why it was chosen; record architectural context, alternatives, decision rationale, constraints, and explicit conditions for revisiting choices; that control is assessed by ability to identify affected decisions after simulated policy and workload changes; the incident instead requires completeness of sensitive-data flow paths identified during security review.
D:A high-level diagram alone is not an executable incident response procedure; maintain runbooks for detection, containment, safe fallback, rollback, and communication with named incident roles; that control is assessed by successful containment time in a simulated stale-index incident; the incident instead requires completeness of sensitive-data flow paths identified during security review.
E:Recording only the model name omits other behavior-changing dependencies; maintain a release manifest linking model configuration, prompt, tool contracts, data versions, and evaluation results; that control is assessed by reproducibility of a sampled historical answer using its full manifest; the incident instead requires completeness of sensitive-data flow paths identified during security review.
Question 403
A board attends a demonstration of an automated service. The review now includes missing inputs, escalation paths, and degraded operation. Which audience result would demonstrate that nontechnical sponsors understand the limits?
- Measured reconstructability of key design decisions after a simulated staff handoff
- Completion and ownership of high-severity mitigations before the release gate
- Stakeholder identification of boundaries and fallback responsibilities after the demo
- Correct retrieval of active decisions and their supersession chain during review
Explanation
Correct Answer
C:Understanding checks reveal whether decision-makers grasp operational limits; measure stakeholder identification of boundaries and fallback responsibilities after the demo at the release gate; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership.
Incorrect Answers
A:Decision records preserve rationale beyond a particular architect's memory; use an ADR recording context, considered options, decision, trade-offs, implications, and conditions for revisiting it; this tests reconstructability of key design decisions after a simulated staff handoff; the actual release condition is stakeholder identification of boundaries and fallback responsibilities after the demo.
B:A control register makes risk decisions reviewable rather than rhetorical; record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; this tests completion and ownership of high-severity mitigations before the release gate; the actual release condition is stakeholder identification of boundaries and fallback responsibilities after the demo.
D:Version-aware retrieval prevents implementation against outdated agreements; maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; this tests correct retrieval of active decisions and their supersession chain during review; the actual release condition is stakeholder identification of boundaries and fallback responsibilities after the demo.
Question 404
A customer assistant needs independent inventory, pricing, and fulfillment reads before generating an answer. After concurrent execution is enabled, which outcome would show lower latency without missing data?
- Cache-read rate, p95 latency, and cost per accepted answer under repeated queries
- p95 response time with all three source checks present under load
- Supported-answer rate per input token on labeled audit questions
- Accepted resolution rate and cost per case across routed difficulty levels
- Cost and successful throughput per nightly processing window
Explanation
Correct Answer
B:The acceptance signal guards against a fast but incomplete response; measure p95 response time with all three source checks present under load at the release gate; parallel retrieval with per-call budgets reduces wall time while preserving authorization and complete aggregation.
Incorrect Answers
A:Joint economics and quality checks demonstrate that reuse helps without sacrificing correctness; keep a stable eligible prompt prefix and measure cache-read tokens to reduce repeated processing; this tests cache-read rate, p95 latency, and cost per accepted answer under repeated queries; the actual release condition is p95 response time with all three source checks present under load.
C:Evidence efficiency tests whether trimming preserves the information that matters; retrieve relevant excerpts and retain enough surrounding evidence for interpretation rather than indiscriminately filling the context window; this tests supported-answer rate per input token on labeled audit questions; the actual release condition is p95 response time with all three source checks present under load.
D:Stratified monitoring confirms economic gains do not displace expensive errors onto users; evaluate a smaller model on straightforward request strata and escalate only where measurable capability is needed; this tests accepted resolution rate and cost per case across routed difficulty levels; the actual release condition is p95 response time with all three source checks present under load.
E:Deadline-oriented throughput is the relevant performance objective for noninteractive work; use suitable batch orchestration and queue control to trade immediate response time for predictable throughput and expense; this tests cost and successful throughput per nightly processing window; the actual release condition is p95 response time with all three source checks present under load.
Question 405
A patient education assistant drafts explanations whose clinical nuance cannot be evaluated by exact text matching. Which approach best meets the stated requirement?
- Quantify human annotator consistency before trusting labels
- Use deterministic code checks for objective output contracts
- Include hostile prompt-injection and no-answer probes
- Use expert rubrics for nuanced qualitative outputs
- Stratify a golden dataset by real workload segments
- Protect a separate holdout set for regression decisions
Explanation
Correct Answer
D:Use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; a semantic rubric tests meaning while allowing legitimate variation in language; evaluate the design with expert-rated factual fidelity and calibrated grader agreement on sampled explanations.
Incorrect Answers
A:Use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; more test volume does not fix unreliable or undefined labels; this alternative would measure inter-annotator agreement and unresolved-label rate after adjudication; it does not establish expert-rated factual fidelity and calibrated grader agreement on sampled explanations for the described use case.
B:Apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; subjective model grading is unnecessary and less reliable for an objectively checkable parser contract; this alternative would measure zero schema violations accepted by the downstream consumer across generated cases; it does not establish expert-rated factual fidelity and calibrated grader agreement on sampled explanations for the described use case.
C:Add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; testing normal prompts alone cannot estimate resilience to malicious or unsatisfiable requests; this alternative would measure attack success rate and safe handling of intentionally unanswerable prompts; it does not establish expert-rated factual fidelity and calibrated grader agreement on sampled explanations for the described use case.
E:Build a curated, versioned golden set covering volume, business risk, language, format, and difficult edge cases; a random sample alone can underrepresent important low-frequency and high-consequence events; this alternative would measure per-stratum pass rates with no uncovered critical scenario classes; it does not establish expert-rated factual fidelity and calibrated grader agreement on sampled explanations for the described use case.
F:Maintain untouched holdout cases and periodic refreshes representing real traffic, with clear version and leakage controls; reusing a public tuning set as the sole release gate invites overfitting rather than generalization; this alternative would measure holdout quality stability across successive prompt versions; it does not establish expert-rated factual fidelity and calibrated grader agreement on sampled explanations for the described use case.
Question 406
A credit-scoring pilot combines data from systems that disagree on permitted uses and retention. Another insurance operations project promises labor savings without recording current processing effort or error rates. Which TWO early-stage discovery steps should occur before the architectures are chosen? Choose TWO.
- Map available data and its permitted uses before solution selection
- Prioritize a repetitive step with bounded exceptions
- Measure the current manual task before choosing automation
- Define which decisions remain with people
- Pilot one bounded workflow with acceptance gates
Explanation
Correct Answers
A:Data-readiness and usage-rights mapping identifies legitimate inputs, owners, quality gaps, and prohibited transfers before committing to an implementation; measured coverage of eligible sources exposes blockers that a high demo score conceals; its corresponding workstream should be validated by percentage of required fields with validated owners, quality checks, and approved processing purposes.
C:A baseline separates real business improvement from attractive demos; use completed cases, reviewer effort, errors, and exception handling as measurable comparators; comparing complete case outcomes includes downstream corrections rather than rewarding only faster draft generation; its corresponding workstream should be validated by change in verified minutes and rework per adjudicated claim versus the documented baseline.
Incorrect Answers
B:Select a high-frequency, well-described task with measurable work reduction and explicit escalation for ambiguous records; prioritization improves initial return and feasibility; its natural acceptance measure is accepted cases per reviewer-hour with exception share tracked separately, which does not satisfy the separate workstreams being assessed.
D:Explicit decision-rights boundaries distinguish advisory generation from actions requiring a credentialed human or deterministic policy engine; decision ownership is an authorization and accountability control; its natural acceptance measure is share of binding decisions showing the required named approver and rationale, which does not satisfy the separate workstreams being assessed.
E:A narrow pilot validates one representative flow and pre-agreed quality, latency, and escalation thresholds before scaling to other workflows; a pilot bounds deployment exposure; its natural acceptance measure is passing the acceptance gate on a representative pilot cohort including difficult exceptions, which does not satisfy the separate workstreams being assessed.
Question 407
Running every fragment through synchronous chat requests saturates interactive capacity with no user waiting for replies. Which change should the architect recommend first?
- Use a deterministic workflow for known ordered stages
- Ground dynamic policy answers with retrieval
- Use offline batch processing when immediate interactivity is unnecessary
- Embed a small agent inside a controlled workflow
Explanation
Correct Answer
C:Batch-oriented orchestration handles throughput, retries, and cost planning while preserving consistent model instructions for a noninteractive workload; throughput and unit economics measure the benefit that a user-facing latency metric would miss; verify the repair using completed valid classifications per batch window and cost per accepted record.
Incorrect Answers
A:A rigid sequence becomes cumbersome when the next action is genuinely unknown until exploratory results arrive; explicit state transitions preserve required order, testing points, and deterministic branch conditions; Claude can supply judgment inside selected stages; that control is assessed by rate of recommendations backed by all required completed stage checks; the incident instead requires completed valid classifications per batch window and cost per accepted record.
B:Retrieval does not create new authoritative policy; it depends on freshness, correct permissions, and source quality; retrieval from a versioned, access-filtered source grounds answers in current documents and can expose the authoritative citation; that control is assessed by correct citations to currently effective policy versions across update tests; the incident instead requires completed valid classifications per batch window and cost per accepted record.
D:A fully autonomous controller is not needed when most transitions and side effects are governed by policy; place bounded exploratory reasoning inside one stage of a deterministic state machine; enforce approval and write permissions outside the agent; that control is assessed by disputed-case resolution with every refund still passing the identity and approval gates; the incident instead requires completed valid classifications per batch window and cost per accepted record.
Question 408
A production bug affects customer-specific prompts containing sensitive internal configuration data. Which design decision directly addresses this constraint?
- Classify model-routing faults separately from execution-stage failures
- Reproduce incidents using correlated logs and configuration versions
- Roll back to a known-good tool and prompt release
- Replay failures safely with sanitized inputs
Explanation
Correct Answer
D:Build redacted, access-controlled replay fixtures preserving relevant syntax and error conditions without unnecessary secrets or personal data; safe replay demonstrates that privacy and diagnostic value can coexist; evaluate the design with bug reproduction on sanitized fixtures with no sensitive token disclosure.
Incorrect Answers
A:Classify tool errors separately from model responses, validate return codes, and probe the failing integration before changing prompts; a more capable model cannot read data from a tool that never returns successfully; this alternative would measure correct root-cause attribution under injected search timeouts; it does not establish bug reproduction on sanitized fixtures with no sensitive token disclosure for the described use case.
B:Collect privacy-safe traces, request IDs, environment details, and exact configuration revisions to establish a reproducible failure case; a guessed prompt rewrite is unlikely to repair a fault that has not been isolated; this alternative would measure reproduction rate of sampled incidents from stored sanitized diagnostic metadata; it does not establish bug reproduction on sanitized fixtures with no sensitive token disclosure for the described use case.
C:Maintain versioned releases and documented rollback steps for hooks, prompts, model routes, and connector configuration; repeated untracked hotfixes can compound an outage and obscure the initial trigger; this alternative would measure time to restore normal operation using an approved previous release; it does not establish bug reproduction on sanitized fixtures with no sensitive token disclosure for the described use case.
Question 409
A software team gives a development assistant a local command interface limited to approved build and test scripts. Which result would prove that runs are reproducible and unrelated commands remain unavailable?
- Reproducible build results with denied commands outside the approved allowlist
- Event-to-index update delay and duplicate-event idempotency under replay
- Successful authenticated task exchange with each agent retaining its own permission checks
- Integration error rate and p95 latency on the fixed status-check path
Explanation
Correct Answer
A:Execution tests check both the developer outcome and the sandbox boundary; measure reproducible build results with denied commands outside the approved allowlist at the release gate; a bounded CLI execution environment can reuse standard developer tools with explicit directory and command permissions.
Incorrect Answers
B:Event-driven tests verify timely updates without wasteful polling loops; an authenticated event or webhook integration can trigger bounded processing only when upstream state changes; this tests event-to-index update delay and duplicate-event idempotency under replay; the actual release condition is reproducible build results with denied commands outside the approved allowlist.
C:Boundary tests verify interoperability without merging authorities; an agent-to-agent contract can exchange bounded tasks and outputs while preserving local autonomy, identity, and authority of each specialist system; this tests successful authenticated task exchange with each agent retaining its own permission checks; the actual release condition is reproducible build results with denied commands outside the approved allowlist.
D:A stable-contract benchmark confirms that extra orchestration is unnecessary; a direct typed API client is appropriate when the interface, arguments, and control flow are known and stable; this tests integration error rate and p95 latency on the fixed status-check path; the actual release condition is reproducible build results with denied commands outside the approved allowlist.
Question 410
A large repository needs independent code search, test analysis, and documentation review without letting every helper deploy changes. What is the most targeted architecture choice?
- Use specialized subagents with bounded tools and instructions
- Version team-wide CLAUDE.md guidance with the repository
- Use deterministic hooks to enforce required developer gates
- Keep credentials out of prompts and versioned configuration
- Scope MCP connections by project and user identity
- Use managed configuration to constrain dangerous permissions
Explanation
Correct Answer
A:Give each delegated role explicit objectives, context limits, and tool permissions with clear return artifacts; role-isolation tests prove that decomposition also restricts authority; evaluate the design with subagent task completion with zero out-of-scope tool access.
Incorrect Answers
B:Put reviewed repository-specific guidance in a shared CLAUDE.md file with appropriate scope and code review; a documentation file guides behavior but cannot enforce operating-system permissions or replace CI tests; this alternative would measure consistency of generated changes with repository build and test conventions across fresh sessions; it does not establish subagent task completion with zero out-of-scope tool access for the described use case.
C:Configure supported hooks for deterministic checks at appropriate lifecycle points, with clear failure behavior and maintenance ownership; a written instruction alone is best-effort and does not guarantee enforcement of a critical command gate; this alternative would measure hook execution and blocked noncompliant operations under simulated bypass attempts; it does not establish subagent task completion with zero out-of-scope tool access for the described use case.
D:Use approved secret storage, scoped environment access, and redaction while avoiding credentials in repositories, prompts, or logs; masking output after committing a secret cannot undo credential exposure or grant least privilege; this alternative would measure secret-scanning results and blocked exfiltration on seeded repository credentials; it does not establish subagent task completion with zero out-of-scope tool access for the described use case.
E:Choose appropriate configuration scope and per-user or service authorization for MCP servers, avoiding shared cross-project secrets; a working connection does not prove the assistant is restricted to the user's intended project resources; this alternative would measure access denial for cross-project MCP resources under user identity tests; it does not establish subagent task completion with zero out-of-scope tool access for the described use case.
F:Use organization-managed policy and supported permission configuration so enforced restrictions cannot be casually overridden by per-user convenience settings; personal preference files are not a trustworthy substitute for centrally governed high-risk permissions; this alternative would measure denial of prohibited commands after attempts to override local and project configuration; it does not establish subagent task completion with zero out-of-scope tool access for the described use case.