Question 420
Increasing the raw top-k count improves recall yet overwhelms Claude with irrelevant policy fragments. What design correction resolves the reported failure?
- Rerank a short candidate set for ambiguous retrieval queries
- Cache safe, version-keyed retrieval responses
- Parallelize independent read-only tool calls
- Use a safe limited answer when the deadline would otherwise be missed
Explanation
Correct Answer
A:Retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; a quality-latency frontier shows whether an extra ranking pass is worth its overhead; verify the repair using grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.
Incorrect Answers
B:An unversioned cache can improve speed while silently sacrificing freshness and access correctness; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; that control is assessed by repeated-query latency with zero stale or cross-scope cache responses after updates; the incident instead requires grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.
C:Parallel execution offers no benefit when one call requires another's output or modifies shared state; execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; that control is assessed by end-to-end p95 improvement with all required sources still present; the incident instead requires grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.
D:Silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; that control is assessed by deadline compliance and correct indication of omitted enrichment; the incident instead requires grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths.
Question 421
A research coordinator repeatedly reassigns unresolved tasks to itself and subordinate agents. What is the most targeted architecture choice?
- Delegate only specialized subtasks that add measurable value
- Use a coordinator to route distinct specialist tasks
- Bound handoffs, retries, and recursion across agents
- Separate independent critique from generation
Explanation
Correct Answer
C:Enforce maximum depth, handoff count, timeouts, and explicit unresolved-case escalation in the orchestrator; stressing unresolved cases proves that the system can fail predictably and within budget; evaluate the design with zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record.
Incorrect Answers
A:Introduce a specialist only when its independent expertise, tool scope, or parallelism beats a simpler call on representative evaluations; more agents by themselves do not provide additional knowledge, authority, or better outcomes; this alternative would measure marginal quality improvement per added specialist against its incremental cost and latency; it does not establish zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record for the described use case.
B:A coordinator can dispatch scoped subagents and merge their typed outputs while restricting each specialist's context and tools; specialist routing adds handoffs and can be wasteful for one small, homogeneous transformation; this alternative would measure completed reports with correctly scoped evidence and no cross-specialty data leakage; it does not establish zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record for the described use case.
D:A distinct verifier with a rubric and evidence access can challenge unsupported assertions before an external document is released; a critic agent does not repair missing source documents or guarantee that all facts are verifiable; this alternative would measure verified claim support rate after independent critique on seeded unsupported drafts; it does not establish zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record for the described use case.
Question 422
A procurement agent has a prompt instruction not to approve high-value orders but its API credential can approve any amount. What is the most targeted architecture choice?
- Confirm tenant ownership of resource identifiers
- Record the user, tenant, and action in audit trails
- Use delegated user identity for per-user data access
- Enforce authorization inside the tool backend
Explanation
Correct Answer
D:Implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; direct API tests distinguish true backend policy from model cooperation; evaluate the design with rejection of unauthorized tool calls even when sent outside the model workflow.
Incorrect Answers
A:Check the requested resource against the authenticated tenant and role at the data service rather than trusting user-provided identifiers; syntactically valid IDs and a logged-in session do not establish ownership of a resource; this alternative would measure denial of cross-tenant identifier enumeration attempts in direct tool tests; it does not establish rejection of unauthorized tool calls even when sent outside the model workflow for the described use case.
B:Propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records; logging only a model response cannot attribute which authenticated principal caused a backend change; this alternative would measure audit reconstruction of actor, authority, target, and result for sampled mutations; it does not establish rejection of unauthorized tool calls even when sent outside the model workflow for the described use case.
C:Carry an appropriate user-bound authorization context to downstream services and enforce record-level access per request; application-level authentication alone does not prove permission to read every record returned by a shared account; this alternative would measure zero cross-user CRM record disclosures in permission-matrix tests; it does not establish rejection of unauthorized tool calls even when sent outside the model workflow for the described use case.
Question 423
A reporting copilot needs broad analysis of account trends but must not alter account balances. Which approach best meets the stated requirement?
- Audit tool manifests as versioned production configuration
- Remove duplicate and deprecated tools from the catalog
- Write unambiguous tool descriptions and input schemas
- Separate read-only retrieval from mutating tools
Explanation
Correct Answer
D:Expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; permission tests reveal whether supposedly analytical agents can cause side effects; evaluate the design with zero mutation permissions for report-only identities across tool contract checks.
Incorrect Answers
A:Maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; ad hoc tool registration lacks the inventory needed to catch capability expansion before deployment; this alternative would measure changes rejected when requested tool scopes exceed the approved task manifest; it does not establish zero mutation permissions for report-only identities across tool contract checks for the described use case.
B:Consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; more tool variants do not add genuine capability when they expose the same action with conflicting contracts; this alternative would measure single correct connector selection and consistent ticket states across repeated requests; it does not establish zero mutation permissions for report-only identities across tool contract checks for the described use case.
C:Define task-specific descriptions, typed arguments, required fields, and clear side-effect semantics so tool selection is meaningful; a richer schema improves selection but does not override a backend that ignores caller authorization; this alternative would measure correct tool-selection rate for ambiguous intents and invalid-argument rejection; it does not establish zero mutation permissions for report-only identities across tool contract checks for the described use case.
Question 424
A retail question passes through search, inference, and stock-check services. Engineers inject artificial delay at each dependency in turn. Which result proves that tracing can correctly identify the cause?
- Detection time for an injected single-tenant degradation
- Capture rate of seeded critical failures at a bounded telemetry cost
- Time to detect and stop a simulated unauthorized bulk-write burst
- Trace completeness and correct bottleneck attribution on instrumented fault injections
- Error diagnosability with seeded sensitive fields absent from collected logs
Explanation
Correct Answer
D:Distributed spans expose ownership boundaries across the whole execution path; measure trace completeness and correct bottleneck attribution on instrumented fault injections at the release gate; end-to-end traces with consistent request and span identifiers identify which service and step consumed time.
Incorrect Answers
A:Segmented monitoring finds localized issues that global summary statistics miss; break down latency, errors, and tool success by permission-safe tenant and workload dimensions with cardinality controls; this tests detection time for an injected single-tenant degradation; the actual release condition is trace completeness and correct bottleneck attribution on instrumented fault injections.
B:Risk-aware sampling preserves evidence for rare faults without uncontrolled storage growth; retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; this tests capture rate of seeded critical failures at a bounded telemetry cost; the actual release condition is trace completeness and correct bottleneck attribution on instrumented fault injections.
C:Action telemetry and response drills measure whether harmful changes can be contained promptly; monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; this tests time to detect and stop a simulated unauthorized bulk-write burst; the actual release condition is trace completeness and correct bottleneck attribution on instrumented fault injections.
E:Redaction and observability must be validated together, not traded off blindly; record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; this tests error diagnosability with seeded sensitive fields absent from collected logs; the actual release condition is trace completeness and correct bottleneck attribution on instrumented fault injections.
Question 425
The answers are factually sound but key instructions cannot be navigated by screen-reader users. Which action best removes the underlying failure mode?
- Test accessibility of generated explanations and interfaces
- Publish scope, limitations, and residual risks to stakeholders
- Design an override path proportionate to consequences
- Provide a route to challenge significant recommendations
Explanation
Correct Answer
A:Include accessibility requirements, assistive-technology testing, and inclusive content rubrics in evaluation and design; direct accessibility testing measures practical use rather than formal content correctness only; verify the repair using task completion by users of tested assistive technologies on representative flows.
Incorrect Answers
B:A generic disclaimer cannot compensate for hidden high-risk use or unavailable oversight; maintain clear documentation of intended use, unsupported scenarios, measured performance, and escalation requirements; that control is assessed by accuracy of user understanding of supported versus unsupported tasks; the incident instead requires task completion by users of tested assistive technologies on representative flows.
C:Human review after an irreversible action may be too late to prevent harm; preserve an authorized override and recovery route scaled to decision consequences, with feedback recorded for evaluation; that control is assessed by successful intervention before seeded high-impact incorrect actions are committed; the incident instead requires task completion by users of tested assistive technologies on representative flows.
D:A fairness dashboard alone cannot remedy individual incorrect records or decisions; provide a contest and correction process with accountable human ownership for significant impacts; that control is assessed by time to resolve substantiated challenges and proportion of cases with documented review; the incident instead requires task completion by users of tested assistive technologies on representative flows.
Question 426
The interface blocks a time-critical warning while waiting for nonessential narrative enrichment. Which corrective change best addresses the root cause?
- Move slow operations into asynchronous jobs with progress states
- Use a safe limited answer when the deadline would otherwise be missed
- Direct expensive reasoning to high-uncertainty cases
- Cache safe, version-keyed retrieval responses
- Rerank a short candidate set for ambiguous retrieval queries
Explanation
Correct Answer
B:Keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; timed fault injection confirms graceful degradation without inventing evidence; verify the repair using deadline compliance and correct indication of omitted enrichment.
Incorrect Answers
A:A longer synchronous timeout would still tie up request resources and expose users to unreliable waits; acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; that control is assessed by job completion reliability and duplicate-submission rate during slow-system simulations; the incident instead requires deadline compliance and correct indication of omitted enrichment.
C:Uniform deep reasoning does not add value to deterministic records and may increase response time; use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; that control is assessed by accuracy by difficulty segment and unit cost under p95 constraints; the incident instead requires deadline compliance and correct indication of omitted enrichment.
D:An unversioned cache can improve speed while silently sacrificing freshness and access correctness; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; that control is assessed by repeated-query latency with zero stale or cross-scope cache responses after updates; the incident instead requires deadline compliance and correct indication of omitted enrichment.
E:Taking every retrieved hit into context may raise token cost and distraction without raising answer quality; retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; that control is assessed by grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths; the incident instead requires deadline compliance and correct indication of omitted enrichment.
Question 427
Cloud spend per request declines while the fully reviewed cost per usable document rises. What design correction resolves the reported failure?
- Translate response-time expectations into a percentile SLO
- Design a degraded mode that protects the value proposition
- Choose model and architecture tiers by case impact
- Track cost per accepted business outcome
Explanation
Correct Answer
D:Combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; unit cost tied to accepted results distinguishes economical quality from cheap failed attempts; verify the repair using total operational cost divided by approved documents meeting the quality rubric.
Incorrect Answers
A:A percentile SLO does not alone reveal whether the answers are factually correct; define a p95 or p99 target against real end-to-end requests, including retrieval and tools, rather than reporting an average model time; that control is assessed by end-to-end p95 latency and error rate against the agreed service objective; the incident instead requires total operational cost divided by approved documents meeting the quality rubric.
B:High answer accuracy in normal mode says little about process continuity during dependency failures; separate critical deterministic operations from AI enhancement and provide explicit fallback and manual paths when model service is unavailable; that control is assessed by successful completion of critical workflows during injected model outages; the incident instead requires total operational cost divided by approved documents meeting the quality rubric.
C:A uniformly premium model can raise cost without proportionate value on simple, low-risk tasks; route by validated task complexity and consequence, using stronger reasoning or human review where evidence justifies the incremental spend; that control is assessed by quality and total cost segmented by request risk tier under a fixed accuracy floor; the incident instead requires total operational cost divided by approved documents meeting the quality rubric.
Question 428
An order assistant must return a constrained status enum, case identifier, and nullable reason field to an internal API. Which choice most directly satisfies the documented goal?
- Define an explicit abstention path for insufficient evidence
- Delimit retrieved content as untrusted evidence
- Keep invariant policy instructions distinct from per-request input
- Use a machine-readable output contract and validator
Explanation
Correct Answer
D:Request a documented structured format and validate schema, allowed values, and business semantics before downstream use; parser-level rejection tests verify the actual interface boundary; evaluate the design with valid schema rate and zero unauthorized enum values reaching the consumer.
Incorrect Answers
A:Tell the assistant when to identify missing evidence, ask for clarification, or escalate, and reinforce this with evidence validation; a fallback message cannot repair a poor retrieval index; it only prevents unsupported certainty; this alternative would measure unsupported-answer rate and appropriate abstention precision on no-answer cases; it does not establish valid schema rate and zero unauthorized enum values reaching the consumer for the described use case.
B:Clearly separate lower-trust source material from application instructions and enforce permissions at the tool layer as well; delimiters aid instruction hierarchy but do not sanitize malicious destinations or replace output security controls; this alternative would measure prompt-injection resistance on hostile retrieved snippets without loss of legitimate source use; it does not establish valid schema rate and zero unauthorized enum values reaching the consumer for the described use case.
C:Separate stable system-level policy from user content and tool results, retaining consistent privilege boundaries across requests; a modular instruction layout does not itself prove that external API permissions are enforced; this alternative would measure rate of user messages that fail to override defined task boundaries in regression tests; it does not establish valid schema rate and zero unauthorized enum values reaching the consumer for the described use case.
Question 429
Engineers only track final answer helpfulness and cannot distinguish missing evidence from poor synthesis. Which corrective change best addresses the root cause?
- Chunk documents along meaningful structural boundaries
- Evaluate retrieval quality independently of generation
- Filter retrieval by access control before model exposure
- Version embeddings and reindex when representation changes
Explanation
Correct Answer
B:Measure retrieval recall, ranking relevance, and evidence coverage on labeled query-document pairs before optimizing generation; separating retrieval metrics identifies which pipeline stage needs repair; verify the repair using recall at k and evidence-coverage rate for known-answer queries.
Incorrect Answers
A:Arbitrary equal-length chunks may split dependencies despite fitting an embedding model's input limit; chunk by headings and semantic sections with deliberate overlap where necessary while maintaining parent-document references; that control is assessed by retrieval of both governing clause and exception for labeled contract questions; the incident instead requires recall at k and evidence-coverage rate for known-answer queries.
C:A model instruction to ignore confidential items is not a data-access boundary once the text is supplied; apply document permissions to retrieval candidates before material enters model context or logs; that control is assessed by unauthorized chunk exposure rate under staff permission matrices, targeting zero; the incident instead requires recall at k and evidence-coverage rate for known-answer queries.
D:Updating only the query encoder can corrupt retrieval ranking even if the documents are unchanged; record embedding model and preprocessing versions, re-embed consistently, and use controlled index migration; that control is assessed by retrieval recall before and after a complete embedding-index migration; the incident instead requires recall at k and evidence-coverage rate for known-answer queries.
Question 430
A legal-review workflow is tested with agreements whose risk hinges on exceptions spread over several exhibits. Before changing its reasoning-model route, which evidence would demonstrate that the selected capacity handles those dependencies?
- Correct equipment identification and recommendation accuracy on labeled photographs
- Per-segment acceptance rate across language, length, and noisy-input strata
- No contract regressions and acceptable quality/latency deltas on the proposed release
- Correct resolution of cross-exhibit conflicts at the required latency and review cost
Explanation
Correct Answer
D:A targeted complex-case benchmark tests the capability that routine classification metrics miss; measure correct resolution of cross-exhibit conflicts at the required latency and review cost at the release gate; escalate tasks with demonstrated reasoning difficulty to a capable model after measuring gains on cases involving cross-document dependencies.
Incorrect Answers
A:A multimodal acceptance set measures the capability absent from text-only trials; confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; this tests correct equipment identification and recommendation accuracy on labeled photographs; the actual release condition is correct resolution of cross-exhibit conflicts at the required latency and review cost.
B:Stratified scores expose failures that an overall aggregate conceals; representative workload evaluation must include input quality, document lengths, languages, and failure consequences before setting model policy; this tests per-segment acceptance rate across language, length, and noisy-input strata; the actual release condition is correct resolution of cross-exhibit conflicts at the required latency and review cost.
C:Version-aware tests make improvement claims meaningful for the deployed application; treat model replacements as versioned releases with task-specific eval gates, contract tests, canaries, and rollback capability; this tests no contract regressions and acceptable quality/latency deltas on the proposed release; the actual release condition is correct resolution of cross-exhibit conflicts at the required latency and review cost.