Question 600
A low-traffic legal system repeats a long analysis prefix only once every forty minutes. Which design decision directly addresses this constraint?
- Instrument cache effectiveness with token usage counters
- Modularize shared task instructions for consistent reuse
- Compare cache lifetime with actual reuse patterns
- Cache a stable, high-volume prompt prefix
Explanation
Correct Answer
C:Assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; reuse-adjusted economics tests whether the caching decision pays off under actual traffic; evaluate the design with net cost per accepted request across realistic interarrival times.
Incorrect Answers
A:Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; a faster model might improve latency but would not explain a missing cache hit on identical prompts; this alternative would measure sustained cache-read token share under repeated workload replay; it does not establish net cost per accepted request across realistic interarrival times for the described use case.
B:Maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; this alternative would measure cross-application prompt-version alignment and regression pass rate; it does not establish net cost per accepted request across realistic interarrival times for the described use case.
D:Put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; caching helps repeated identical prefixes; it does not improve a model's answer accuracy on missing knowledge; this alternative would measure cache-read input tokens and end-to-end latency over repeated requests; it does not establish net cost per accepted request across realistic interarrival times for the described use case.
Question 601
The chat request times out while backend processing eventually finishes, causing duplicate user submissions. Which change should the architect recommend first?
- Parallelize independent read-only tool calls
- Move slow operations into asynchronous jobs with progress states
- Use a safe limited answer when the deadline would otherwise be missed
- Cache safe, version-keyed retrieval responses
Explanation
Correct Answer
B:Acknowledge an asynchronous job, persist its state, and let the client poll or receive authorized completion notifications; durable job identity decouples user interaction time from background work duration; verify the repair using job completion reliability and duplicate-submission rate during slow-system simulations.
Incorrect Answers
A:Parallel execution offers no benefit when one call requires another's output or modifies shared state; execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; that control is assessed by end-to-end p95 improvement with all required sources still present; the incident instead requires job completion reliability and duplicate-submission rate during slow-system simulations.
C:Silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; that control is assessed by deadline compliance and correct indication of omitted enrichment; the incident instead requires job completion reliability and duplicate-submission rate during slow-system simulations.
D:An unversioned cache can improve speed while silently sacrificing freshness and access correctness; use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; that control is assessed by repeated-query latency with zero stale or cross-scope cache responses after updates; the incident instead requires job completion reliability and duplicate-submission rate during slow-system simulations.
Question 602
A maintenance support portal uses a high-cost deep analysis path even when the required answer is a cached part-number lookup. Which intervention is most appropriate for this use case?
- Parallelize independent read-only tool calls
- Rerank a short candidate set for ambiguous retrieval queries
- Cache safe, version-keyed retrieval responses
- Route expensive reasoning to high-uncertainty cases
- Provide a bounded partial answer when full enrichment would miss the deadline
Explanation
Correct Answer
D:Use an evaluated routing policy that sends straightforward lookups to a light path and escalates uncertain cases to more capable reasoning; joint segmented reporting reveals whether expensive processing is spent where it helps; evaluate the design with accuracy by difficulty segment and unit cost under p95 constraints.
Incorrect Answers
A:Execute independent lookups concurrently with bounded timeouts and aggregate the results after all necessary dependencies complete; parallel execution offers no benefit when one call requires another's output or modifies shared state; this alternative would measure end-to-end p95 improvement with all required sources still present; it does not establish accuracy by difficulty segment and unit cost under p95 constraints for the described use case.
B:Retrieve a bounded candidate set and rerank against the actual question to concentrate useful evidence without feeding all candidates to the model; taking every retrieved hit into context may raise token cost and distraction without raising answer quality; this alternative would measure grounded-answer accuracy and retrieval-stage p95 latency at several rerank depths; it does not establish accuracy by difficulty segment and unit cost under p95 constraints for the described use case.
C:Use a cache keyed by query, permissions where relevant, and content version, with invalidation tied to source updates; an unversioned cache can improve speed while silently sacrificing freshness and access correctness; this alternative would measure repeated-query latency with zero stale or cross-scope cache responses after updates; it does not establish accuracy by difficulty segment and unit cost under p95 constraints for the described use case.
E:Keep the critical verified signal independent and return a clearly limited result when optional enrichment exceeds its budget; silently presenting an unverified enriched conclusion would violate accuracy requirements despite meeting a timer; this alternative would measure deadline compliance and correct indication of omitted enrichment; it does not establish accuracy by difficulty segment and unit cost under p95 constraints for the described use case.
Question 603
An appointment-reminder service composes notification requests with addresses and customer data fields that must not be expanded by generated text. A customer-support agent also reads untrusted input and uses account tools on personal records. Which TWO control designs fit the outbound validation and layered safety requirements? Choose TWO.
- Enforce least privilege where tool actions execute
- Allowlist generated fields and destinations before external calls
- Apply controls at input, tool execution, and output boundaries
- Screen unsafe outputs in proportion to the use case
- Treat retrieved instructions as untrusted data
Explanation
Correct Answers
B:Validate destinations, argument types, sensitive fields, and authorization before dispatch; reject values outside the approved contract; contract-level checks demonstrate control over the actual data leaving the system; its corresponding workstream should be validated by zero unapproved recipients or fields accepted by the outbound service.
C:Layer input handling, model instructions, backend authorization, schema checks, and output review so one failed defense does not grant action authority; end-to-end adversarial cases assess the combined defenses and their failure isolation; its corresponding workstream should be validated by denial of unauthorized actions and harmful outputs across multi-stage red-team paths.
Incorrect Answers
A:Backend least privilege is a necessary baseline for protected account actions, yet it does not by itself validate every notification destination and data field or provide the full input, execution, and output safety stack required for the separate personal-data workflow.
D:Post-generation screening can catch harmful final messages, but a notification service can leak private data through tool arguments before any final chat answer exists; this case therefore needs an outbound field-and-destination allowlist plus controls across all stages.
E:Keep source contents in a lower-trust evidence role and prevent embedded commands from crossing into tool authority; assuming retrieved text is authoritative because it came from a connector opens a prompt-injection path; its natural acceptance measure is attack success rate on adversarial web pages and zero unauthorized egress calls, which does not satisfy the separate workstreams being assessed.
Question 604
Offline benchmark accuracy stays constant while current customer corrections steadily increase. Which action best removes the underlying failure mode?
- Assess production metrics by model and prompt version
- Alert on unexpected token and cost anomalies
- Instrument end-to-end execution spans for causal debugging
- Detect distribution and quality drift in ongoing samples
Explanation
Correct Answer
D:Periodically sample representative production outputs for labeling, segment analysis, and drift signals without retaining unnecessary personal data; live sampling reveals degradation that unchanged historical tests cannot see; verify the repair using quality delta between recent audited production samples and the launch baseline.
Incorrect Answers
A:A global metric is inadequate when several behaviors are concurrently deployed; partition traces and scored outcomes by deployed configuration, using release identifiers to support rollback and diagnosis; that control is assessed by detection of version-specific quality regressions in a staged rollout; the incident instead requires quality delta between recent audited production samples and the launch baseline.
B:Request-volume alerts alone miss a sudden rise in tokens per individual workflow; monitor input/output tokens, cache reads, per-stage costs, and cost-per-task anomalies with accountable ownership; that control is assessed by detection lag for injected per-request token inflation; the incident instead requires quality delta between recent audited production samples and the launch baseline.
C:Model usage totals do not reveal which integration failed within a particular request; correlation and span IDs across components allow latency and failure attribution along the actual path; that control is assessed by percent of failures with a complete trace to the responsible stage; the incident instead requires quality delta between recent audited production samples and the launch baseline.
Question 605
Following an automated database update, an auditor needs to identify the human initiator, service authority, affected row, and final result. Which trace-quality measure best supports accountability?
- Audit reconstruction of actor, authority, target, and result for sampled mutations
- Zero cross-user CRM record disclosures in permission-matrix tests
- Credential-expiry and scope-denial tests after simulated token compromise
- Rejection of unauthorized tool calls even when sent outside the model workflow
Explanation
Correct Answer
A:End-to-end attribution makes unauthorized use investigable rather than anonymous; measure audit reconstruction of actor, authority, target, and result for sampled mutations at the release gate; propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records.
Incorrect Answers
B:User-context tests verify enforcement of the source system's individual access rules; carry an appropriate user-bound authorization context to downstream services and enforce record-level access per request; this tests zero cross-user CRM record disclosures in permission-matrix tests; the actual release condition is audit reconstruction of actor, authority, target, and result for sampled mutations.
C:Short validity and minimum scope bound the impact of a leaked credential; prefer tightly scoped, expiring credentials with secure rotation and audited issuance for the precise operation; this tests credential-expiry and scope-denial tests after simulated token compromise; the actual release condition is audit reconstruction of actor, authority, target, and result for sampled mutations.
D:Direct API tests distinguish true backend policy from model cooperation; implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; this tests rejection of unauthorized tool calls even when sent outside the model workflow; the actual release condition is audit reconstruction of actor, authority, target, and result for sampled mutations.
Question 606
Several coding teams repeat the same release checklist and repository diagnostic procedure across projects. Which approach best meets the stated requirement?
- Adopt test-first acceptance cases for generated changes
- Package reusable engineering instructions as governed Skills
- Run reproducible CI gates on assisted code
- Break large refactors into independently reviewable changes
Explanation
Correct Answer
B:Maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; reuse tests verify that essential steps are consistently available; evaluate the design with checklist completeness across independent sessions using the reviewed Skill version.
Incorrect Answers
A:Define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; merely reading generated code for plausibility does not replace tests of business invariants; this alternative would measure passing unit and property tests including specified rounding boundaries; it does not establish checklist completeness across independent sessions using the reviewed Skill version for the described use case.
C:Execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; local success can be environment-dependent and should not be the only release assurance; this alternative would measure cI pass reproducibility on clean builds with locked dependencies; it does not establish checklist completeness across independent sessions using the reviewed Skill version for the described use case.
D:Use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; one massive patch can reduce initial coordination but increases regression and rollback risk; this alternative would measure review time and defect localization per incremental change set; it does not establish checklist completeness across independent sessions using the reviewed Skill version for the described use case.
Question 607
A contact-center pilot coincides with staffing changes and a revised escalation policy. The sponsor wants evidence that the assistant itself caused improved case resolution. Which measurement is appropriate?
- End-to-end p95 latency and error rate against the agreed service objective
- Total operational cost divided by approved documents meeting the quality rubric
- Quality and total cost segmented by request risk tier under a fixed accuracy floor
- Incremental resolution lift compared with a comparable non-assisted cohort
Explanation
Correct Answer
D:A defensible counterfactual is required to credit the architecture with the observed benefit; measure incremental resolution lift compared with a comparable non-assisted cohort at the release gate; use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders.
Incorrect Answers
A:Tail measurement detects the bad experiences that averages can obscure; define a p95 or p99 target against real end-to-end requests, including retrieval and tools, rather than reporting an average model time; this tests end-to-end p95 latency and error rate against the agreed service objective; the actual release condition is incremental resolution lift compared with a comparable non-assisted cohort.
B:Unit cost tied to accepted results distinguishes economical quality from cheap failed attempts; combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; this tests total operational cost divided by approved documents meeting the quality rubric; the actual release condition is incremental resolution lift compared with a comparable non-assisted cohort.
C:Segmented evaluation reveals whether additional reasoning spend is targeted at cases that benefit; route by validated task complexity and consequence, using stronger reasoning or human review where evidence justifies the incremental spend; this tests quality and total cost segmented by request risk tier under a fixed accuracy floor; the actual release condition is incremental resolution lift compared with a comparable non-assisted cohort.
Question 608
Loading every procedure at startup bloats context and distracts the agent with unrelated instructions. Which change should the architect recommend first?
- Load specialized Skills only when the task needs them
- Instrument cache effectiveness with token usage counters
- Modularize shared task instructions for consistent reuse
- Compare cache lifetime with actual reuse patterns
Explanation
Correct Answer
A:Keep Skills discoverable and load task-relevant instructions or resources on demand, maintaining scope and permission constraints; on-demand loading must retain quality while avoiding unnecessary context overhead; verify the repair using task success with reduced irrelevant instruction tokens across mixed workflows.
Incorrect Answers
B:A faster model might improve latency but would not explain a missing cache hit on identical prompts; compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; that control is assessed by sustained cache-read token share under repeated workload replay; the incident instead requires task success with reduced irrelevant instruction tokens across mixed workflows.
C:Prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; that control is assessed by cross-application prompt-version alignment and regression pass rate; the incident instead requires task success with reduced irrelevant instruction tokens across mixed workflows.
D:A long cache lifetime is not automatically economical when there is little repeated work; assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; that control is assessed by net cost per accepted request across realistic interarrival times; the incident instead requires task success with reduced irrelevant instruction tokens across mixed workflows.
Question 609
Human reviewers sign off agent recommendations in a regulated process. The test suite plants plausible but incorrect conclusions with contradictory supporting records. Which metric would show that review is a meaningful control rather than a formality?
- Reviewer error interception on seeded incorrect recommendations
- Zero external-result fields promoted to internal approvals in test cases
- Blocked unauthorized destinations and absence of confidential fields in egress logs
- Maximum resource usage and orderly termination on deliberately unsolvable tasks
Explanation
Correct Answer
A:The review test measures meaningful human oversight, not click-through compliance; measure reviewer error interception on seeded incorrect recommendations at the release gate; design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button.
Incorrect Answers
B:Provenance-sensitive checks stop evidence laundering through connector outputs; preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; this tests zero external-result fields promoted to internal approvals in test cases; the actual release condition is reviewer error interception on seeded incorrect recommendations.
C:Network- and tool-level evidence catches leakage not visible in user-facing text; restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; this tests blocked unauthorized destinations and absence of confidential fields in egress logs; the actual release condition is reviewer error interception on seeded incorrect recommendations.
D:Budget stress tests show whether runaway loops fail safely; set per-task step, token, cost, and elapsed-time budgets with a clear abort and escalation state; this tests maximum resource usage and orderly termination on deliberately unsolvable tasks; the actual release condition is reviewer error interception on seeded incorrect recommendations.
Question 610
A helpdesk service sends an identical policy manual with every request, followed by unique user questions. Which choice most directly satisfies the documented goal?
- Cache a stable, high-volume prompt prefix
- Assess cache lifetime with actual reuse patterns
- Instrument cache effectiveness with token usage counters
- Keep dynamic fields out of cached prefix content
Explanation
Correct Answer
A:Put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; cache-hit telemetry verifies the optimization rather than relying on estimated token savings; evaluate the design with cache-read input tokens and end-to-end latency over repeated requests.
Incorrect Answers
B:Assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; a long cache lifetime is not automatically economical when there is little repeated work; this alternative would measure net cost per accepted request across realistic interarrival times; it does not establish cache-read input tokens and end-to-end latency over repeated requests for the described use case.
C:Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; a faster model might improve latency but would not explain a missing cache hit on identical prompts; this alternative would measure sustained cache-read token share under repeated workload replay; it does not establish cache-read input tokens and end-to-end latency over repeated requests for the described use case.
D:Move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; a cache breakpoint cannot compensate for changes to earlier content in the hashed prefix; this alternative would measure prefix reuse ratio when request metadata changes across calls; it does not establish cache-read input tokens and end-to-end latency over repeated requests for the described use case.