Question 540
People reference obsolete decisions from a meeting note and approve incompatible changes. How should the design be corrected?
- Compare options on a shared multi-dimensional scorecard
- Demonstrate known limitations alongside successful workflows
- Present executive decisions through outcomes, cost, and risk
- Keep a versioned decision and change history
Explanation
Correct Answer
D:Maintain a version-controlled decision log linked to implementation and review milestones, including superseded guidance; version-aware retrieval prevents implementation against outdated agreements; verify the repair using correct retrieval of active decisions and their supersession chain during review.
Incorrect Answers
A:Isolated benchmark rankings cannot settle a decision with several business constraints; present the same quality, latency, cost, risk, and maintainability measures for each candidate under agreed weighting; that control is assessed by stakeholder-approved selection using common scored evidence and disclosed trade-offs; the incident instead requires correct retrieval of active decisions and their supersession chain during review.
B:A demo focused solely on best cases can mask residual risks and create unrealistic expectations; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; that control is assessed by stakeholder identification of boundaries and fallback responsibilities after the demo; the incident instead requires correct retrieval of active decisions and their supersession chain during review.
C:A detailed API design is valuable to implementers but not sufficient for a funding decision; summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; that control is assessed by sponsor comprehension of expected outcomes, decision options, and risk ownership; the incident instead requires correct retrieval of active decisions and their supersession chain during review.
Question 541
Uniformly dropping logs removes rare failures that investigators need to reproduce. Which intervention targets the defect rather than its symptoms?
- Correlate model, retrieval, and tool spans in distributed traces
- Sample complete traces strategically under heavy traffic
- Log structured tool outcomes without exposing sensitive payloads
- Separate retrieval, provider, and tool latency budgets
Explanation
Correct Answer
B:Retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; risk-aware sampling preserves evidence for rare faults without uncontrolled storage growth; verify the repair using capture rate of seeded critical failures at a bounded telemetry cost.
Incorrect Answers
A:A model-only latency chart cannot explain retrieval or downstream network bottlenecks; end-to-end traces with consistent request and span identifiers identify which service and step consumed time; that control is assessed by trace completeness and correct bottleneck attribution on instrumented fault injections; the incident instead requires capture rate of seeded critical failures at a bounded telemetry cost.
C:Full payload capture may help debugging but expands sensitive-data exposure and retention obligations; record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; that control is assessed by error diagnosability with seeded sensitive fields absent from collected logs; the incident instead requires capture rate of seeded critical failures at a bounded telemetry cost.
D:Optimizing generation tokens cannot repair a slow connector that dominates the critical path; instrument per-stage timing and compare each segment against its allocated latency budget before changing model configuration; that control is assessed by stage-specific p95 deltas identifying injected bottlenecks; the incident instead requires capture rate of seeded critical failures at a bounded telemetry cost.
Question 542
The assistant fabricates a precise interest-rate term despite retrieval returning no applicable document. What design correction resolves the reported failure?
- Define an explicit abstention path for insufficient evidence
- Delimit retrieved content as untrusted evidence
- Write explicit roles, scope, and prohibited operations
- Keep invariant policy instructions distinct from per-request input
Explanation
Correct Answer
A:Tell the assistant when to identify missing evidence, ask for clarification, or escalate, and reinforce this with evidence validation; an abstention test verifies whether uncertainty is handled safely without refusing answerable questions; verify the repair using unsupported-answer rate and appropriate abstention precision on no-answer cases.
Incorrect Answers
B:Delimiters aid instruction hierarchy but do not sanitize malicious destinations or replace output security controls; clearly separate lower-trust source material from application instructions and enforce permissions at the tool layer as well; that control is assessed by prompt-injection resistance on hostile retrieved snippets without loss of legitimate source use; the incident instead requires unsupported-answer rate and appropriate abstention precision on no-answer cases.
C:Role instructions improve behavior but cannot grant or restrict database permissions on their own; define the assistant's permitted tasks, out-of-scope handling, and escalation behavior in stable application-level instructions, backed by independent access enforcement; that control is assessed by out-of-scope refusal and handoff accuracy on a policy-boundary test set; the incident instead requires unsupported-answer rate and appropriate abstention precision on no-answer cases.
D:A modular instruction layout does not itself prove that external API permissions are enforced; separate stable system-level policy from user content and tool results, retaining consistent privilege boundaries across requests; that control is assessed by rate of user messages that fail to override defined task boundaries in regression tests; the incident instead requires unsupported-answer rate and appropriate abstention precision on no-answer cases.
Question 543
The team cites a parent vendor's security page without confirming that the chosen deployment is within an authorized boundary. What design correction resolves the reported failure?
- Minimize personal data and specify retention purpose
- Implement accountable access, deletion, and retention processes
- Assess privacy risks and lawful processing grounds early
- Validate FedRAMP authorization against exact service boundaries
Explanation
Correct Answer
D:Verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection; boundary-specific checks prevent assuming certification transfers between unrelated services; verify the repair using documented authorization-boundary mapping for every service handling regulated data.
Incorrect Answers
A:Broad encryption does not justify collecting more personal data than the task needs; limit processing to necessary authorized fields, define a lawful purpose and retention controls, and document relevant privacy obligations; that control is assessed by sensitive-field inclusion rate in prompts and retention-policy compliance; the incident instead requires documented authorization-boundary mapping for every service handling regulated data.
B:Deleting the original row does not automatically erase embeddings, caches, or observability copies; map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion; that control is assessed by deletion-request completion across all inventoried repositories within policy deadlines; the incident instead requires documented authorization-boundary mapping for every service handling regulated data.
C:Technical model accuracy does not establish permission to process sensitive personal information; involve privacy counsel to assess applicable lawful bases, transparency, rights, and whether a data-protection impact assessment is required before release; that control is assessed by completion of required privacy assessment and recorded resolution of high risks before go-live; the incident instead requires documented authorization-boundary mapping for every service handling regulated data.
Question 544
Engineers use an AI coding assistant that generates a different release checklist every session and sometimes omits security review. How should the required workflow be distributed consistently?
- Generate documentation from verified source and review it
- Break large refactors into independently reviewable changes
- Package reusable engineering instructions as governed Skills
- Run reproducible CI gates on assisted code
Explanation
Correct Answer
C:Maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; reuse tests verify that essential steps are consistently available; verify the repair using checklist completeness across independent sessions using the reviewed Skill version.
Incorrect Answers
A:Fluent prose generated from old context is not a reliable record of a current API contract; ground documentation generation in current code, schemas, and tests, then validate examples and have owners review changes; that control is assessed by documented endpoint examples executing successfully against the reviewed build; the incident instead requires checklist completeness across independent sessions using the reviewed Skill version.
B:One massive patch can reduce initial coordination but increases regression and rollback risk; use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; that control is assessed by review time and defect localization per incremental change set; the incident instead requires checklist completeness across independent sessions using the reviewed Skill version.
D:Local success can be environment-dependent and should not be the only release assurance; execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; that control is assessed by cI pass reproducibility on clean builds with locked dependencies; the incident instead requires checklist completeness across independent sessions using the reviewed Skill version.
Question 545
A patient-support evaluation team introduces a data-minimized reviewer interface. Which outcome would demonstrate that reviewers can judge answer quality without access to unrelated clinical details?
- Critical-action dual-approval compliance and zero bypasses during staged tests
- Recurrence rate of categorized, previously corrected error types
- Correct escalation of ambiguous legal cases with reviewer attribution
- Unauthorized field exposure and successful review accuracy after data minimization
Explanation
Correct Answer
D:Review workflows should preserve decision quality while protecting privacy; measure unauthorized field exposure and successful review accuracy after data minimization at the release gate; minimize, redact, and scope information in review interfaces according to purpose and authorized reviewer role.
Incorrect Answers
A:Approval testing must target the cases where one mistake affects many users; apply risk-tiered approvals, including dual control or independent sign-off where policy demands, to exceptionally consequential mutations; this tests critical-action dual-approval compliance and zero bypasses during staged tests; the actual release condition is unauthorized field exposure and successful review accuracy after data minimization.
B:Structured override trends reveal whether feedback improves the system over time; record correction reasons, provenance, and case labels under retention rules to support evaluation and process redesign; this tests recurrence rate of categorized, previously corrected error types; the actual release condition is unauthorized field exposure and successful review accuracy after data minimization.
C:A consequence-aware test checks whether unsupported authoritative decisions are avoided; define escalation thresholds and route consequential interpretations to qualified legal or compliance owners with source context; this tests correct escalation of ambiguous legal cases with reviewer attribution; the actual release condition is unauthorized field exposure and successful review accuracy after data minimization.
Question 546
A vendor-onboarding tool proposes changes that must pass a strict business schema before any write. A document-processing portal experiences slow morning jobs that exceed request timeouts. Separately, editors make corrections that never become reusable quality evidence. Which THREE implementation practices directly solve these problems? Choose THREE.
- Validate model output against a business schema before side effects
- Use a queue and asynchronous workers for bursty workloads
- Link each output to source and processing trace IDs
- Persist workflow state and make retries idempotent
- Capture reviewer corrections as labeled feedback
- Normalize incoming documents into a validated input contract
Explanation
Correct Answers
A:Use structured output parsing, semantic validation, and a separate authorized command handler so generated text is never treated directly as a write instruction; pre-write enforcement proves that the model cannot bypass the target system's integrity contract; its corresponding workstream should be validated by invalid write payloads rejected before execution, with zero partial supplier mutations.
B:Decouple intake from processing through durable queues, bounded workers, and status callbacks rather than keeping user connections open; measuring backlog and completion under load tests the exact burst tolerance the design promises; its corresponding workstream should be validated by job completion rate and queue age during the peak intake window.
E:Preserve authorized reviewer labels linked to case versions and turn them into regression examples with data-retention controls; retesting concrete corrections detects whether a release reintroduces known business mistakes; its corresponding workstream should be validated by regression pass rate on previously corrected clause-omission cases.
Incorrect Answers
C:Correlation IDs, source versions, and stage logs establish end-to-end lineage across extraction, retrieval, inference, and review; traceability explains decisions after the fact; its natural acceptance measure is fraction of sampled outputs reproducibly linked to input, model configuration, and reviewer disposition, which does not satisfy the separate workstreams being assessed.
D:Durable state and idempotency would prevent duplicate supplier updates after retries, but the procurement workstream is concerned with schema validity before writes, the intake service needs asynchronous processing, and legal review needs corrections captured as feedback.
F:An input-adapter layer validates document type, field types, provenance, and required metadata before the model reasons over values; input normalization addresses upstream ambiguity, not the authorization needed before writing to a finance ledger; its natural acceptance measure is percentage of ingested documents meeting schema checks before any Claude call, which does not satisfy the separate workstreams being assessed.
Question 547
A structured extractor returns valid objects for standard inputs but omits mandatory properties whenever notes contain unusual punctuation. Which diagnostic approach should examine the prompt-to-schema boundary?
- Verify whether claims are supported by supplied sources
- Trace malformed outputs to prompt or schema boundaries
- Test for model capability mismatch on complex reasoning
- Investigate retrieval recall before changing the language model
Explanation
Correct Answer
B:Inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; edge-specific parser tests identify contract failures separately from content comprehension; verify the repair using schema error rate across multiline, null, and unusual-field edge cases.
Incorrect Answers
A:A well-formatted citation is not proof that the cited text supports a particular assertion; compare material claims against cited evidence and flag unsupported statements for abstention or review; that control is assessed by supported-claim rate on source-checked summaries with seeded unsupported numbers; the incident instead requires schema error rate across multiline, null, and unusual-field edge cases.
C:Rebuilding retrieval is unlikely to help when required evidence was already consistently supplied; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; that control is assessed by correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the incident instead requires schema error rate across multiline, null, and unusual-field edge cases.
D:Model upgrades cannot recover information that never reaches the answer context; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; that control is assessed by recall of newly published policy passages and removal of stale hits; the incident instead requires schema error rate across multiline, null, and unusual-field edge cases.
Question 548
A customer-facing assistant has agreed reliability targets and an error-budget policy. During an artificial incident, which result would establish that staff take the prescribed actions when those targets are breached?
- SLO compliance and prescribed actions during a simulated error-budget breach
- Affected-user awareness and support-ticket volume following staged changes
- Ownership coverage for failures across model, application, data, and tool layers
- Time from actionable feedback to triage, owner assignment, and verified fix
- Time to appropriate escalation and fallback activation during drills
Explanation
Correct Answer
A:Operational SLO tests connect observed service quality to response commitments; measure sLO compliance and prescribed actions during a simulated error-budget breach at the release gate; establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions.
Incorrect Answers
B:Communication quality is reflected in predictable adoption and fewer avoidable surprises; define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; this tests affected-user awareness and support-ticket volume following staged changes; the actual release condition is sLO compliance and prescribed actions during a simulated error-budget breach.
C:Responsibility tests prevent gaps between contractual expectations and implemented controls; define provider, platform, application, security, and business-owner responsibilities in an explicit operating model; this tests ownership coverage for failures across model, application, data, and tool layers; the actual release condition is sLO compliance and prescribed actions during a simulated error-budget breach.
D:Issue lifecycle metrics show whether feedback produces controlled improvement; classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; this tests time from actionable feedback to triage, owner assignment, and verified fix; the actual release condition is sLO compliance and prescribed actions during a simulated error-budget breach.
E:Incident response tests verify that operational ownership works under pressure; define incident severity, on-call ownership, manual fallback authority, and customer communication triggers; this tests time to appropriate escalation and fallback activation during drills; the actual release condition is sLO compliance and prescribed actions during a simulated error-budget breach.
Question 549
Cheap responses increase review labor because more outputs need manual correction. Which change should the architect recommend first?
- Measure rate and severity of safety policy violations
- Evaluate whether abstention and escalation are calibrated
- Use tail-latency indicators for interactive promises
- Track cost per verified successful task
Explanation
Correct Answer
D:Relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; this metric includes rework rather than rewarding low-quality cheap drafts; verify the repair using fully loaded cost per correctly completed document.
Incorrect Answers
A:General task correctness does not measure policy-conformance failures on dangerous requests; define safety incident categories, risk-weighted violation rates, and appropriate exposure denominators for adversarial and real traffic; that control is assessed by severity-weighted policy violation incidence on a reviewed adversarial test set; the incident instead requires fully loaded cost per correctly completed document.
B:High refusal frequency alone can appear safe while making the assistant operationally useless; measure selective risk: error rate on answered items alongside abstention precision, coverage, and escalation cost; that control is assessed by correct-answer rate at each supported coverage threshold including no-answer cases; the incident instead requires fully loaded cost per correctly completed document.
C:Mean latency underweights rare long waits and omits retrieval and external-tool delays; report p95 or p99 end-to-end latency over representative traffic rather than relying only on averages or raw model runtime; that control is assessed by p95 time from user submission to fully validated answer under load; the incident instead requires fully loaded cost per correctly completed document.
Question 550
A case-management office wants to know whether AI assistance reduces its actual workload after corrections and appeals, not just the time to first draft. What result should be reported?
- End-to-end completed cases per paid work-hour with rework and appeal volumes
- Total operational cost divided by approved documents meeting the quality rubric
- Incremental resolution lift compared with a comparable non-assisted cohort
- End-to-end p95 latency and error rate against the agreed service objective
Explanation
Correct Answer
A:Net throughput reflects business output rather than the volume of AI text produced; measure end-to-end completed cases per paid work-hour with rework and appeal volumes at the release gate; measure net time saved after review, appeals, and escalations, segmented by complexity, before claiming productivity.
Incorrect Answers
B:Unit cost tied to accepted results distinguishes economical quality from cheap failed attempts; combine inference, retrieval, tools, review, and rework costs into cost per accepted deliverable; this tests total operational cost divided by approved documents meeting the quality rubric; the actual release condition is end-to-end completed cases per paid work-hour with rework and appeal volumes.
C:A defensible counterfactual is required to credit the architecture with the observed benefit; use matched baselines, controlled rollout, or appropriate difference-in-differences analysis to isolate contribution under plausible confounders; this tests incremental resolution lift compared with a comparable non-assisted cohort; the actual release condition is end-to-end completed cases per paid work-hour with rework and appeal volumes.
D:Tail measurement detects the bad experiences that averages can obscure; define a p95 or p99 target against real end-to-end requests, including retrieval and tools, rather than reporting an average model time; this tests end-to-end p95 latency and error rate against the agreed service objective; the actual release condition is end-to-end completed cases per paid work-hour with rework and appeal volumes.