Question 590
A software squad wants AI-assisted API documentation after a major service refactor. Which option addresses the immediate design need?
- Review AI-generated diffs for logic and security regressions
- Break large refactors into independently reviewable changes
- Use test-first acceptance cases for generated changes
- Generate documentation from verified source and review it
- Run reproducible CI gates on assisted code
Explanation
Correct Answer
D:Ground documentation generation in current code, schemas, and tests, then validate examples and have owners review changes; executable examples reveal fabricated or obsolete instructions; evaluate the design with documented endpoint examples executing successfully against the reviewed build.
Incorrect Answers
A:Require focused code review of the actual diff, threat-relevant tests, and approval before merging sensitive changes; successful compilation does not establish preservation of security controls; this alternative would measure detection of seeded authorization regressions during peer review and CI; it does not establish documented endpoint examples executing successfully against the reviewed build for the described use case.
B:Use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; one massive patch can reduce initial coordination but increases regression and rollback risk; this alternative would measure review time and defect localization per incremental change set; it does not establish documented endpoint examples executing successfully against the reviewed build for the described use case.
C:Define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; merely reading generated code for plausibility does not replace tests of business invariants; this alternative would measure passing unit and property tests including specified rounding boundaries; it does not establish documented endpoint examples executing successfully against the reviewed build for the described use case.
E:Execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; local success can be environment-dependent and should not be the only release assurance; this alternative would measure cI pass reproducibility on clean builds with locked dependencies; it does not establish documented endpoint examples executing successfully against the reviewed build for the described use case.
Question 591
A mailroom team is piloting a lower-cost classifier for routine correspondence, but missed urgent notices would be expensive. Which validation result would demonstrate that the substitution is both economical and safe for minority labels?
- Per-segment acceptance rate across language, length, and noisy-input strata
- High-risk escalation recall and total cost under the required error ceiling
- Correct equipment identification and recommendation accuracy on labeled photographs
- No contract regressions and acceptable quality/latency deltas on the proposed release
- Macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic
Explanation
Correct Answer
E:Joint quality and price measurement prevents choosing a cheap model that fails costly minority categories; measure macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic at the release gate; compare an efficient Claude model against labeled classification cases and select the lowest-cost model that meets the defined accuracy floor.
Incorrect Answers
A:Stratified scores expose failures that an overall aggregate conceals; representative workload evaluation must include input quality, document lengths, languages, and failure consequences before setting model policy; this tests per-segment acceptance rate across language, length, and noisy-input strata; the actual release condition is macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic.
B:Routing is valuable only if it captures difficult cases without needlessly escalating every simple request; combine an evaluated uncertainty trigger with escalation to a stronger model or reviewer, using explicit limits and post-route quality measurement; this tests high-risk escalation recall and total cost under the required error ceiling; the actual release condition is macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic.
C:A multimodal acceptance set measures the capability absent from text-only trials; confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; this tests correct equipment identification and recommendation accuracy on labeled photographs; the actual release condition is macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic.
D:Version-aware tests make improvement claims meaningful for the deployed application; treat model replacements as versioned releases with task-specific eval gates, contract tests, canaries, and rollback capability; this tests no contract regressions and acceptable quality/latency deltas on the proposed release; the actual release condition is macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic.
Question 592
A newly deployed coding hook unexpectedly blocks all commit operations. The team runs an approved rollback drill. Which restoration result would demonstrate that prior working configurations can be recovered promptly?
- Bug reproduction on sanitized fixtures with no sensitive token disclosure
- Correct root-cause attribution under injected search timeouts
- Time to restore normal operation using an approved previous release
- Recurrence rate of the diagnosed failure and completion of preventive actions
Explanation
Correct Answer
C:A rollback drill verifies that recovery does not depend on the original developer being online; measure time to restore normal operation using an approved previous release at the release gate; maintain versioned releases and documented rollback steps for hooks, prompts, model routes, and connector configuration.
Incorrect Answers
A:Safe replay demonstrates that privacy and diagnostic value can coexist; build redacted, access-controlled replay fixtures preserving relevant syntax and error conditions without unnecessary secrets or personal data; this tests bug reproduction on sanitized fixtures with no sensitive token disclosure; the actual release condition is time to restore normal operation using an approved previous release.
B:Controlled fault injection tests whether the team diagnoses the failing layer; classify tool errors separately from model responses, validate return codes, and probe the failing integration before changing prompts; this tests correct root-cause attribution under injected search timeouts; the actual release condition is time to restore normal operation using an approved previous release.
D:Follow-up measures show whether operational learning survives the immediate repair; run blameless root-cause analysis, link corrective actions to owners, and add regression or operational safeguards before closing the incident; this tests recurrence rate of the diagnosed failure and completion of preventive actions; the actual release condition is time to restore normal operation using an approved previous release.
Question 593
A production RAG release causes stale policy answers after source indexing fails. What is the most targeted architecture choice?
- Publish an incident runbook with containment and rollback steps
- Prepare operator onboarding and support handoff materials
- Define tool schemas, errors, and authorization contracts
- Maintain an end-to-end data-flow and trust-boundary diagram
Explanation
Correct Answer
A:Maintain runbooks for detection, containment, safe fallback, rollback, and communication with named incident roles; a runbook is effective when a different on-call engineer can perform the recovery; evaluate the design with successful containment time in a simulated stale-index incident.
Incorrect Answers
B:Provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; source code alone rarely communicates the operational decisions and permission steps operators need; this alternative would measure successful completion of standard support drills by staff who did not build the system; it does not establish successful containment time in a simulated stale-index incident for the described use case.
C:Define typed inputs, outputs, error semantics, idempotency, required scopes, and audit obligations for tool boundaries; aPI examples without failure and permission semantics leave dangerous integration ambiguity; this alternative would measure consumer contract pass rate under timeouts, invalid arguments, and permission denials; it does not establish successful containment time in a simulated stale-index incident for the described use case.
D:A current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects; a feature list does not reveal crossing of sensitive trust boundaries; this alternative would measure completeness of sensitive-data flow paths identified during security review; it does not establish successful containment time in a simulated stale-index incident for the described use case.
Question 594
A service shifts a fraction of traffic to a revised prompt, then sees a higher hallucination rate. Which strategy should be prioritized for this requirement?
- Compare production metrics by model and prompt version
- Instrument end-to-end execution spans for causal debugging
- Alert on unexpected token and cost anomalies
- Track safety policy events across product updates
Explanation
Correct Answer
A:Partition traces and scored outcomes by deployed configuration, using release identifiers to support rollback and diagnosis; release segmentation isolates changes before broad impact; evaluate the design with detection of version-specific quality regressions in a staged rollout.
Incorrect Answers
B:Correlation and span IDs across components allow latency and failure attribution along the actual path; model usage totals do not reveal which integration failed within a particular request; this alternative would measure percent of failures with a complete trace to the responsible stage; it does not establish detection of version-specific quality regressions in a staged rollout for the described use case.
C:Monitor input/output tokens, cache reads, per-stage costs, and cost-per-task anomalies with accountable ownership; request-volume alerts alone miss a sudden rise in tokens per individual workflow; this alternative would measure detection lag for injected per-request token inflation; it does not establish detection of version-specific quality regressions in a staged rollout for the described use case.
D:Monitor both harmful-output violations and false-positive blocks by version and user segment; a falling count of harmful completions can coexist with an unusable overrestrictive assistant; this alternative would measure safety precision/recall and legitimate-task completion before and after the change; it does not establish detection of version-specific quality regressions in a staged rollout for the described use case.
Question 595
A developer assistant receives internal debugging traces that sometimes contain tokens or credentials. What action best fits the business and technical boundary?
- Delimit retrieved content as untrusted evidence
- Keep invariant policy instructions distinct from per-request input
- Write explicit roles, scope, and prohibited operations
- Limit disclosure of secrets and confidential instructions
Explanation
Correct Answer
D:Redact sensitive fields upstream, prohibit secret reproduction in instructions, and screen outbound content before release; seeded canaries allow a measurable check of unwanted disclosure paths; evaluate the design with secret-exposure rate under seeded credential-bearing diagnostic inputs.
Incorrect Answers
A:Clearly separate lower-trust source material from application instructions and enforce permissions at the tool layer as well; delimiters aid instruction hierarchy but do not sanitize malicious destinations or replace output security controls; this alternative would measure prompt-injection resistance on hostile retrieved snippets without loss of legitimate source use; it does not establish secret-exposure rate under seeded credential-bearing diagnostic inputs for the described use case.
B:Separate stable system-level policy from user content and tool results, retaining consistent privilege boundaries across requests; a modular instruction layout does not itself prove that external API permissions are enforced; this alternative would measure rate of user messages that fail to override defined task boundaries in regression tests; it does not establish secret-exposure rate under seeded credential-bearing diagnostic inputs for the described use case.
C:Define the assistant's permitted tasks, out-of-scope handling, and escalation behavior in stable application-level instructions, backed by independent access enforcement; role instructions improve behavior but cannot grant or restrict database permissions on their own; this alternative would measure out-of-scope refusal and handoff accuracy on a policy-boundary test set; it does not establish secret-exposure rate under seeded credential-bearing diagnostic inputs for the described use case.
Question 596
A project assistant compresses extensive working discussions while keeping approved implementation requirements in durable memory. Which regression result would establish that compaction preserves critical commitments?
- Correct recovery of approvals and pending actions after forced session reset
- Retention of signed-off requirements after long simulated conversations
- Citation correctness by effective-date version in temporal regression questions
- Complete mandatory report sections under the worst supported input size
Explanation
Correct Answer
B:A long-horizon test checks memory of obligations rather than raw conversation recall; measure retention of signed-off requirements after long simulated conversations at the release gate; store a structured, reviewable summary of decisions, open issues, and provenance; inject it at checkpoints rather than replaying every chat turn.
Incorrect Answers
A:Durable-state testing proves behavior survives context eviction; store approvals and workflow state in an authoritative durable system; treat chat history as context, not a transaction ledger; this tests correct recovery of approvals and pending actions after forced session reset; the actual release condition is retention of signed-off requirements after long simulated conversations.
C:Metadata-aware checks distinguish legitimate historic references from current-policy answers; supply source identity, version, effective date, and retrieval relevance alongside each chunk so the model can attribute evidence correctly; this tests citation correctness by effective-date version in temporal regression questions; the actual release condition is retention of signed-off requirements after long simulated conversations.
D:Budgeted completion tests directly reveal truncation at maximum expected load; reserve output capacity and bound each stage's retrieved context and response length according to the deliverable structure; this tests complete mandatory report sections under the worst supported input size; the actual release condition is retention of signed-off requirements after long simulated conversations.
Question 597
An organization must honor approved deletion requests for customer data used by an AI service. Which design decision directly addresses this constraint?
- Implement accountable access, deletion, and retention processes
- Check HIPAA business associate and safeguard requirements
- Minimize personal data and specify retention purpose
- Validate FedRAMP authorization against exact service boundaries
Explanation
Correct Answer
A:Map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion; end-to-end lifecycle tests verify more than the source system's delete operation; evaluate the design with deletion-request completion across all inventoried repositories within policy deadlines.
Incorrect Answers
B:Confirm the specific service and contract support relevant HIPAA obligations, including any required business associate agreement and safeguards; a provider security claim alone does not prove that the purchased deployment is contractually suitable for PHI; this alternative would measure approved contracting, access controls, and PHI data-flow verification for the selected service path; it does not establish deletion-request completion across all inventoried repositories within policy deadlines for the described use case.
C:Limit processing to necessary authorized fields, define a lawful purpose and retention controls, and document relevant privacy obligations; broad encryption does not justify collecting more personal data than the task needs; this alternative would measure sensitive-field inclusion rate in prompts and retention-policy compliance; it does not establish deletion-request completion across all inventoried repositories within policy deadlines for the described use case.
D:Verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection; a general government-cloud label or vendor marketing statement is not equivalent to FedRAMP authorization; this alternative would measure documented authorization-boundary mapping for every service handling regulated data; it does not establish deletion-request completion across all inventoried repositories within policy deadlines for the described use case.
Question 598
A credential is copied into a shared instruction file and later appears in a generated diagnostic diff. What design correction resolves the reported failure?
- Limit MCP connections by project and user identity
- Use specialized subagents with bounded tools and instructions
- Keep credentials out of prompts and versioned configuration
- Use deterministic hooks to enforce required developer gates
Explanation
Correct Answer
C:Use approved secret storage, scoped environment access, and redaction while avoiding credentials in repositories, prompts, or logs; source and runtime scans target the actual paths by which developer secrets can leak; verify the repair using secret-scanning results and blocked exfiltration on seeded repository credentials.
Incorrect Answers
A:A working connection does not prove the assistant is restricted to the user's intended project resources; choose appropriate configuration scope and per-user or service authorization for MCP servers, avoiding shared cross-project secrets; that control is assessed by access denial for cross-project MCP resources under user identity tests; the incident instead requires secret-scanning results and blocked exfiltration on seeded repository credentials.
B:Delegation without role scoping merely multiplies the number of powerful agents; give each delegated role explicit objectives, context limits, and tool permissions with clear return artifacts; that control is assessed by subagent task completion with zero out-of-scope tool access; the incident instead requires secret-scanning results and blocked exfiltration on seeded repository credentials.
D:A written instruction alone is best-effort and does not guarantee enforcement of a critical command gate; configure supported hooks for deterministic checks at appropriate lifecycle points, with clear failure behavior and maintenance ownership; that control is assessed by hook execution and blocked noncompliant operations under simulated bypass attempts; the incident instead requires secret-scanning results and blocked exfiltration on seeded repository credentials.
Question 599
A diligence platform assigns confidential material to legally distinct review specialists. Security is assessing whether each specialist's evidence stays inside its approved scope. Which result establishes that the separation works?
- Zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record
- Zero stale-artifact merges across concurrent revision and retry scenarios
- Marginal quality improvement per added specialist against its incremental cost and latency
- Completed reports with correctly scoped evidence and no cross-specialty data leakage
Explanation
Correct Answer
D:Routing correctness must include scope enforcement, not just readable final reports; measure completed reports with correctly scoped evidence and no cross-specialty data leakage at the release gate; a coordinator can dispatch scoped subagents and merge their typed outputs while restricting each specialist's context and tools.
Incorrect Answers
A:Stressing unresolved cases proves that the system can fail predictably and within budget; enforce maximum depth, handoff count, timeouts, and explicit unresolved-case escalation in the orchestrator; this tests zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record; the actual release condition is completed reports with correctly scoped evidence and no cross-specialty data leakage.
B:Version-aware handoffs can be tested deterministically under racing writes; immutable, versioned handoff artifacts and typed schemas prevent one agent from consuming ambiguous or mismatched intermediate state; this tests zero stale-artifact merges across concurrent revision and retry scenarios; the actual release condition is completed reports with correctly scoped evidence and no cross-specialty data leakage.
C:Measuring the contribution of each delegate reveals needless orchestration layers; introduce a specialist only when its independent expertise, tool scope, or parallelism beats a simpler call on representative evaluations; this tests marginal quality improvement per added specialist against its incremental cost and latency; the actual release condition is completed reports with correctly scoped evidence and no cross-specialty data leakage.
Question 600
A low-traffic legal system repeats a long analysis prefix only once every forty minutes. Which design decision directly addresses this constraint?
- Instrument cache effectiveness with token usage counters
- Modularize shared task instructions for consistent reuse
- Compare cache lifetime with actual reuse patterns
- Cache a stable, high-volume prompt prefix
Explanation
Correct Answer
C:Assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; reuse-adjusted economics tests whether the caching decision pays off under actual traffic; evaluate the design with net cost per accepted request across realistic interarrival times.
Incorrect Answers
A:Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; a faster model might improve latency but would not explain a missing cache hit on identical prompts; this alternative would measure sustained cache-read token share under repeated workload replay; it does not establish net cost per accepted request across realistic interarrival times for the described use case.
B:Maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; this alternative would measure cross-application prompt-version alignment and regression pass rate; it does not establish net cost per accepted request across realistic interarrival times for the described use case.
D:Put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; caching helps repeated identical prefixes; it does not improve a model's answer accuracy on missing knowledge; this alternative would measure cache-read input tokens and end-to-end latency over repeated requests; it does not establish net cost per accepted request across realistic interarrival times for the described use case.