Question 200
A research summarizer may produce numerical findings even when the cited source lacks those numbers. What should the solution architect recommend?
- Validate tool-result provenance and semantics
- Counter automation bias with visible uncertainty and review
- Use claim verification to mitigate confident fabrication
- Threat-model prompt injection across lower-trust inputs
Explanation
Correct Answer
C:Require support from retrieved source passages and provide an explicit unknown path for unsupported claims; grounding tests detect confident fabrication at the statement level; evaluate the design with proportion of numeric claims supported by independently inspected references.
Incorrect Answers
A:Preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; a successful HTTP status does not mean returned data has decision authority; this alternative would measure zero external-result fields promoted to internal approvals in test cases; it does not establish proportion of numeric claims supported by independently inspected references for the described use case.
B:Design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button; a nominal human checkpoint is weak when the interface encourages blind confirmation; this alternative would measure reviewer error interception on seeded incorrect recommendations; it does not establish proportion of numeric claims supported by independently inspected references for the described use case.
D:Model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; stronger wording in the system prompt is not a complete defense against an authorized dangerous tool; this alternative would measure unauthorized tool-action success rate over injected emails and documents; it does not establish proportion of numeric claims supported by independently inspected references for the described use case.
Question 201
The model cites absent or obsolete passages even though its model version and prompt are unchanged. Which action best removes the underlying failure mode?
- Investigate retrieval recall before changing the language model
- Test for model capability mismatch on complex reasoning
- Revalidate few-shot examples after policy changes
- Separate external tool faults from model decisions
Explanation
Correct Answer
A:Inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; a retrieval-specific test isolates the component that changed; verify the repair using recall of newly published policy passages and removal of stale hits.
Incorrect Answers
B:Rebuilding retrieval is unlikely to help when required evidence was already consistently supplied; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; that control is assessed by correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the incident instead requires recall of newly published policy passages and removal of stale hits.
C:Index re-embedding cannot resolve a stale rule embedded directly in the prompt template; audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; that control is assessed by correct labels on scenarios affected by the policy revision; the incident instead requires recall of newly published policy passages and removal of stale hits.
D:Prompt changes alone cannot restore a failing service or make an empty placeholder a valid business result; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; that control is assessed by correct 'unknown' handling and no false denials during injected API failures; the incident instead requires recall of newly published policy passages and removal of stale hits.
Question 202
A helpdesk deployment consolidates overlapping ticket connectors into a single versioned route. Replay tests use the same account and issue details repeatedly. Which result demonstrates that the routing is deterministic and state-safe?
- Zero mutation permissions for report-only identities across tool contract checks
- Changes rejected when requested tool scopes exceed the approved task manifest
- Absence of privileged tool calls before the confirmed transaction stage
- Unauthorized tool availability and invocation count in hostile transcript tests
- Single correct connector selection and consistent ticket states across repeated requests
Explanation
Correct Answer
E:Removing ambiguity should reduce routing variance without reducing supported tasks; measure single correct connector selection and consistent ticket states across repeated requests at the release gate; consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing.
Incorrect Answers
A:Permission tests reveal whether supposedly analytical agents can cause side effects; expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; this tests zero mutation permissions for report-only identities across tool contract checks; the actual release condition is single correct connector selection and consistent ticket states across repeated requests.
B:Policy-based manifest validation detects excessive authority before runtime exposure; maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; this tests changes rejected when requested tool scopes exceed the approved task manifest; the actual release condition is single correct connector selection and consistent ticket states across repeated requests.
C:Stage-bound tests prove that tool exposure follows authorization and intent; reveal high-impact tools only within a confirmed, authorized transaction stage; maintain server-side gating independently of model choice; this tests absence of privileged tool calls before the confirmed transaction stage; the actual release condition is single correct connector selection and consistent ticket states across repeated requests.
D:Capability inventory and adversarial execution tests verify that unnecessary actions are unreachable; give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; this tests unauthorized tool availability and invocation count in hostile transcript tests; the actual release condition is single correct connector selection and consistent ticket states across repeated requests.
Question 203
One business unit revises a support escalation rule, but three applications still generate answers from older local prompt copies. What design would prevent that instruction drift?
- Modularize shared task instructions for consistent reuse
- Keep dynamic fields out of cached prefix content
- Cache a stable, high-volume prompt prefix
- Compare cache lifetime with actual reuse patterns
Explanation
Correct Answer
A:Maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; a shared module succeeds when consuming workflows inherit the same reviewed rule set; verify the repair using cross-application prompt-version alignment and regression pass rate.
Incorrect Answers
B:A cache breakpoint cannot compensate for changes to earlier content in the hashed prefix; move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; that control is assessed by prefix reuse ratio when request metadata changes across calls; the incident instead requires cross-application prompt-version alignment and regression pass rate.
C:Caching helps repeated identical prefixes; it does not improve a model's answer accuracy on missing knowledge; put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; that control is assessed by cache-read input tokens and end-to-end latency over repeated requests; the incident instead requires cross-application prompt-version alignment and regression pass rate.
D:A long cache lifetime is not automatically economical when there is little repeated work; assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; that control is assessed by net cost per accepted request across realistic interarrival times; the incident instead requires cross-application prompt-version alignment and regression pass rate.
Question 204
A learning portal receives questions phrased unlike the wording used in its official manuals. Which strategy should be prioritized for this requirement?
- Use semantic vector search for paraphrased knowledge questions
- Query structured data through authorized SQL or typed APIs
- Retrieve parent context after finding a specific chunk
- Use lexical search for exact identifiers and literals
- Rerank candidate passages for question relevance
Explanation
Correct Answer
A:Vector retrieval can match conceptual similarity between user wording and document text, followed by evidence verification; paraphrase benchmark cases test the capability for which embeddings are chosen; evaluate the design with recall on human-authored paraphrases absent from manual headings.
Incorrect Answers
B:Use a governed structured query interface with parameter checks and business semantics for exact aggregates; unstructured document retrieval cannot ensure completeness or mathematical correctness for database totals; this alternative would measure exact aggregate agreement with a trusted query on authorized records; it does not establish recall on human-authored paraphrases absent from manual headings for the described use case.
C:Use child-chunk retrieval for recall and fetch validated parent context or adjacent governing sections for interpretation; passing isolated small chunks may preserve relevance while losing the context needed for a correct conclusion; this alternative would measure conditional-rule accuracy with and without parent-document expansion; it does not establish recall on human-authored paraphrases absent from manual headings for the described use case.
D:Keyword or lexical matching is effective for codes, exact phrases, and terms whose characters carry the meaning; semantic similarity alone can rank conceptually related but operationally different identifiers; this alternative would measure exact-ID hit rate at the top retrieval ranks; it does not establish recall on human-authored paraphrases absent from manual headings for the described use case.
E:Apply a query-aware reranker after broad retrieval, respecting freshness and permissions, to prioritize evidence for synthesis; a bigger first-stage candidate set without selection can dilute context and increase cost; this alternative would measure relevant-procedure rank and end-to-end answer support rate after reranking; it does not establish recall on human-authored paraphrases absent from manual headings for the described use case.
Question 205
A finance assistant rarely needs a write-off action but receives the write tool on every exploratory chat turn. Which approach best meets the stated requirement?
- Remove duplicate and deprecated tools from the catalog
- Offer each agent only the tools required for its role
- Separate read-only retrieval from mutating tools
- Stage privileged capabilities only after explicit intent confirmation
Explanation
Correct Answer
D:Reveal high-impact tools only within a confirmed, authorized transaction stage; maintain server-side gating independently of model choice; stage-bound tests prove that tool exposure follows authorization and intent; evaluate the design with absence of privileged tool calls before the confirmed transaction stage.
Incorrect Answers
A:Consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; more tool variants do not add genuine capability when they expose the same action with conflicting contracts; this alternative would measure single correct connector selection and consistent ticket states across repeated requests; it does not establish absence of privileged tool calls before the confirmed transaction stage for the described use case.
B:Give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; monitoring a broad tool catalog does not remove the authority to invoke dangerous operations; this alternative would measure unauthorized tool availability and invocation count in hostile transcript tests; it does not establish absence of privileged tool calls before the confirmed transaction stage for the described use case.
C:Expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; a generic read-write interface offers more authority than the reporting workflow requires; this alternative would measure zero mutation permissions for report-only identities across tool contract checks; it does not establish absence of privileged tool calls before the confirmed transaction stage for the described use case.
Question 206
Reported cache reads remain zero for most sessions despite repeated policy templates. Which change should the architect recommend first?
- Cache a stable, high-volume prompt prefix
- Instrument cache effectiveness with token usage counters
- Keep dynamic fields out of cached prefix content
- Modularize shared task instructions for consistent reuse
Explanation
Correct Answer
B:Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; cache-specific counters distinguish a configuration issue from ordinary inference variance; verify the repair using sustained cache-read token share under repeated workload replay.
Incorrect Answers
A:Caching helps repeated identical prefixes; it does not improve a model's answer accuracy on missing knowledge; put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; that control is assessed by cache-read input tokens and end-to-end latency over repeated requests; the incident instead requires sustained cache-read token share under repeated workload replay.
C:A cache breakpoint cannot compensate for changes to earlier content in the hashed prefix; move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; that control is assessed by prefix reuse ratio when request metadata changes across calls; the incident instead requires sustained cache-read token share under repeated workload replay.
D:Prompt modularity simplifies governance; it cannot alone guarantee that a sensitive tool call is authorized; maintain a versioned reusable prompt module with explicit parameters and regression tests across consumers; that control is assessed by cross-application prompt-version alignment and regression pass rate; the incident instead requires sustained cache-read token share under repeated workload replay.
Question 207
A request template inserts the current time and a session identifier before a shared compliance manual. What is the most defensible next step?
- Keep dynamic fields out of cached prefix content
- Cache a stable, high-volume prompt prefix
- Compare cache lifetime with actual reuse patterns
- Instrument cache effectiveness with token usage counters
Explanation
Correct Answer
A:Move changing metadata after the stable prefix so cache keys remain reusable across otherwise similar requests; controlled request pairs reveal avoidable cache invalidation; evaluate the design with prefix reuse ratio when request metadata changes across calls.
Incorrect Answers
B:Put reusable instructions and reference material in a consistent prefix and use supported prompt-cache controls while placing variable requests afterward; caching helps repeated identical prefixes; it does not improve a model's answer accuracy on missing knowledge; this alternative would measure cache-read input tokens and end-to-end latency over repeated requests; it does not establish prefix reuse ratio when request metadata changes across calls for the described use case.
C:Assess eligible caching windows, traffic frequency, and write/read economics against real reuse intervals before selecting a cache policy; a long cache lifetime is not automatically economical when there is little repeated work; this alternative would measure net cost per accepted request across realistic interarrival times; it does not establish prefix reuse ratio when request metadata changes across calls for the described use case.
D:Compare cache-creation and cache-read token telemetry and inspect prefix ordering, eligible size, and reuse timing; a faster model might improve latency but would not explain a missing cache hit on identical prompts; this alternative would measure sustained cache-read token share under repeated workload replay; it does not establish prefix reuse ratio when request metadata changes across calls for the described use case.
Question 208
Users give high satisfaction ratings to an invoice-processing assistant while audited misclassification rates keep increasing. Which performance measure should govern the release decision?
- Test whether the tool layer denies unauthorized requests
- Track cost per verified successful task
- Use task-correctness metrics tied to verified labels
- Evaluate whether abstention and escalation are calibrated
Explanation
Correct Answer
C:Measure the business decision against a labeled reference, with metrics weighted for costly errors and explicit acceptance thresholds; label-based metrics catch mistakes that surface-level satisfaction may conceal; verify the repair using critical-class precision and recall on independently verified tax-code cases.
Incorrect Answers
A:Model refusal scores can be high even when the underlying API is vulnerable; treat backend authorization denial rate and cross-scope access tests as primary security metrics, not merely refusal wording; that control is assessed by unauthorized API requests rejected under role, tenant, and resource attack matrices; the incident instead requires critical-class precision and recall on independently verified tax-code cases.
B:Cost per token can improve while cost per correct business outcome deteriorates; relate full inference and review costs to accepted outcomes that meet the task rubric, not to requests attempted; that control is assessed by fully loaded cost per correctly completed document; the incident instead requires critical-class precision and recall on independently verified tax-code cases.
D:High refusal frequency alone can appear safe while making the assistant operationally useless; measure selective risk: error rate on answered items alongside abstention precision, coverage, and escalation cost; that control is assessed by correct-answer rate at each supported coverage threshold including no-answer cases; the incident instead requires critical-class precision and recall on independently verified tax-code cases.
Question 209
A customer requests removal of stored information used by an AI application. The architecture has a primary database, vector index, and incident archives. Which outcome would establish complete deletion within the required policy window?
- Deletion-request completion across all inventoried repositories within policy deadlines
- Approved contracting, access controls, and PHI data-flow verification for the selected service path
- Documented authorization-boundary mapping for every service handling regulated data
- Completion of required privacy assessment and recorded resolution of high risks before go-live
Explanation
Correct Answer
A:End-to-end lifecycle tests verify more than the source system's delete operation; measure deletion-request completion across all inventoried repositories within policy deadlines at the release gate; map data copies, implement lifecycle deletion where required, and retain audit evidence of the authorized request and its completion.
Incorrect Answers
B:Compliance verification must match the actual product configuration and agreement; confirm the specific service and contract support relevant HIPAA obligations, including any required business associate agreement and safeguards; this tests approved contracting, access controls, and PHI data-flow verification for the selected service path; the actual release condition is deletion-request completion across all inventoried repositories within policy deadlines.
C:Boundary-specific checks prevent assuming certification transfers between unrelated services; verify the specific service, region, impact level, boundary, and agency authorization requirements with security officials before selection; this tests documented authorization-boundary mapping for every service handling regulated data; the actual release condition is deletion-request completion across all inventoried repositories within policy deadlines.
D:Compliance readiness must be demonstrated upstream of irreversible data processing; involve privacy counsel to assess applicable lawful bases, transparency, rights, and whether a data-protection impact assessment is required before release; this tests completion of required privacy assessment and recorded resolution of high risks before go-live; the actual release condition is deletion-request completion across all inventoried repositories within policy deadlines.
Question 210
A single giant context drops older filings and conceals conflicts between reporting periods. How should the design be corrected?
- Checkpoint long-running work for replay and recovery
- Specify input and output contracts for each subtask
- Partition research by source and reconcile conflicts explicitly
- Divide work along independently verifiable artifacts
Explanation
Correct Answer
C:Analyze source partitions separately, retain provenance, and reconcile contradictions through a dedicated synthesis stage; the final synthesis must expose disagreements rather than average them away; verify the repair using conflict detection recall and citation coverage across independently processed source sets.
Incorrect Answers
A:Checkpointing reduces replay cost but cannot correct a defective classification rubric; durable checkpoints record validated intermediate artifacts and resume at safe boundaries with idempotent side effects; that control is assessed by successful recovery after injected worker failures without duplicated writes or stages; the incident instead requires conflict detection recall and citation coverage across independently processed source sets.
B:Contracts define allowed shapes and semantics but cannot ensure a source is truthful; typed versioned contracts and compatibility tests let stages evolve while preserving their callers' assumptions; that control is assessed by contract test pass rate for independently upgraded stages and invalid payload rejection; the incident instead requires conflict detection recall and citation coverage across independently processed source sets.
D:Artifact decomposition controls work structure but does not set the organization-wide release SLA; separate stages with typed artifacts allow validation and targeted reruns at meaningful boundaries; that control is assessed by stage-specific defect localization and repair without redoing previously validated stages; the incident instead requires conflict detection recall and citation coverage across independently processed source sets.