Question 630
A market-analysis report contains precise figures and citations, but reviewers suspect several amounts are unsupported by the underlying documents. Which audit measure would establish factual traceability for numeric claims?
- Blocked unauthorized destinations and absence of confidential fields in egress logs
- Proportion of numeric claims supported by independently inspected references
- Maximum resource usage and orderly termination on deliberately unsolvable tasks
- Unauthorized tool-action success rate over injected emails and documents
Explanation
Correct Answer
B:Grounding tests detect confident fabrication at the statement level; measure proportion of numeric claims supported by independently inspected references at the release gate; require support from retrieved source passages and provide an explicit unknown path for unsupported claims.
Incorrect Answers
A:Network- and tool-level evidence catches leakage not visible in user-facing text; restrict egress destinations, data classes, and tool arguments at the network and application layers, with audit trails; this tests blocked unauthorized destinations and absence of confidential fields in egress logs; the actual release condition is proportion of numeric claims supported by independently inspected references.
C:Budget stress tests show whether runaway loops fail safely; set per-task step, token, cost, and elapsed-time budgets with a clear abort and escalation state; this tests maximum resource usage and orderly termination on deliberately unsolvable tasks; the actual release condition is proportion of numeric claims supported by independently inspected references.
D:A realistic attack corpus exercises both model behavior and execution controls; model external content as untrusted input, test role spoofing and instruction attempts, and enforce downstream permissions regardless of the prompt; this tests unauthorized tool-action success rate over injected emails and documents; the actual release condition is proportion of numeric claims supported by independently inspected references.
Question 631
A change passes one developer's manual checks but fails in production because dependencies differed. Which change should the architect recommend first?
- Package reusable engineering instructions as governed Skills
- Generate documentation from verified source and review it
- Use test-first acceptance cases for generated changes
- Run reproducible CI gates on assisted code
Explanation
Correct Answer
D:Execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; independent automation prevents environmental drift from masquerading as code correctness; verify the repair using cI pass reproducibility on clean builds with locked dependencies.
Incorrect Answers
A:Shared instructions improve consistency but do not replace enforced approval or automated tests; maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; that control is assessed by checklist completeness across independent sessions using the reviewed Skill version; the incident instead requires cI pass reproducibility on clean builds with locked dependencies.
B:Fluent prose generated from old context is not a reliable record of a current API contract; ground documentation generation in current code, schemas, and tests, then validate examples and have owners review changes; that control is assessed by documented endpoint examples executing successfully against the reviewed build; the incident instead requires cI pass reproducibility on clean builds with locked dependencies.
C:Merely reading generated code for plausibility does not replace tests of business invariants; define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; that control is assessed by passing unit and property tests including specified rounding boundaries; the incident instead requires cI pass reproducibility on clean builds with locked dependencies.
Question 632
An extended design conversation reaches the context limit; old content crowds out a previously approved security requirement. How should the assistant preserve decisions across long sessions?
- Retain provenance metadata alongside each selected context segment
- Compact extended sessions without discarding commitments
- Persist operational decisions outside transient chat history
- Allocate input and output token budgets per workflow step
Explanation
Correct Answer
B:Store a structured, reviewable summary of decisions, open issues, and provenance; inject it at checkpoints rather than replaying every chat turn; a long-horizon test checks memory of obligations rather than raw conversation recall; verify the repair using retention of signed-off requirements after long simulated conversations.
Incorrect Answers
A:Provenance metadata does not guarantee the upstream search selected the right document in the first place; supply source identity, version, effective date, and retrieval relevance alongside each chunk so the model can attribute evidence correctly; that control is assessed by citation correctness by effective-date version in temporal regression questions; the incident instead requires retention of signed-off requirements after long simulated conversations.
C:Long context alone cannot provide transactional durability or a trustworthy authorization trail; store approvals and workflow state in an authoritative durable system; treat chat history as context, not a transaction ledger; that control is assessed by correct recovery of approvals and pending actions after forced session reset; the incident instead requires retention of signed-off requirements after long simulated conversations.
D:Merely expanding every context window can increase cost while leaving output truncation unaddressed; reserve output capacity and bound each stage's retrieved context and response length according to the deliverable structure; that control is assessed by complete mandatory report sections under the worst supported input size; the incident instead requires retention of signed-off requirements after long simulated conversations.
Question 633
An incident copilot must preserve the resolution owner's approval after worker restarts and session expiry. What action best fits the business and technical boundary?
- Compact extended sessions without discarding commitments
- Allocate input and output token budgets per workflow step
- Retrieve only relevant material instead of full archives
- Persist operational decisions outside transient chat history
Explanation
Correct Answer
D:Store approvals and workflow state in an authoritative durable system; treat chat history as context, not a transaction ledger; durable-state testing proves behavior survives context eviction; evaluate the design with correct recovery of approvals and pending actions after forced session reset.
Incorrect Answers
A:Store a structured, reviewable summary of decisions, open issues, and provenance; inject it at checkpoints rather than replaying every chat turn; blind summarization can lose critical exceptions unless durable commitments are explicitly represented; this alternative would measure retention of signed-off requirements after long simulated conversations; it does not establish correct recovery of approvals and pending actions after forced session reset for the described use case.
B:Reserve output capacity and bound each stage's retrieved context and response length according to the deliverable structure; merely expanding every context window can increase cost while leaving output truncation unaddressed; this alternative would measure complete mandatory report sections under the worst supported input size; it does not establish correct recovery of approvals and pending actions after forced session reset for the described use case.
C:Apply query-driven retrieval and evidence selection before prompting Claude, retaining citations and permission filters; broad context stuffing can degrade relevance and cost even when a large context window is available; this alternative would measure answer support and retrieved-token volume on a sampled policy-question set; it does not establish correct recovery of approvals and pending actions after forced session reset for the described use case.
Question 634
A deletion incident needs attribution to the requesting user, yet the audit log records only the shared assistant service account. Which logging change would restore accountability?
- Check every resource identifier against authenticated tenant ownership
- Enforce authorization inside the tool backend
- Record the user, tenant, and action in audit trails
- Use delegated user identity for per-user data access
Explanation
Correct Answer
C:Propagate authenticated actor, tenant, delegated scope, request ID, and action outcome into tamper-resistant audit records; end-to-end attribution makes unauthorized use investigable rather than anonymous; verify the repair using audit reconstruction of actor, authority, target, and result for sampled mutations.
Incorrect Answers
A:Syntactically valid IDs and a logged-in session do not establish ownership of a resource; check the requested resource against the authenticated tenant and role at the data service rather than trusting user-provided identifiers; that control is assessed by denial of cross-tenant identifier enumeration attempts in direct tool tests; the incident instead requires audit reconstruction of actor, authority, target, and result for sampled mutations.
B:Prompt-only restrictions can be bypassed and are not an independently audited authorization control; implement enforceable role, amount, and resource checks at the action service, not just in natural-language instructions; that control is assessed by rejection of unauthorized tool calls even when sent outside the model workflow; the incident instead requires audit reconstruction of actor, authority, target, and result for sampled mutations.
D:Application-level authentication alone does not prove permission to read every record returned by a shared account; carry an appropriate user-bound authorization context to downstream services and enforce record-level access per request; that control is assessed by zero cross-user CRM record disclosures in permission-matrix tests; the incident instead requires audit reconstruction of actor, authority, target, and result for sampled mutations.
Question 635
A connector deployment adds a verbose tool response that is repeated in every prompt. What action best fits the business and technical boundary?
- Associate human corrections with exact release versions
- Instrument end-to-end execution spans for causal debugging
- Alert on unexpected token and cost anomalies
- Detect distribution and quality drift in ongoing samples
- Compare production metrics by model and prompt version
- Track safety policy events across product updates
Explanation
Correct Answer
C:Monitor input/output tokens, cache reads, per-stage costs, and cost-per-task anomalies with accountable ownership; unit-level telemetry catches runaway expense before invoices provide delayed evidence; evaluate the design with detection lag for injected per-request token inflation.
Incorrect Answers
A:Log consented feedback alongside prompt revision, model route, source version, and case context for reproducible analysis; aggregate thumbs-down counts without version linkage cannot identify regressions or prove a fix; this alternative would measure reproduction of sampled reviewer defects against their original configuration; it does not establish detection lag for injected per-request token inflation for the described use case.
B:Correlation and span IDs across components allow latency and failure attribution along the actual path; model usage totals do not reveal which integration failed within a particular request; this alternative would measure percent of failures with a complete trace to the responsible stage; it does not establish detection lag for injected per-request token inflation for the described use case.
D:Periodically sample representative production outputs for labeling, segment analysis, and drift signals without retaining unnecessary personal data; a static validation set cannot reflect every change in real input distribution over time; this alternative would measure quality delta between recent audited production samples and the launch baseline; it does not establish detection lag for injected per-request token inflation for the described use case.
E:Partition traces and scored outcomes by deployed configuration, using release identifiers to support rollback and diagnosis; a global metric is inadequate when several behaviors are concurrently deployed; this alternative would measure detection of version-specific quality regressions in a staged rollout; it does not establish detection lag for injected per-request token inflation for the described use case.
F:Monitor both harmful-output violations and false-positive blocks by version and user segment; a falling count of harmful completions can coexist with an unusable overrestrictive assistant; this alternative would measure safety precision/recall and legitimate-task completion before and after the change; it does not establish detection lag for injected per-request token inflation for the described use case.
Question 636
A client project defers privacy and tool-permission review until after a working pilot is complete. What should the solution architect recommend?
- Run a recurring monitoring-to-improvement loop
- Plan deprecation and rollback from the start
- Use bounded pilots with exit criteria before expansion
- Include security and governance checks during design
Explanation
Correct Answer
D:Incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens; early controls reduce late redesign and exposure; evaluate the design with resolution of high-severity architecture risks before implementation approval.
Incorrect Answers
A:Feed production samples and feedback into versioned evals, prioritize changes, and release improvements behind regression gates; a one-time launch test cannot guarantee reliability under changing input distributions; this alternative would measure cycle time from detected drift to validated improvement without safety regression; it does not establish resolution of high-severity architecture risks before implementation approval for the described use case.
B:Inventory consumers, define migration dates and fallback, preserve data obligations, and test the rollback or retirement path; removing an old endpoint alone does not manage hidden dependencies or operational continuity; this alternative would measure successful retirement with zero unplanned dependent-system failures; it does not establish resolution of high-severity architecture risks before implementation approval for the described use case.
C:Define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; more launch enthusiasm does not replace production-like testing under load and edge conditions; this alternative would measure pilot gate pass rate with defined performance and safety thresholds; it does not establish resolution of high-severity architecture risks before implementation approval for the described use case.
Question 637
The model's outputs mirror historical exemplars despite a corrected class definition in a later section. Which action best removes the underlying failure mode?
- Revalidate few-shot examples after policy changes
- Diagnose third-party tool failures separately from model decisions
- Investigate retrieval recall before changing the language model
- Test for model capability mismatch on complex reasoning
Explanation
Correct Answer
A:Audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; a targeted regression set proves the prompt no longer anchors on obsolete examples; verify the repair using correct labels on scenarios affected by the policy revision.
Incorrect Answers
B:Prompt changes alone cannot restore a failing service or make an empty placeholder a valid business result; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; that control is assessed by correct 'unknown' handling and no false denials during injected API failures; the incident instead requires correct labels on scenarios affected by the policy revision.
C:Model upgrades cannot recover information that never reaches the answer context; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; that control is assessed by recall of newly published policy passages and removal of stale hits; the incident instead requires correct labels on scenarios affected by the policy revision.
D:Rebuilding retrieval is unlikely to help when required evidence was already consistently supplied; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; that control is assessed by correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the incident instead requires correct labels on scenarios affected by the policy revision.
Question 638
A meeting-summary agent sees payroll updates, deletion endpoints, and scheduling tools alongside transcript retrieval. What is the most defensible next step?
- Offer each agent only the tools required for its role
- Audit tool manifests as versioned production configuration
- Separate read-only retrieval from mutating tools
- Remove duplicate and deprecated tools from the catalog
Explanation
Correct Answer
A:Give the agent a least-capability tool set mapped to the exact tasks it may perform, reducing accidental and adversarial action paths; capability inventory and adversarial execution tests verify that unnecessary actions are unreachable; evaluate the design with unauthorized tool availability and invocation count in hostile transcript tests.
Incorrect Answers
B:Maintain owner-approved manifests with risk classification, contract checks, permission reviews, and change-control gates; ad hoc tool registration lacks the inventory needed to catch capability expansion before deployment; this alternative would measure changes rejected when requested tool scopes exceed the approved task manifest; it does not establish unauthorized tool availability and invocation count in hostile transcript tests for the described use case.
C:Expose narrowly scoped read-only capabilities and gate changes through separate authenticated write paths with explicit approvals; a generic read-write interface offers more authority than the reporting workflow requires; this alternative would measure zero mutation permissions for report-only identities across tool contract checks; it does not establish unauthorized tool availability and invocation count in hostile transcript tests for the described use case.
D:Consolidate overlapping operations behind a maintained interface and retire obsolete tool definitions after migration testing; more tool variants do not add genuine capability when they expose the same action with conflicting contracts; this alternative would measure single correct connector selection and consistent ticket states across repeated requests; it does not establish unauthorized tool availability and invocation count in hostile transcript tests for the described use case.
Question 639
A research analyst notices the assistant supplying precise percentages absent from its cited material. In a fraud service, external enrichment fields that look like approvals must not acquire internal decision authority. Which TWO verification measures address unsupported claims and untrusted tool-result semantics? Choose TWO.
- Validate tool-result provenance and semantics
- Mitigate automation bias with visible uncertainty and review
- Constrain outbound channels to prevent data exfiltration
- Threat-model prompt injection across lower-trust inputs
- Use claim verification to mitigate confident fabrication
Explanation
Correct Answers
A:Preserve source identity and trust level, validate schemas, and separate observed facts from authoritative approval signals; provenance-sensitive checks stop evidence laundering through connector outputs; its corresponding workstream should be validated by zero external-result fields promoted to internal approvals in test cases.
E:Require support from retrieved source passages and provide an explicit unknown path for unsupported claims; grounding tests detect confident fabrication at the statement level; its corresponding workstream should be validated by proportion of numeric claims supported by independently inspected references.
Incorrect Answers
B:Design review interfaces that expose citations, uncertainty, and targeted challenge checks rather than only an approval button; a nominal human checkpoint is weak when the interface encourages blind confirmation; its natural acceptance measure is reviewer error interception on seeded incorrect recommendations, which does not satisfy the separate workstreams being assessed.
C:Outbound restrictions can prevent data exfiltration, but neither workstream describes an unauthorized transmission; one needs numeric claim verification against sources and the other needs provenance and semantics checks before trusting enrichment fields.
D:Prompt-injection threat modeling defends against hostile instructions in retrieved content, but fabricated numerical claims require reference checking and a third-party fraud result must not be mistaken for internal authority; the question does not present a command-injection attempt.
Question 640
A support organization says the assistant must be 'high quality' but cannot agree whether correctness, tone, or resolution matters most. Which option addresses the immediate design need?
- Map available data and its permitted uses before solution selection
- Translate vague accuracy demands into task-specific criteria
- Measure the current manual task before choosing automation
- Define which decisions remain with people
Explanation
Correct Answer
B:Convert vague requests into observable task outcomes, rubric-based quality dimensions, and failure thresholds that stakeholders can agree to test; consistent scoring makes a go-live decision defensible rather than impressionistic; evaluate the design with agreement between domain reviewers on a rubric and pass rate against a labeled acceptance set.
Incorrect Answers
A:Data-readiness and usage-rights mapping identifies legitimate inputs, owners, quality gaps, and prohibited transfers before committing to an implementation; a data inventory does not alone identify target SLOs or decide whether an orchestration loop is necessary; this alternative would measure percentage of required fields with validated owners, quality checks, and approved processing purposes; it does not establish agreement between domain reviewers on a rubric and pass rate against a labeled acceptance set for the described use case.
C:A baseline separates real business improvement from attractive demos; use completed cases, reviewer effort, errors, and exception handling as measurable comparators; process measurement establishes value, not the identity, data-permission, or runtime architecture of a Claude application; this alternative would measure change in verified minutes and rework per adjudicated claim versus the documented baseline; it does not establish agreement between domain reviewers on a rubric and pass rate against a labeled acceptance set for the described use case.
D:Explicit decision-rights boundaries distinguish advisory generation from actions requiring a credentialed human or deterministic policy engine; decision ownership is an authorization and accountability control; it does not fix missing input records or prompt-token waste; this alternative would measure share of binding decisions showing the required named approver and rationale; it does not establish agreement between domain reviewers on a rubric and pass rate against a labeled acceptance set for the described use case.