Question 260
An enterprise platform wants automatic upgrades as soon as a newer Claude model identifier appears. What should the solution architect recommend?
- Gate model-version updates on regression tests
- Select a model path compatible with required modalities
- Use stronger reasoning capacity for genuinely complex decisions
- Evaluate model choice against the real input distribution
Explanation
Correct Answer
A:Treat model replacements as versioned releases with task-specific eval gates, contract tests, canaries, and rollback capability; version-aware tests make improvement claims meaningful for the deployed application; evaluate the design with no contract regressions and acceptable quality/latency deltas on the proposed release.
Incorrect Answers
B:Confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; model selection based solely on text benchmark accuracy does not prove image interpretation reliability; this alternative would measure correct equipment identification and recommendation accuracy on labeled photographs; it does not establish no contract regressions and acceptable quality/latency deltas on the proposed release for the described use case.
C:Escalate tasks with demonstrated reasoning difficulty to a capable model after measuring gains on cases involving cross-document dependencies; high reasoning capacity is wasteful if the task only extracts a known field or matches a stable label; this alternative would measure correct resolution of cross-exhibit conflicts at the required latency and review cost; it does not establish no contract regressions and acceptable quality/latency deltas on the proposed release for the described use case.
D:Representative workload evaluation must include input quality, document lengths, languages, and failure consequences before setting model policy; vendor benchmark rankings do not establish suitability for a domain-specific and skewed workload; this alternative would measure per-segment acceptance rate across language, length, and noisy-input strata; it does not establish no contract regressions and acceptable quality/latency deltas on the proposed release for the described use case.
Question 261
Two engineering teams advocate different Claude configurations because one prioritizes quality and the other prioritizes cost. Which decision record would demonstrate agreement against explicit, comparable criteria?
- Stakeholder-approved selection using common scored evidence and disclosed trade-offs
- Sponsor comprehension of expected outcomes, decision options, and risk ownership
- Reconstructability of key design decisions after a simulated staff handoff
- Stakeholder identification of boundaries and fallback responsibilities after the demo
- Completion and ownership of high-severity mitigations before the release gate
Explanation
Correct Answer
A:A shared rubric exposes value conflicts instead of disguising them as technical disputes; measure stakeholder-approved selection using common scored evidence and disclosed trade-offs at the release gate; present the same quality, latency, cost, risk, and maintainability measures for each candidate under agreed weighting.
Incorrect Answers
B:Decision-focused review checks whether the audience can act on the information; summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; this tests sponsor comprehension of expected outcomes, decision options, and risk ownership; the actual release condition is stakeholder-approved selection using common scored evidence and disclosed trade-offs.
C:Decision records preserve rationale beyond a particular architect's memory; use an ADR recording context, considered options, decision, trade-offs, implications, and conditions for revisiting it; this tests reconstructability of key design decisions after a simulated staff handoff; the actual release condition is stakeholder-approved selection using common scored evidence and disclosed trade-offs.
D:Understanding checks reveal whether decision-makers grasp operational limits; use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; this tests stakeholder identification of boundaries and fallback responsibilities after the demo; the actual release condition is stakeholder-approved selection using common scored evidence and disclosed trade-offs.
E:A control register makes risk decisions reviewable rather than rhetorical; record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; this tests completion and ownership of high-severity mitigations before the release gate; the actual release condition is stakeholder-approved selection using common scored evidence and disclosed trade-offs.
Question 262
A quoted vendor note is mistaken for an instruction about the evaluator's scoring rubric. Which action best removes the underlying failure mode?
- Separate instructions and source regions with clear tags
- Add few-shot examples for difficult boundary cases
- Use contrastive examples when two outputs look similar
- Break multi-step reasoning into checkable intermediate outputs
- Chain prompts when one result is a prerequisite for another
Explanation
Correct Answer
A:Use clearly named sections or XML-style tags so Claude can distinguish task policy, input, and supporting documents; the acceptance set targets role confusion caused by mixed prompt sections; verify the repair using correct separation of document claims and application instructions in adversarial input tests.
Incorrect Answers
B:Examples are not a substitute for a clean class definition or representative test labels; provide a few diverse, accurately labeled counterexamples that illustrate the exact class boundary, then verify against held-out cases; that control is assessed by recall on previously confused boundary examples outside the prompt set; the incident instead requires correct separation of document claims and application instructions in adversarial input tests.
C:Adding examples without highlighting the differentiating condition can reinforce superficial formatting only; show a paired positive and negative example with the decision rule that separates the formats; then test unseen cases; that control is assessed by correct type selection on unseen evidence-versus-action pairs; the incident instead requires correct separation of document claims and application instructions in adversarial input tests.
D:Requesting longer explanations without validation can make unsupported conclusions appear more persuasive; require explicit intermediate evidence and structured rule checks that can be independently validated; avoid treating verbose reasoning text as proof; that control is assessed by intermediate artifact validity and final score agreement with a policy-engine oracle; the incident instead requires correct separation of document claims and application instructions in adversarial input tests.
E:Chaining every trivial substep increases latency without improving reliability for simple independent tasks; use a staged prompt chain where a verified intermediate classification is required before the next generation step; that control is assessed by explanations generated only for records that passed the eligibility stage; the incident instead requires correct separation of document claims and application instructions in adversarial input tests.
Question 263
A mobile technician captures blurry photographs of serial labels and adds written fault descriptions. What acceptance evidence would establish that the deployed Claude route uses both modalities accurately?
- Macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic
- High-risk escalation recall and total cost under the required error ceiling
- Per-segment acceptance rate across language, length, and noisy-input strata
- Correct equipment identification and recommendation accuracy on labeled photographs
Explanation
Correct Answer
D:A multimodal acceptance set measures the capability absent from text-only trials; measure correct equipment identification and recommendation accuracy on labeled photographs at the release gate; confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together.
Incorrect Answers
A:Joint quality and price measurement prevents choosing a cheap model that fails costly minority categories; compare an efficient Claude model against labeled classification cases and select the lowest-cost model that meets the defined accuracy floor; this tests macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic; the actual release condition is correct equipment identification and recommendation accuracy on labeled photographs.
B:Routing is valuable only if it captures difficult cases without needlessly escalating every simple request; combine an evaluated uncertainty trigger with escalation to a stronger model or reviewer, using explicit limits and post-route quality measurement; this tests high-risk escalation recall and total cost under the required error ceiling; the actual release condition is correct equipment identification and recommendation accuracy on labeled photographs.
C:Stratified scores expose failures that an overall aggregate conceals; representative workload evaluation must include input quality, document lengths, languages, and failure consequences before setting model policy; this tests per-segment acceptance rate across language, length, and noisy-input strata; the actual release condition is correct equipment identification and recommendation accuracy on labeled photographs.
Question 264
A product team needs web search integrated into Claude interactions without operating its own search-execution service. Which approach best meets the stated requirement?
- Use agent-to-agent exchange for independently owned specialist systems
- Use event notifications to drive state changes rather than constant polling
- Prefer managed server-side tools when appropriate to the platform
- Use a controlled CLI interface for developer automation
- Use MCP to expose reusable tools and resources across compatible clients
Explanation
Correct Answer
C:A supported server-executed tool can offload execution to the provider while the application retains usage, privacy, and result-validation responsibilities; managed execution should reduce infrastructure burden without weakening answer grounding; evaluate the design with search coverage, tool cost, and provenance checks with no application-owned execution endpoint.
Incorrect Answers
A:An agent-to-agent contract can exchange bounded tasks and outputs while preserving local autonomy, identity, and authority of each specialist system; messaging between agents does not make their outputs trustworthy or authorize cross-system actions automatically; this alternative would measure successful authenticated task exchange with each agent retaining its own permission checks; it does not establish search coverage, tool cost, and provenance checks with no application-owned execution endpoint for the described use case.
B:An authenticated event or webhook integration can trigger bounded processing only when upstream state changes; events still require deduplication, retries, and authorization checks to avoid duplicate downstream effects; this alternative would measure event-to-index update delay and duplicate-event idempotency under replay; it does not establish search coverage, tool cost, and provenance checks with no application-owned execution endpoint for the described use case.
D:A bounded CLI execution environment can reuse standard developer tools with explicit directory and command permissions; general CLI execution is dangerous when exposed to untrusted instructions without sandboxing and command policy; this alternative would measure reproducible build results with denied commands outside the approved allowlist; it does not establish search coverage, tool cost, and provenance checks with no application-owned execution endpoint for the described use case.
E:A maintained MCP server can standardize discoverable tool and resource contracts while each client still enforces authentication and trust; mCP is an interface protocol, not a substitute for access policy or secure server implementation; this alternative would measure cross-client contract compatibility and unauthorized-action denial for the same MCP service; it does not establish search coverage, tool cost, and provenance checks with no application-owned execution endpoint for the described use case.
Question 265
A manufacturing copilot depends on an external ERP provider. The pilot must survive documented usage quotas and scheduled downtime before the organization signs off. Which result provides the strongest dependency-readiness evidence?
- Percentage of required sources with approved owners, legal uses, and integration paths
- Resolution rate of prioritized unknowns through measured prototypes
- Agreed p95, availability, and load thresholds exercised in an acceptance environment
- Verified readiness of high-impact external dependencies against test quotas and outage cases
Explanation
Correct Answer
D:Dependency validation closes gaps between prototype success and operational feasibility; measure verified readiness of high-impact external dependencies against test quotas and outage cases at the release gate; maintain a dependency register covering third-party SLAs, quotas, outage paths, and contractual conditions with mitigation owners.
Incorrect Answers
A:Data ownership completeness measures feasibility before engineering spends effort; create a source inventory with accountable owners, sensitivity, update rate, and approval dependencies before committing to the design; this tests percentage of required sources with approved owners, legal uses, and integration paths; the actual release condition is verified readiness of high-impact external dependencies against test quotas and outage cases.
B:Evidence-based discovery reduces commitment to unverified technical assumptions; identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; this tests resolution rate of prioritized unknowns through measured prototypes; the actual release condition is verified readiness of high-impact external dependencies against test quotas and outage cases.
C:Measurable NFRs turn vague expectations into testable architecture constraints; specify capacity, tail latency, availability, privacy, and recovery objectives with measurement windows and ownership; this tests agreed p95, availability, and load thresholds exercised in an acceptance environment; the actual release condition is verified readiness of high-impact external dependencies against test quotas and outage cases.
Question 266
A support assistant can resolve common questions quickly but ambiguous billing disputes require additional reasoning or a person. What should the solution architect recommend?
- Gate model-version updates on regression tests
- Benchmark a smaller model for routine classification
- Select a model path compatible with required modalities
- Use confidence-aware routing with a bounded fallback
Explanation
Correct Answer
D:Combine an evaluated uncertainty trigger with escalation to a stronger model or reviewer, using explicit limits and post-route quality measurement; routing is valuable only if it captures difficult cases without needlessly escalating every simple request; evaluate the design with high-risk escalation recall and total cost under the required error ceiling.
Incorrect Answers
A:Treat model replacements as versioned releases with task-specific eval gates, contract tests, canaries, and rollback capability; newer model branding alone cannot ensure identical tool-use or output behavior in an existing workflow; this alternative would measure no contract regressions and acceptable quality/latency deltas on the proposed release; it does not establish high-risk escalation recall and total cost under the required error ceiling for the described use case.
B:Compare an efficient Claude model against labeled classification cases and select the lowest-cost model that meets the defined accuracy floor; a lightweight route should not be assumed suitable for complex multi-step decisions without a representative evaluation; this alternative would measure macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic; it does not establish high-risk escalation recall and total cost under the required error ceiling for the described use case.
C:Confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; model selection based solely on text benchmark accuracy does not prove image interpretation reliability; this alternative would measure correct equipment identification and recommendation accuracy on labeled photographs; it does not establish high-risk escalation recall and total cost under the required error ceiling for the described use case.
Question 267
A sponsor sees a polished prototype that answers typical requests without showing missing-data or outage behavior. What is the most targeted architecture choice?
- Demonstrate known limitations alongside successful workflows
- Present executive decisions through outcomes, cost, and risk
- Maintain a risk register with controls and accountable owners
- Record alternatives and rationale in an architecture decision record
Explanation
Correct Answer
A:Use representative happy-path and failure-path demonstrations with explicit fallback, uncertainty, and human ownership; understanding checks reveal whether decision-makers grasp operational limits; evaluate the design with stakeholder identification of boundaries and fallback responsibilities after the demo.
Incorrect Answers
B:Summarize business value, alternatives, cost envelope, measured evidence, and requested decisions in concise nontechnical terms; a detailed API design is valuable to implementers but not sufficient for a funding decision; this alternative would measure sponsor comprehension of expected outcomes, decision options, and risk ownership; it does not establish stakeholder identification of boundaries and fallback responsibilities after the demo for the described use case.
C:Record probability and impact where appropriate, control owners, evidence, residual risk, and required approvals; a high-level statement that AI has risks does not create an actionable governance plan; this alternative would measure completion and ownership of high-severity mitigations before the release gate; it does not establish stakeholder identification of boundaries and fallback responsibilities after the demo for the described use case.
D:Use an ADR recording context, considered options, decision, trade-offs, implications, and conditions for revisiting it; a final diagram alone often omits rejected alternatives and the reasoning behind the choice; this alternative would measure reconstructability of key design decisions after a simulated staff handoff; it does not establish stakeholder identification of boundaries and fallback responsibilities after the demo for the described use case.
Question 268
A lightweight route fails on multi-document conflict resolution while retrieval contains all correct passages. Which strategy should be prioritized for this requirement?
- Revalidate few-shot examples after policy changes
- Confirm whether claims are supported by supplied sources
- Separate external tool faults from model decisions
- Test for model capability mismatch on complex reasoning
Explanation
Correct Answer
D:Hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; evaluate the design with correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations.
Incorrect Answers
A:Audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; index re-embedding cannot resolve a stale rule embedded directly in the prompt template; this alternative would measure correct labels on scenarios affected by the policy revision; it does not establish correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations for the described use case.
B:Compare material claims against cited evidence and flag unsupported statements for abstention or review; a well-formatted citation is not proof that the cited text supports a particular assertion; this alternative would measure supported-claim rate on source-checked summaries with seeded unsupported numbers; it does not establish correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations for the described use case.
C:Propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; prompt changes alone cannot restore a failing service or make an empty placeholder a valid business result; this alternative would measure correct 'unknown' handling and no false denials during injected API failures; it does not establish correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations for the described use case.
Question 269
A retail assistant works across search, inference, and inventory tools deployed by different teams. Which option addresses the immediate design need?
- Segment reliability metrics by tenant and workload
- Log structured tool outcomes without exposing sensitive payloads
- Correlate model, retrieval, and tool spans in distributed traces
- Alert on unusual high-impact tool activity
- Sample complete traces strategically under heavy traffic
Explanation
Correct Answer
C:End-to-end traces with consistent request and span identifiers identify which service and step consumed time; distributed spans expose ownership boundaries across the whole execution path; evaluate the design with trace completeness and correct bottleneck attribution on instrumented fault injections.
Incorrect Answers
A:Break down latency, errors, and tool success by permission-safe tenant and workload dimensions with cardinality controls; overall averages can hide concentrated failures without meaningful dimensional analysis; this alternative would measure detection time for an injected single-tenant degradation; it does not establish trace completeness and correct bottleneck attribution on instrumented fault injections for the described use case.
B:Record tool type, status, duration, correlation IDs, and carefully redacted diagnostic fields under controlled retention; full payload capture may help debugging but expands sensitive-data exposure and retention obligations; this alternative would measure error diagnosability with seeded sensitive fields absent from collected logs; it does not establish trace completeness and correct bottleneck attribution on instrumented fault injections for the described use case.
D:Monitor write volume, denials, unusual resource patterns, and action-specific thresholds with fast containment routing; general chat satisfaction metrics are too delayed and indirect to detect dangerous mutation spikes; this alternative would measure time to detect and stop a simulated unauthorized bulk-write burst; it does not establish trace completeness and correct bottleneck attribution on instrumented fault injections for the described use case.
E:Retain error and high-risk traces at a higher rate while sampling successful routine traffic with privacy controls; keeping every raw trace is costly and may violate retention principles without improving detection proportionally; this alternative would measure capture rate of seeded critical failures at a bounded telemetry cost; it does not establish trace completeness and correct bottleneck attribution on instrumented fault injections for the described use case.
Question 270
An AI code review integration experiences recurring outages after untested connector changes. Which design decision directly addresses this constraint?
- Reproduce incidents using correlated logs and configuration versions
- Replay failures safely with sanitized inputs
- Roll back to a known-good tool and prompt release
- Produce incident learning actions with accountable follow-up
Explanation
Correct Answer
D:Run blameless root-cause analysis, link corrective actions to owners, and add regression or operational safeguards before closing the incident; follow-up measures show whether operational learning survives the immediate repair; evaluate the design with recurrence rate of the diagnosed failure and completion of preventive actions.
Incorrect Answers
A:Collect privacy-safe traces, request IDs, environment details, and exact configuration revisions to establish a reproducible failure case; a guessed prompt rewrite is unlikely to repair a fault that has not been isolated; this alternative would measure reproduction rate of sampled incidents from stored sanitized diagnostic metadata; it does not establish recurrence rate of the diagnosed failure and completion of preventive actions for the described use case.
B:Build redacted, access-controlled replay fixtures preserving relevant syntax and error conditions without unnecessary secrets or personal data; raw transcript reuse can expose confidential inputs and violate data-handling policies; this alternative would measure bug reproduction on sanitized fixtures with no sensitive token disclosure; it does not establish recurrence rate of the diagnosed failure and completion of preventive actions for the described use case.
C:Maintain versioned releases and documented rollback steps for hooks, prompts, model routes, and connector configuration; repeated untracked hotfixes can compound an outage and obscure the initial trigger; this alternative would measure time to restore normal operation using an approved previous release; it does not establish recurrence rate of the diagnosed failure and completion of preventive actions for the described use case.