Question 350
The orchestration graph grows while token cost doubles and ordinary cases show unchanged resolution accuracy. Which change should the architect recommend first?
- Bound handoffs, retries, and recursion across agents
- Use a coordinator to route distinct specialist tasks
- Delegate only specialized subtasks that add measurable value
- Run independent specialists in parallel with a merge contract
- Separate independent critique from generation
Explanation
Correct Answer
C:Introduce a specialist only when its independent expertise, tool scope, or parallelism beats a simpler call on representative evaluations; measuring the contribution of each delegate reveals needless orchestration layers; verify the repair using marginal quality improvement per added specialist against its incremental cost and latency.
Incorrect Answers
A:Recursive collaboration is not inherently self-terminating and cannot be made safe only by asking agents to cooperate; enforce maximum depth, handoff count, timeouts, and explicit unresolved-case escalation in the orchestrator; that control is assessed by zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record; the incident instead requires marginal quality improvement per added specialist against its incremental cost and latency.
B:Specialist routing adds handoffs and can be wasteful for one small, homogeneous transformation; a coordinator can dispatch scoped subagents and merge their typed outputs while restricting each specialist's context and tools; that control is assessed by completed reports with correctly scoped evidence and no cross-specialty data leakage; the incident instead requires marginal quality improvement per added specialist against its incremental cost and latency.
D:Parallel fan-out is unsafe when tasks mutate the same state or depend on each other's evolving outputs; parallel fan-out followed by deterministic aggregation can shorten wall-clock time if each branch has its own contract and errors are handled; that control is assessed by p95 end-to-end latency and deterministic merge completeness under one-branch failures; the incident instead requires marginal quality improvement per added specialist against its incremental cost and latency.
E:A critic agent does not repair missing source documents or guarantee that all facts are verifiable; a distinct verifier with a rubric and evidence access can challenge unsupported assertions before an external document is released; that control is assessed by verified claim support rate after independent critique on seeded unsupported drafts; the incident instead requires marginal quality improvement per added specialist against its incremental cost and latency.
Question 351
Several teams share a Claude tool that writes incident comments into a service-management platform. Which choice most directly satisfies the documented goal?
- Maintain an end-to-end data-flow and trust-boundary diagram
- Prepare operator onboarding and support handoff materials
- Specify tool schemas, errors, and authorization contracts
- Publish an incident runbook with containment and rollback steps
Explanation
Correct Answer
C:Define typed inputs, outputs, error semantics, idempotency, required scopes, and audit obligations for tool boundaries; contract tests validate the documented behavior under non-happy paths; evaluate the design with consumer contract pass rate under timeouts, invalid arguments, and permission denials.
Incorrect Answers
A:A current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects; a feature list does not reveal crossing of sensitive trust boundaries; this alternative would measure completeness of sensitive-data flow paths identified during security review; it does not establish consumer contract pass rate under timeouts, invalid arguments, and permission denials for the described use case.
B:Provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; source code alone rarely communicates the operational decisions and permission steps operators need; this alternative would measure successful completion of standard support drills by staff who did not build the system; it does not establish consumer contract pass rate under timeouts, invalid arguments, and permission denials for the described use case.
D:Maintain runbooks for detection, containment, safe fallback, rollback, and communication with named incident roles; a high-level diagram alone is not an executable incident response procedure; this alternative would measure successful containment time in a simulated stale-index incident; it does not establish consumer contract pass rate under timeouts, invalid arguments, and permission denials for the described use case.
Question 352
A service team is preparing to launch an assistant that drafts dispute recommendations rather than making final resolutions. Which artifact would demonstrate that product, engineering, and operations have agreed on accountable release boundaries?
- Signed acceptance criteria mapped to demonstrable tests and named owners
- Resolution rate of prioritized unknowns through measured prototypes
- Verified readiness of high-impact external dependencies against test quotas and outage cases
- Percentage of required sources with approved owners, legal uses, and integration paths
Explanation
Correct Answer
A:An explicit agreement limits scope drift and creates a common release standard; measure signed acceptance criteria mapped to demonstrable tests and named owners at the release gate; document in-scope outcomes, exclusions, metrics, responsibilities, and go-live gates for accountable approval.
Incorrect Answers
B:Evidence-based discovery reduces commitment to unverified technical assumptions; identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; this tests resolution rate of prioritized unknowns through measured prototypes; the actual release condition is signed acceptance criteria mapped to demonstrable tests and named owners.
C:Dependency validation closes gaps between prototype success and operational feasibility; maintain a dependency register covering third-party SLAs, quotas, outage paths, and contractual conditions with mitigation owners; this tests verified readiness of high-impact external dependencies against test quotas and outage cases; the actual release condition is signed acceptance criteria mapped to demonstrable tests and named owners.
D:Data ownership completeness measures feasibility before engineering spends effort; create a source inventory with accountable owners, sensitivity, update rate, and approval dependencies before committing to the design; this tests percentage of required sources with approved owners, legal uses, and integration paths; the actual release condition is signed acceptance criteria mapped to demonstrable tests and named owners.
Question 353
A procurement leader requests monthly proof that the assistant is saving time and staying within risk tolerance. Which approach best meets the stated requirement?
- Communicate release behavior changes before rollout
- Route user feedback into a triaged, owned backlog
- Define SLOs and error budgets for agreed user outcomes
- Set observability and reporting expectations with stakeholders
Explanation
Correct Answer
D:Agree on outcome-aligned service reports, cadence, escalation thresholds, and audiences during solution definition; reporting completeness lets stakeholders govern the system using relevant evidence; evaluate the design with delivery of agreed business and risk indicators on schedule with decision-ready interpretation.
Incorrect Answers
A:Define release notes, behavior-change notices, affected audiences, and support channels proportional to the deployment's impact; quiet deployment may reduce coordination overhead but damages trust when workflows or permissions change; this alternative would measure affected-user awareness and support-ticket volume following staged changes; it does not establish delivery of agreed business and risk indicators on schedule with decision-ready interpretation for the described use case.
B:Classify feedback by severity and root-cause type, assign owners, link trace IDs, and feed validated cases into regression tests; unstructured rating totals cannot by themselves direct a safe corrective action; this alternative would measure time from actionable feedback to triage, owner assignment, and verified fix; it does not establish delivery of agreed business and risk indicators on schedule with decision-ready interpretation for the described use case.
C:Establish measurable user-facing service objectives, reporting windows, and error-budget policies tied to accountable actions; generic uptime claims can conceal slow or unusable answer experiences; this alternative would measure sLO compliance and prescribed actions during a simulated error-budget breach; it does not establish delivery of agreed business and risk indicators on schedule with decision-ready interpretation for the described use case.
Question 354
Success in the lab collapses after rollout on scanned documents and language variants absent from the trial. Which change should the architect recommend first?
- Evaluate model choice against the real input distribution
- Select a model path compatible with required modalities
- Use confidence-aware routing with a bounded fallback
- Benchmark a smaller model for routine classification
Explanation
Correct Answer
A:Representative workload evaluation must include input quality, document lengths, languages, and failure consequences before setting model policy; stratified scores expose failures that an overall aggregate conceals; verify the repair using per-segment acceptance rate across language, length, and noisy-input strata.
Incorrect Answers
B:Model selection based solely on text benchmark accuracy does not prove image interpretation reliability; confirm that the chosen Claude interface supports the required input modality and test image quality, extraction, and reasoning together; that control is assessed by correct equipment identification and recommendation accuracy on labeled photographs; the incident instead requires per-segment acceptance rate across language, length, and noisy-input strata.
C:Blindly using one model tier on all cases neither controls the hardest error modes nor optimizes cost; combine an evaluated uncertainty trigger with escalation to a stronger model or reviewer, using explicit limits and post-route quality measurement; that control is assessed by high-risk escalation recall and total cost under the required error ceiling; the incident instead requires per-segment acceptance rate across language, length, and noisy-input strata.
D:A lightweight route should not be assumed suitable for complex multi-step decisions without a representative evaluation; compare an efficient Claude model against labeled classification cases and select the lowest-cost model that meets the defined accuracy floor; that control is assessed by macro-F1, critical-class recall, and cost per correctly triaged message on held-out traffic; the incident instead requires per-segment acceptance rate across language, length, and noisy-input strata.
Question 355
A regulated pilot is approaching implementation authorization. Before proceeding, which release result would show that high-severity privacy and tool-privilege risks were resolved rather than deferred?
- Pilot gate pass rate with defined performance and safety thresholds
- Resolution of high-severity architecture risks before implementation approval
- Closure of high-risk feasibility questions before design approval
- Cycle time from detected drift to validated improvement without safety regression
- Named owner coverage and support-drill completion at handoff
Explanation
Correct Answer
B:Early controls reduce late redesign and exposure; measure resolution of high-severity architecture risks before implementation approval at the release gate; incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens.
Incorrect Answers
A:A controlled pilot turns rollout into a measurable decision; define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; this tests pilot gate pass rate with defined performance and safety thresholds; the actual release condition is resolution of high-severity architecture risks before implementation approval.
C:Stage gates prevent costly commitment to untested assumptions; require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; this tests closure of high-risk feasibility questions before design approval; the actual release condition is resolution of high-severity architecture risks before implementation approval.
D:Continuous assessment links real usage to controlled change; feed production samples and feedback into versioned evals, prioritize changes, and release improvements behind regression gates; this tests cycle time from detected drift to validated improvement without safety regression; the actual release condition is resolution of high-severity architecture risks before implementation approval.
E:Ownership evidence shows the service can be maintained after consultants leave; establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; this tests named owner coverage and support-drill completion at handoff; the actual release condition is resolution of high-severity architecture risks before implementation approval.
Question 356
A claims classification rule has been updated, but old prompt demonstrations still bias predictions. Which targeted test would verify that the revised definition is applied to affected cases?
- Correct labels on scenarios affected by the policy revision
- Supported-claim rate on source-checked summaries with seeded unsupported numbers
- Schema error rate across multiline, null, and unusual-field edge cases
- Recall of newly published policy passages and removal of stale hits
- Correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations
Explanation
Correct Answer
A:A targeted regression set proves the prompt no longer anchors on obsolete examples; measure correct labels on scenarios affected by the policy revision at the release gate; audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations.
Incorrect Answers
B:Claim-level verification targets hallucinations that ordinary coherence checks miss; compare material claims against cited evidence and flag unsupported statements for abstention or review; this tests supported-claim rate on source-checked summaries with seeded unsupported numbers; the actual release condition is correct labels on scenarios affected by the policy revision.
C:Edge-specific parser tests identify contract failures separately from content comprehension; inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; this tests schema error rate across multiline, null, and unusual-field edge cases; the actual release condition is correct labels on scenarios affected by the policy revision.
D:A retrieval-specific test isolates the component that changed; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; this tests recall of newly published policy passages and removal of stale hits; the actual release condition is correct labels on scenarios affected by the policy revision.
E:Controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; this tests correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the actual release condition is correct labels on scenarios affected by the policy revision.
Question 357
A customer-helpdesk pilot can invoke specialist agents, but architects need evidence that each additional specialist is worth the orchestration overhead. Which comparison should control expansion?
- Completed reports with correctly scoped evidence and no cross-specialty data leakage
- p95 end-to-end latency and deterministic merge completeness under one-branch failures
- Zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record
- Marginal quality improvement per added specialist against its incremental cost and latency
Explanation
Correct Answer
D:Measuring the contribution of each delegate reveals needless orchestration layers; measure marginal quality improvement per added specialist against its incremental cost and latency at the release gate; introduce a specialist only when its independent expertise, tool scope, or parallelism beats a simpler call on representative evaluations.
Incorrect Answers
A:Routing correctness must include scope enforcement, not just readable final reports; a coordinator can dispatch scoped subagents and merge their typed outputs while restricting each specialist's context and tools; this tests completed reports with correctly scoped evidence and no cross-specialty data leakage; the actual release condition is marginal quality improvement per added specialist against its incremental cost and latency.
B:This validates concurrency benefits without silently dropping a missing assessment; parallel fan-out followed by deterministic aggregation can shorten wall-clock time if each branch has its own contract and errors are handled; this tests p95 end-to-end latency and deterministic merge completeness under one-branch failures; the actual release condition is marginal quality improvement per added specialist against its incremental cost and latency.
C:Stressing unresolved cases proves that the system can fail predictably and within budget; enforce maximum depth, handoff count, timeouts, and explicit unresolved-case escalation in the orchestrator; this tests zero unbounded cycles under adversarial unresolved tasks while retaining a useful escalation record; the actual release condition is marginal quality improvement per added specialist against its incremental cost and latency.
Question 358
A fixed invoice workflow requires one deterministic status lookup with tightly controlled error handling. What should the solution architect recommend?
- Use agent-to-agent exchange for independently owned specialist systems
- Prefer managed server-side tools when appropriate to the platform
- Use event notifications to drive state changes rather than constant polling
- Use a controlled CLI interface for developer automation
- Use direct APIs for a narrow stable application integration
Explanation
Correct Answer
E:A direct typed API client is appropriate when the interface, arguments, and control flow are known and stable; a stable-contract benchmark confirms that extra orchestration is unnecessary; evaluate the design with integration error rate and p95 latency on the fixed status-check path.
Incorrect Answers
A:An agent-to-agent contract can exchange bounded tasks and outputs while preserving local autonomy, identity, and authority of each specialist system; messaging between agents does not make their outputs trustworthy or authorize cross-system actions automatically; this alternative would measure successful authenticated task exchange with each agent retaining its own permission checks; it does not establish integration error rate and p95 latency on the fixed status-check path for the described use case.
B:A supported server-executed tool can offload execution to the provider while the application retains usage, privacy, and result-validation responsibilities; hosted tools do not automatically make external search content authoritative or safe from prompt injection; this alternative would measure search coverage, tool cost, and provenance checks with no application-owned execution endpoint; it does not establish integration error rate and p95 latency on the fixed status-check path for the described use case.
C:An authenticated event or webhook integration can trigger bounded processing only when upstream state changes; events still require deduplication, retries, and authorization checks to avoid duplicate downstream effects; this alternative would measure event-to-index update delay and duplicate-event idempotency under replay; it does not establish integration error rate and p95 latency on the fixed status-check path for the described use case.
D:A bounded CLI execution environment can reuse standard developer tools with explicit directory and command permissions; general CLI execution is dangerous when exposed to untrusted instructions without sandboxing and command policy; this alternative would measure reproducible build results with denied commands outside the approved allowlist; it does not establish integration error rate and p95 latency on the fixed status-check path for the described use case.
Question 359
A product manager asks for a 'fast and reliable' assistant without stating peak load, downtime tolerance, or p95 latency. What should the solution architect recommend?
- Interview each role that owns a distinct outcome or risk
- Record nonfunctional requirements as measurable thresholds
- Obtain sign-off on explicit acceptance criteria and exclusions
- Validate risky assumptions through narrow experiments
Explanation
Correct Answer
B:Specify capacity, tail latency, availability, privacy, and recovery objectives with measurement windows and ownership; measurable NFRs turn vague expectations into testable architecture constraints; evaluate the design with agreed p95, availability, and load thresholds exercised in an acceptance environment.
Incorrect Answers
A:Use role-based discovery to capture goals, current workflows, authority boundaries, and failure impacts from relevant stakeholders; a sponsor's high-level vision does not fully describe operational controls or edge cases; this alternative would measure coverage of approved requirements traced to affected stakeholder groups; it does not establish agreed p95, availability, and load thresholds exercised in an acceptance environment for the described use case.
C:Document in-scope outcomes, exclusions, metrics, responsibilities, and go-live gates for accountable approval; informal meeting consensus may not survive staff changes or new deployment expectations; this alternative would measure signed acceptance criteria mapped to demonstrable tests and named owners; it does not establish agreed p95, availability, and load thresholds exercised in an acceptance environment for the described use case.
D:Identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; detailed architecture diagrams do not prove uncertain input quality or model capability; this alternative would measure resolution rate of prioritized unknowns through measured prototypes; it does not establish agreed p95, availability, and load thresholds exercised in an acceptance environment for the described use case.
Question 360
A subscription service reports temporary upstream errors during eligibility checks. The test suite distinguishes missing facts from unavailable services. Which result would prove that technical failures do not become false customer denials?
- Supported-claim rate on source-checked summaries with seeded unsupported numbers
- Recall of newly published policy passages and removal of stale hits
- Correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations
- Correct 'unknown' handling and no false denials during injected API failures
Explanation
Correct Answer
D:Fault injection distinguishes tool availability from genuine absence of data; measure correct 'unknown' handling and no false denials during injected API failures at the release gate; propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts.
Incorrect Answers
A:Claim-level verification targets hallucinations that ordinary coherence checks miss; compare material claims against cited evidence and flag unsupported statements for abstention or review; this tests supported-claim rate on source-checked summaries with seeded unsupported numbers; the actual release condition is correct 'unknown' handling and no false denials during injected API failures.
B:A retrieval-specific test isolates the component that changed; inspect search coverage, index freshness, filters, and chunk retrieval before attributing the regression to Claude's reasoning; this tests recall of newly published policy passages and removal of stale hits; the actual release condition is correct 'unknown' handling and no false denials during injected API failures.
C:Controlled component isolation determines whether the bottleneck is reasoning rather than retrieval; hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; this tests correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; the actual release condition is correct 'unknown' handling and no false denials during injected API failures.