Question 20
A distributed team uses Claude to generate code on different laptops and sees inconsistent local test results. Which approach best meets the stated requirement?
- Run reproducible CI gates on assisted code
- Generate documentation from verified source and review it
- Use test-first acceptance cases for generated changes
- Break large refactors into independently reviewable changes
Explanation
Correct Answer
A:Execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; independent automation prevents environmental drift from masquerading as code correctness; evaluate the design with cI pass reproducibility on clean builds with locked dependencies.
Incorrect Answers
B:Ground documentation generation in current code, schemas, and tests, then validate examples and have owners review changes; fluent prose generated from old context is not a reliable record of a current API contract; this alternative would measure documented endpoint examples executing successfully against the reviewed build; it does not establish cI pass reproducibility on clean builds with locked dependencies for the described use case.
C:Define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; merely reading generated code for plausibility does not replace tests of business invariants; this alternative would measure passing unit and property tests including specified rounding boundaries; it does not establish cI pass reproducibility on clean builds with locked dependencies for the described use case.
D:Use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; one massive patch can reduce initial coordination but increases regression and rollback risk; this alternative would measure review time and defect localization per incremental change set; it does not establish cI pass reproducibility on clean builds with locked dependencies for the described use case.
Question 21
The first recurring model expense spike has no accountable team to investigate it. Which change should the architect recommend first?
- Set an evidence gate before moving beyond discovery
- Use bounded pilots with exit criteria before expansion
- Transfer durable ownership and operations before handoff
- Include security and governance checks during design
Explanation
Correct Answer
C:Establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; ownership evidence shows the service can be maintained after consultants leave; verify the repair using named owner coverage and support-drill completion at handoff.
Incorrect Answers
A:A procurement timeline does not eliminate uncertainty about whether the problem is solvable; require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; that control is assessed by closure of high-risk feasibility questions before design approval; the incident instead requires named owner coverage and support-drill completion at handoff.
B:More launch enthusiasm does not replace production-like testing under load and edge conditions; define pilot size, critical acceptance metrics, escalation paths, and scale-up criteria before broad deployment; that control is assessed by pilot gate pass rate with defined performance and safety thresholds; the incident instead requires named owner coverage and support-drill completion at handoff.
D:Post-build security sign-off is costly and may not undo prior unauthorized processing; incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens; that control is assessed by resolution of high-severity architecture risks before implementation approval; the incident instead requires named owner coverage and support-drill completion at handoff.
Question 22
A code modernization task spans services, schemas, and API client libraries in one enormous generated patch. Which design decision directly addresses this constraint?
- Break large refactors into independently reviewable changes
- Package reusable engineering instructions as governed Skills
- Review AI-generated diffs for logic and security regressions
- Run reproducible CI gates on assisted code
- Use test-first acceptance cases for generated changes
Explanation
Correct Answer
A:Use small scoped changes, compatibility checkpoints, and separate pull requests tied to specific tests; small-batch evidence shows whether modernization remains controlled and reversible; evaluate the design with review time and defect localization per incremental change set.
Incorrect Answers
B:Maintain task-scoped reusable Skills or playbooks with versioning, ownership, and documented prerequisites; shared instructions improve consistency but do not replace enforced approval or automated tests; this alternative would measure checklist completeness across independent sessions using the reviewed Skill version; it does not establish review time and defect localization per incremental change set for the described use case.
C:Require focused code review of the actual diff, threat-relevant tests, and approval before merging sensitive changes; successful compilation does not establish preservation of security controls; this alternative would measure detection of seeded authorization regressions during peer review and CI; it does not establish review time and defect localization per incremental change set for the described use case.
D:Execute standardized linting, type checks, tests, and security scanning in versioned CI environments before merges; local success can be environment-dependent and should not be the only release assurance; this alternative would measure cI pass reproducibility on clean builds with locked dependencies; it does not establish review time and defect localization per incremental change set for the described use case.
E:Define executable tests and expected behavior before relying on AI-generated implementation, then review failures and iterate; merely reading generated code for plausibility does not replace tests of business invariants; this alternative would measure passing unit and property tests including specified rounding boundaries; it does not establish review time and defect localization per incremental change set for the described use case.
Question 23
A warehouse-assistance prototype is moving from a narrow depot trial toward a multi-site rollout. The sponsor wants a gate that exposes difficult exceptions before expansion. Which result should determine whether rollout proceeds?
- Passing the acceptance gate on a representative pilot cohort including difficult exceptions
- Change in verified minutes and rework per adjudicated claim versus the documented baseline
- Share of binding decisions showing the required named approver and rationale
- Percentage of required fields with validated owners, quality checks, and approved processing purposes
Explanation
Correct Answer
A:A bounded deployment can prove business fitness before a large enterprise-wide dependency is created; measure passing the acceptance gate on a representative pilot cohort including difficult exceptions at the release gate; a narrow pilot validates one representative flow and pre-agreed quality, latency, and escalation thresholds before scaling to other workflows.
Incorrect Answers
B:Comparing complete case outcomes includes downstream corrections rather than rewarding only faster draft generation; a baseline separates real business improvement from attractive demos; use completed cases, reviewer effort, errors, and exception handling as measurable comparators; this tests change in verified minutes and rework per adjudicated claim versus the documented baseline; the actual release condition is passing the acceptance gate on a representative pilot cohort including difficult exceptions.
C:An auditable decision trail demonstrates that AI outputs remained advisory where policy requires it; explicit decision-rights boundaries distinguish advisory generation from actions requiring a credentialed human or deterministic policy engine; this tests share of binding decisions showing the required named approver and rationale; the actual release condition is passing the acceptance gate on a representative pilot cohort including difficult exceptions.
D:Measured coverage of eligible sources exposes blockers that a high demo score conceals; data-readiness and usage-rights mapping identifies legitimate inputs, owners, quality gaps, and prohibited transfers before committing to an implementation; this tests percentage of required fields with validated owners, quality checks, and approved processing purposes; the actual release condition is passing the acceptance gate on a representative pilot cohort including difficult exceptions.
Question 24
A data protection review traces sensitive records through document ingestion, retrieval, inference, and ticket updates. Which outcome would demonstrate that the architecture documentation identifies every material trust boundary?
- Completeness of sensitive-data flow paths identified during security review
- Reproducibility of a sampled historical answer using its full manifest
- Consumer contract pass rate under timeouts, invalid arguments, and permission denials
- Ability to identify affected decisions after simulated policy and workload changes
- Successful completion of standard support drills by staff who did not build the system
Explanation
Correct Answer
A:Architecture diagrams are useful when they expose where protection and authority change; measure completeness of sensitive-data flow paths identified during security review at the release gate; a current diagram should show data classes, flows, processing boundaries, identities, trust zones, and side effects.
Incorrect Answers
B:Versioned provenance enables defensible debugging and rollback; maintain a release manifest linking model configuration, prompt, tool contracts, data versions, and evaluation results; this tests reproducibility of a sampled historical answer using its full manifest; the actual release condition is completeness of sensitive-data flow paths identified during security review.
C:Contract tests validate the documented behavior under non-happy paths; define typed inputs, outputs, error semantics, idempotency, required scopes, and audit obligations for tool boundaries; this tests consumer contract pass rate under timeouts, invalid arguments, and permission denials; the actual release condition is completeness of sensitive-data flow paths identified during security review.
D:Assumption-aware ADRs support controlled adaptation rather than accidental drift; record architectural context, alternatives, decision rationale, constraints, and explicit conditions for revisiting choices; this tests ability to identify affected decisions after simulated policy and workload changes; the actual release condition is completeness of sensitive-data flow paths identified during security review.
E:Independent operator readiness is stronger evidence than architect-led demonstrations; provide system overview, access setup, troubleshooting flowcharts, monitoring thresholds, and named support responsibilities; this tests successful completion of standard support drills by staff who did not build the system; the actual release condition is completeness of sensitive-data flow paths identified during security review.
Question 25
A product team runs an online quality experiment with a planned sample size and decision boundary. Which evidence would show that it did not call a win from random early fluctuations?
- Canary threshold compliance and time to revert a seeded regression
- Balanced treatment cohorts and clean user-level exposure histories
- Pre-registered experiment compliance and interval estimates at the planned sample size
- Time from triggered guardrail breach to verified rollback
Explanation
Correct Answer
C:Sound statistical procedure distinguishes stable benefit from random variance; measure pre-registered experiment compliance and interval estimates at the planned sample size at the release gate; define primary metrics, minimal meaningful effect, sample-size method, and stopping rules before examining results.
Incorrect Answers
A:Controlled exposure and rollback readiness limit the damage of an unexpected release; deploy to a bounded cohort while observing quality, errors, and tail latency, with predefined stop and rollback thresholds; this tests canary threshold compliance and time to revert a seeded regression; the actual release condition is pre-registered experiment compliance and interval estimates at the planned sample size.
B:Stable assignments preserve causal interpretation for repeated sessions; assign a stable randomized unit aligned to the behavior being measured and maintain consistent treatment during the experiment; this tests balanced treatment cohorts and clean user-level exposure histories; the actual release condition is pre-registered experiment compliance and interval estimates at the planned sample size.
D:Operational readiness includes the ability to stop unsafe experiments promptly; establish automated and human-owned rollback triggers tied to critical error and unauthorized-action indicators; this tests time from triggered guardrail breach to verified rollback; the actual release condition is pre-registered experiment compliance and interval estimates at the planned sample size.
Question 26
The design must be rebuilt because sensitive data crossed an unapproved external boundary. Which corrective change best addresses the root cause?
- Set an evidence gate before moving beyond discovery
- Plan deprecation and rollback from the start
- Include security and governance checks during design
- Transfer durable ownership and operations before handoff
Explanation
Correct Answer
C:Incorporate threat modeling, authorization, data processing, and human oversight into architecture acceptance before implementation hardens; early controls reduce late redesign and exposure; verify the repair using resolution of high-severity architecture risks before implementation approval.
Incorrect Answers
A:A procurement timeline does not eliminate uncertainty about whether the problem is solvable; require validated problem scope, value hypothesis, data authority, and feasibility evidence before full design commitment; that control is assessed by closure of high-risk feasibility questions before design approval; the incident instead requires resolution of high-severity architecture risks before implementation approval.
B:Removing an old endpoint alone does not manage hidden dependencies or operational continuity; inventory consumers, define migration dates and fallback, preserve data obligations, and test the rollback or retirement path; that control is assessed by successful retirement with zero unplanned dependent-system failures; the incident instead requires resolution of high-severity architecture risks before implementation approval.
D:A completed repository is not proof of ongoing service operability; establish production owners for reliability, security, spend, data, prompts, and escalation, with verified access and runbooks; that control is assessed by named owner coverage and support-drill completion at handoff; the incident instead requires resolution of high-severity architecture risks before implementation approval.
Question 27
Repeated model-driven polling increases tool calls while missing brief update windows. Which corrective change best addresses the root cause?
- Use event notifications to drive state changes rather than constant polling
- Use a controlled CLI interface for developer automation
- Use direct APIs for a narrow stable application integration
- Use MCP to expose reusable tools and resources across compatible clients
- Prefer managed server-side tools when appropriate to the platform
Explanation
Correct Answer
A:An authenticated event or webhook integration can trigger bounded processing only when upstream state changes; event-driven tests verify timely updates without wasteful polling loops; verify the repair using event-to-index update delay and duplicate-event idempotency under replay.
Incorrect Answers
B:General CLI execution is dangerous when exposed to untrusted instructions without sandboxing and command policy; a bounded CLI execution environment can reuse standard developer tools with explicit directory and command permissions; that control is assessed by reproducible build results with denied commands outside the approved allowlist; the incident instead requires event-to-index update delay and duplicate-event idempotency under replay.
C:Direct API calls offer less automatic discovery when many heterogeneous tools must be used by several assistants; a direct typed API client is appropriate when the interface, arguments, and control flow are known and stable; that control is assessed by integration error rate and p95 latency on the fixed status-check path; the incident instead requires event-to-index update delay and duplicate-event idempotency under replay.
D:MCP is an interface protocol, not a substitute for access policy or secure server implementation; a maintained MCP server can standardize discoverable tool and resource contracts while each client still enforces authentication and trust; that control is assessed by cross-client contract compatibility and unauthorized-action denial for the same MCP service; the incident instead requires event-to-index update delay and duplicate-event idempotency under replay.
E:Hosted tools do not automatically make external search content authoritative or safe from prompt injection; a supported server-executed tool can offload execution to the provider while the application retains usage, privacy, and result-validation responsibilities; that control is assessed by search coverage, tool cost, and provenance checks with no application-owned execution endpoint; the incident instead requires event-to-index update delay and duplicate-event idempotency under replay.
Question 28
A project assumes that archival PDFs can be reliably parsed and grounded within the cost budget. What is the most targeted architecture choice?
- Record nonfunctional requirements as measurable thresholds
- Map the owner and authority of every required data source
- Obtain sign-off on explicit acceptance criteria and exclusions
- Interview each role that owns a distinct outcome or risk
- Validate risky assumptions through narrow experiments
Explanation
Correct Answer
E:Identify high-risk assumptions and design small feasibility spikes with explicit success and stop criteria; evidence-based discovery reduces commitment to unverified technical assumptions; evaluate the design with resolution rate of prioritized unknowns through measured prototypes.
Incorrect Answers
A:Specify capacity, tail latency, availability, privacy, and recovery objectives with measurement windows and ownership; functional task lists do not tell designers what constitutes acceptable production operation; this alternative would measure agreed p95, availability, and load thresholds exercised in an acceptance environment; it does not establish resolution rate of prioritized unknowns through measured prototypes for the described use case.
B:Create a source inventory with accountable owners, sensitivity, update rate, and approval dependencies before committing to the design; a list of desired features cannot grant access or establish source truthfulness; this alternative would measure percentage of required sources with approved owners, legal uses, and integration paths; it does not establish resolution rate of prioritized unknowns through measured prototypes for the described use case.
C:Document in-scope outcomes, exclusions, metrics, responsibilities, and go-live gates for accountable approval; informal meeting consensus may not survive staff changes or new deployment expectations; this alternative would measure signed acceptance criteria mapped to demonstrable tests and named owners; it does not establish resolution rate of prioritized unknowns through measured prototypes for the described use case.
D:Use role-based discovery to capture goals, current workflows, authority boundaries, and failure impacts from relevant stakeholders; a sponsor's high-level vision does not fully describe operational controls or edge cases; this alternative would measure coverage of approved requirements traced to affected stakeholder groups; it does not establish resolution rate of prioritized unknowns through measured prototypes for the described use case.
Question 29
A scientific literature assistant lists convincing numerical outcomes that are not present in any retrieved study. What should the solution architect recommend?
- Test for model capability mismatch on complex reasoning
- Trace malformed outputs to prompt or schema boundaries
- Separate external tool faults from model decisions
- Revalidate few-shot examples after policy changes
- Verify whether claims are supported by supplied sources
Explanation
Correct Answer
E:Compare material claims against cited evidence and flag unsupported statements for abstention or review; claim-level verification targets hallucinations that ordinary coherence checks miss; evaluate the design with supported-claim rate on source-checked summaries with seeded unsupported numbers.
Incorrect Answers
A:Hold evidence constant and compare validated model capabilities or staged reasoning designs on the failed complexity class; rebuilding retrieval is unlikely to help when required evidence was already consistently supplied; this alternative would measure correct conflict resolution with fixed retrieved evidence across candidate reasoning configurations; it does not establish supported-claim rate on source-checked summaries with seeded unsupported numbers for the described use case.
B:Inspect instruction clarity, output-schema validation, and edge-case examples before assuming a core model reasoning failure; switching to a more expensive model may leave the consumer's missing validation problem intact; this alternative would measure schema error rate across multiline, null, and unusual-field edge cases; it does not establish supported-claim rate on source-checked summaries with seeded unsupported numbers for the described use case.
C:Propagate typed tool errors, retries, and unknown states instead of converting transport failures into negative facts; prompt changes alone cannot restore a failing service or make an empty placeholder a valid business result; this alternative would measure correct 'unknown' handling and no false denials during injected API failures; it does not establish supported-claim rate on source-checked summaries with seeded unsupported numbers for the described use case.
D:Audit examples and prompt-version precedence when behavior consistently imitates outdated demonstrations; index re-embedding cannot resolve a stale rule embedded directly in the prompt template; this alternative would measure correct labels on scenarios affected by the policy revision; it does not establish supported-claim rate on source-checked summaries with seeded unsupported numbers for the described use case.
Question 30
A test corpus of only friendly documents gives a false sense of safety before hostile uploads are accepted. What design correction resolves the reported failure?
- Use deterministic code checks for objective output contracts
- Measure human annotator consistency before trusting labels
- Use expert rubrics for nuanced qualitative outputs
- Include hostile prompt-injection and no-answer probes
Explanation
Correct Answer
D:Add adversarial retrieved text, invalid queries, ambiguous requests, and missing-evidence cases with expected safe behavior; red-team cases make trust-boundary performance observable; verify the repair using attack success rate and safe handling of intentionally unanswerable prompts.
Incorrect Answers
A:Subjective model grading is unnecessary and less reliable for an objectively checkable parser contract; apply fast programmatic validators for JSON shape, type constraints, allowed values, and deterministic business rules; that control is assessed by zero schema violations accepted by the downstream consumer across generated cases; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.
B:More test volume does not fix unreliable or undefined labels; use calibration sessions, disagreement analysis, adjudication rules, and documented label provenance before relying on the data as ground truth; that control is assessed by inter-annotator agreement and unresolved-label rate after adjudication; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.
C:Pure string comparison is unsuitable when many substantively correct phrasings exist; use a criterion-based expert rubric, optionally supported by calibrated model grading, for semantic accuracy and appropriateness; that control is assessed by expert-rated factual fidelity and calibrated grader agreement on sampled explanations; the incident instead requires attack success rate and safe handling of intentionally unanswerable prompts.