Microsoft AI-103: Prompt Tuning and Reflection in Foundry

Prompt engineering becomes a production discipline when a model’s answer is only one stage in an application. A generated response may need to cite retrieved evidence, satisfy a strict schema, choose the correct tool or stop rather than perform a risky action. Those requirements change how a developer should design instructions, choose generation settings and judge whether a revision is actually better. A more fluent answer can be a worse answer if it drops a required field or invents a fact.

Microsoft’s AI-103 exam explicitly includes tuning generation behavior, prompt engineering, model parameters, reflection, self-critique loops and evaluations. These objectives are easy to underestimate because a playground makes prompt changes look instantaneous. The engineering work is building an experiment that can distinguish genuine improvement from randomness, task leakage or a prettier mistake.

Begin with the application’s acceptance contract

A useful prompt starts with the work a downstream consumer must perform. Imagine a support assistant that reviews a customer’s message and must produce a JSON record with a category, urgency, an evidence citation and a recommended next step. The result will be used by software, not merely read by a person. That means the output fields, allowed values, missing-information behavior and permission boundaries matter at least as much as the tone of the final explanation.

Define those conditions before writing instructions. Which facts may come from the user, which must come from a trusted knowledge source, and which require a live tool lookup? If the model cannot find an authorized source, may it answer from general knowledge or must it decline? Can it create a draft ticket, or can it submit one? A well-specified task keeps those decisions outside ambiguous prose. For high-impact workflows, authorization lives in application logic even if the prompt also describes the policy.

For structured fields, prefer a supported schema or structured-output capability when the chosen deployment offers it, then validate the result in code. Conforming to JSON shape does not prove factual accuracy: a field can have the right type and the wrong value. A robust implementation checks required fields, enumerated values, provenance, business constraints and consequences. The separation is important for AI-103 because prompts are not a replacement for tool contracts or validation.

The strongest acceptance contract includes negative examples. A message with two conflicting account numbers should not produce an invented certainty; an authorization request from an unverified user should not proceed because a model wrote “approved”; a retrieved policy document containing instructions to ignore safeguards should be treated as data, not authority. Those cases establish the boundaries against which prompt revisions can be evaluated.

Write instructions that separate roles, evidence and output

A common failure is packing the goal, background knowledge, formatting rules, tool policy and retrieved text into one large unstructured paragraph. The model then has to infer which sentences are durable instructions and which are task-specific data. Keep the operating contract separate from the user’s immediate request and from retrieved content. Describe the task in direct language, spell out forbidden actions, identify allowed sources and state what to do when information is missing. Avoid a wall of repetitive prohibitions that obscures the actual objective.

Few-shot examples can help when a task depends on an unusual label, a narrow domain convention or a particular decision threshold. A good example shows the input and the desired behavior—including a difficult edge case—not merely the wording of an attractive final answer. Examples should not silently reveal information that belongs only in a separate test set. If the evaluation questions are almost copies of the prompt examples, a high score may reflect memorization of the demonstration rather than reliable handling of new requests.

Grounding solves a different problem from instructions. A prompt that asks for faithful answers cannot supply facts absent from the model’s context. If the application needs current inventory, contract terms or company policy, retrieve approved evidence and associate it with the specific question. The deeper topic of RAG, vector search and grounding matters because a prompt cannot correct an index that excluded the right document or returned an unauthorized one.

Tool descriptions also influence behavior. A model may call the wrong API when two tools have vague names or overlapping purposes. Use distinct capabilities, meaningful parameter descriptions and narrow input contracts, then test both when a tool should be selected and when it should not. The task of managing agent tools and conversation state is related but separate: tool execution, approval records and durable workflow state must remain enforceable outside natural-language instructions.

Choose model parameters with the deployment in mind

Temperature and top-p affect sampling behavior on models that expose those controls. A lower temperature often reduces variation for constrained classifications and extraction tasks; greater variation may help creative ideation where several valid outputs exist. Top-p changes which probability mass the sampler considers. Their effects are not universal, and increasing determinism does not make an unsupported fact true. Microsoft generally recommends adjusting one of these controls at a time when experimenting with supported chat models so the source of a change remains interpretable.

Before building a tuning matrix, verify which parameters the specific model and API support. Some reasoning deployments do not accept ordinary temperature or top-p values; routing layers may ignore these settings when they select a reasoning model. Output-token budgets and response limits matter too. A low budget can truncate a valid structured response, while an unnecessarily large one raises cost and can delay feedback. The right parameter envelope is a model-and-workload decision, not a universal “best temperature” to memorize.

Test one variable while holding the others constant. If changing model family, prompt examples, retrieval query, sampling settings and schema in the same run improves the output, it is impossible to know which change helped. Instead, record the deployment, model version, system instructions, tool schemas, dataset and decoding settings with each evaluation. Compare repeated outputs when variance is important, and measure failure severity rather than only average success.

Notice that the first thing to tune is often not temperature. A schema error may call for a structured-output feature; an unsupported claim may be a retrieval problem; a wrong business action may require tool authorization. Model selection also belongs in this investigation, because the chosen model must be capable of satisfying the contract before prompt refinements can become useful.

Build reflection and self-critique as bounded verification

A self-critique loop asks a model or a separate evaluator to inspect a candidate answer against explicit criteria, then revise the answer if a meaningful defect is found. For a policy-summary app, the first pass might draft a response; a second pass checks whether each claim is supported by an eligible passage, whether the required action is permitted and whether the JSON contract is complete. A revision should be triggered by a specific defect, not by the vague feeling that the wording could be improved.

Reflection is most useful when the reviewer has an independent basis for judgment. A model asking itself “am I correct?” without new evidence may simply restate the same mistake more confidently. Give the verification stage a rubric, expected schema, retrieved source references or deterministic checks. Where appropriate, use a separate evaluator model or a different prompt designed to find counterexamples rather than to continue the original answer. Independent judgment is valuable, but two models can still share the same blind spot.

Consider an employee policy assistant that answers, “Every employee may expense any equipment purchase.” The retrieved policy actually restricts spending by department and requires manager approval. A useful review identifies which precise sentence is unsupported, retrieves the missing section if necessary, and returns a corrected answer that names the limitation. An unhelpful loop merely rewrites the first sentence in a more formal tone. The distinction is whether the second pass checks evidence and rules, rather than polishing surface fluency.

Bound the loop. Set a maximum number of revisions, a latency or cost budget, and a clear exit condition. If the answer still lacks evidence after one or two checks, the correct next step may be a clarification, a fresh retrieval query or human review. Endless self-critique can increase tokens and latency without improving correctness, and an agent that revises a proposed write action repeatedly should never bypass the original approval boundary.

The AI-103 blueprint also refers to chain-of-thought evaluations. In practice, evaluate observable task outcomes, grounded answers, tool traces and explicitly supplied rationales when they are available and appropriate; do not assume private internal reasoning traces are exposed or that a longer explanation proves correctness. The engineering goal is verifiable behavior. A good decision log records the evidence and control used, not speculative access to hidden model cognition.

Evaluate prompt changes with a real test set

A prompt cannot be approved by testing three friendly examples in the playground. Build a dataset covering common work, rare but consequential edge cases, ambiguous language, absent evidence, policy conflicts, tool failures and adversarial input. Separate development examples from a held-out evaluation sample. When possible, include cases taken from sanitized production failures and annotate what an acceptable response must do, rather than demanding one exact sentence.

Choose metrics that correspond to business risk. For an information assistant, measure relevance and groundedness separately. For extraction, measure required-field accuracy and validation failures. For agents, inspect tool selection, argument correctness, tool execution success and whether approval policies were honored. Tracing and evaluation reveal whether a failure occurred before retrieval, during model generation, inside a tool or during workflow authorization; a single composite score would hide those distinctions.

Microsoft Foundry supports evaluations for answer quality and agent behavior, including task completion and groundedness, and newer optimizer workflows can propose candidate instructions or tool descriptions against a dataset. Such tools accelerate experimentation; they do not justify automatically promoting the candidate with the most attractive average score. Review the evaluator’s own limitations, test safety regressions, and compare with a fixed baseline. A prompt that raises average task completion while causing one unauthorized tool write should fail a risk-sensitive release gate.

Run a paired comparison. The baseline and candidate should see the same representative requests, eligible evidence and tool environment. Record both wins and regressions, including which cases newly fail. Then investigate whether the improvement generalizes to held-out cases. If a prompt only succeeds after examples from the test set are inserted into its instructions, the process has overfit the evaluation rather than improved a reusable capability.

A useful reporting sheet might contain the test ID, intent, policy scope, answer or action, expected evidence, observed outcome, defect type and severity. For every changed prompt, retain its version identifier and the model deployment used. This makes failure triage possible months later when a support ticket refers to a particular response and the original prompt has already been revised several times.

Promote only tested behavior into production

Prompt changes are software behavior changes even when no application source file changes. They can alter tool selection, output shape, retrieval queries, refusal behavior and cost. Store system instructions, prompt examples, schema definitions, tool descriptions and evaluation thresholds in version-controlled artifacts or an equivalent reproducible release process. Match the prompt version to the model, retrieval configuration and agent definition that were actually tested.

CI should catch deterministic defects first: invalid JSON schemas, missing required fields in fixtures, broken API contracts and unauthorized tool configurations. Behavioral evaluations come next, using stable datasets plus targeted adversarial cases. A candidate that clears those checks can move to a controlled rollout, where latency, tool errors, groundedness and safety signals are compared with the previous approved version. The GenAIOps release workflow connects these steps to deployment, telemetry, rollback and operational ownership.

Keep a reversible path for prompt and model changes. Rolling back code without restoring the prior prompt or tool description may leave the broken behavior in place. When a safety evaluation fails, avoid hiding it under a good average score. When an improved prompt relies on a preview-only feature, qualify what can be promoted in the actual target environment. Reliability depends on treating the prompt as one versioned dependency in a larger system, not as a secret string edited in a live portal.

Prepare for the AI-103 decision, not a magic prompt recipe

In a typical exam scenario, ask what went wrong before choosing a prompt technique. If the response violates a schema, validate whether the model and API support structured output. If the answer invents facts, inspect eligible retrieval and citation rules. If an agent chooses an unsafe tool, inspect tool exposure and authorization. If a response is too variable, evaluate supported sampling controls on the specific model. If a first draft is mostly right but misses a policy exception, a bounded evidence-based review step may be useful.

These distinctions explain why prompt engineering, model parameters, reflection and evaluation appear together in the official objective list. The candidate is not simply expected to write a persuasive instruction. The stronger skill is to define what reliable behavior means, measure it with representative cases, repair the actual failure layer and prove the revision does not create a more serious regression. That approach remains valuable even as individual Foundry interfaces and model settings evolve.