Microsoft AB-620: Knowledge Grounding for Enterprise Agents

Enterprise agents are useful only when their answers are grounded in information the organization trusts. AB-620 therefore expects builders to connect agents to enterprise knowledge sources, configure advanced knowledge behavior, use Azure AI Search where appropriate, and combine retrieval with generative responses. The challenge is not merely connecting more data; it is making the right data available under the right identity and with enough structure to produce reliable answers.

Microsoft’s blueprint includes Copilot connectors, Power Platform connectors, Azure AI Search, custom knowledge sources, generative answers, Microsoft Foundry integration, and Fabric data agents. The exam rewards builders who can select the right grounding pattern based on freshness, permissions, structure, scale, and the kind of question users need answered.

Define what counts as authoritative knowledge

Before configuring retrieval, identify which sources are authoritative for the business question. A policy agent should prefer approved policy documents over informal chat threads. A support agent may need a curated knowledge base rather than every historical ticket. An analytics agent may need governed data rather than copied spreadsheets.

Source authority is a design decision because generative output can sound confident even when the retrieved material is outdated or unofficial. Builders should document which source owns the truth, how that source is maintained, and what happens when multiple systems disagree.

Authority can also vary by question. A human-resources policy repository may be authoritative for leave rules, while a finance system is authoritative for an employee’s current expense status. The grounding architecture can use several sources as long as each source has a defined domain of truth.

Keep permissions aligned with the underlying source

Grounding must respect access boundaries. An enterprise agent should not become a shortcut around permissions that users would face when opening the source directly. If a connector, search index, or service identity can see more than the user, the design needs an explicit authorization strategy to avoid unintended disclosure.

This is where identity design and knowledge design intersect. The retrieval path should make it clear whose permissions are applied and whether the returned content can safely be shown to the requesting user. Familiar RBAC principles remain relevant even when the final interface is conversational.

Indexing content introduces another permission question: whether access is evaluated at ingestion time, query time, or both. A secure design must ensure that an index does not flatten important access distinctions and make previously restricted content broadly retrievable.

Use Copilot connectors for supported enterprise content

Copilot connectors can bring supported external enterprise information into the broader Copilot ecosystem. They can be useful when the organization wants discoverable, governed access to systems that sit outside native Microsoft 365 content. The builder still needs to understand mapping, permissions, freshness, and the user experience around retrieved data.

Do not treat a connector as a quality guarantee. If the source content is inconsistent or poorly governed, making it easier to retrieve can amplify the problem. Connector adoption should therefore be paired with content ownership and lifecycle responsibilities.

Use Power Platform connectors for operational knowledge and actions

Power Platform connectors can provide access to many business systems and can support both data retrieval and actions. For grounding, they are useful when the agent needs current information from an operational system rather than a static document corpus. The same connector may later support a transaction based on the retrieved data.

This dual role makes permission design especially important. A connector that can read customer data and update customer records should not automatically expose both capabilities to every agent. Separate the retrieval requirement from the action requirement and grant the minimum surface necessary.

Choose Azure AI Search for indexed retrieval patterns

Azure AI Search is useful when the solution needs controlled indexing, search, filtering, and retrieval across a substantial corpus. AB-620 specifically includes configuring generative answers with Azure AI Search and Microsoft Foundry. The architecture can support retrieval-augmented generation by finding relevant content before the model composes a response.

Good retrieval depends on more than embedding data. Chunking, metadata, filters, index freshness, and query behavior influence what evidence reaches the model. If retrieval routinely returns weak context, prompt changes alone will not fix answer quality.

Metadata is especially useful for narrowing retrieval. Department, document type, effective date, region, or confidentiality level can help the system exclude irrelevant content before generation. Good filters often improve answer quality more reliably than asking the model to ignore weak evidence after retrieval.

Use generative answers with explicit source intent

Generative answers can synthesize retrieved information into a response that is easier for users to consume. The design should still define which knowledge sources are eligible and what the agent should do when evidence is insufficient. A safe agent should be able to say that it lacks enough information rather than inventing a plausible answer.

Grounding also benefits from clear response expectations. If users need citations, source names, or links back to authoritative content, include those requirements in the design. Transparency makes it easier for people to verify important answers and helps support teams diagnose weak responses.

Freshness should be testable. If policies change monthly, the team needs a way to confirm that updated content becomes retrievable within an acceptable interval. A beautifully grounded answer based on last quarter’s policy is still wrong. Source refresh belongs in the operational design.

Connect Microsoft Foundry when the solution needs advanced model services

AB-620 expects familiarity with Microsoft Foundry integration, including using its model catalog in custom prompts. This creates flexibility when the agent needs a model or prompt pattern beyond a default configuration. It also introduces architecture decisions around model selection, evaluation, cost, security, and operational monitoring.

Model choice should follow the workload. A larger model is not automatically better if a smaller option meets quality requirements with lower latency and cost. Enterprise grounding design should evaluate the complete retrieval-plus-generation pipeline rather than optimizing one component in isolation.

Use Fabric data agents for governed analytical context

A Fabric data agent can become part of a multi-agent or data-oriented solution when users need to interact with governed enterprise data. The value is different from document search: structured analytical context can support questions about metrics, trends, and business entities that live in a data platform.

Builders should still define semantic meaning and access boundaries. If a metric is calculated differently by two teams, the agent cannot solve that governance dispute automatically. Reliable analytical grounding depends on consistent data definitions and trusted models.

Evaluate grounding with representative questions

Grounding quality should be tested with questions that reflect real user behavior, including ambiguous wording, missing context, similar documents, stale information, and permission boundaries. A test set should reveal whether the retrieval layer finds the right evidence and whether the model uses that evidence accurately.

When an answer is wrong, isolate the failure. Did retrieval miss the source? Was the source incorrect? Did the model ignore strong evidence? Was the user unauthorized? This diagnostic approach is more effective than broadly rewriting prompts and aligns with the evaluation discipline expected across advanced Microsoft agent solutions.

Include “no answer” cases in the test set. An enterprise agent should be evaluated on whether it declines unsupported questions appropriately, not only whether it handles known questions well. Refusal or escalation can be a sign of quality when the available knowledge is insufficient.

Grounding also requires a strategy for conflicting sources. Two documents may describe the same process differently because one is obsolete, regional, or intended for a different audience. Retrieval alone cannot decide business authority unless metadata and source governance provide enough context. Effective solutions can use effective dates, document types, business ownership, or source ranking to help the agent prefer the right evidence. Without those signals, the model may synthesize a confident answer from contradictory material.

Chunking decisions influence retrieval quality as well. Very large chunks may include several unrelated ideas and reduce precision, while extremely small chunks can lose the context needed to interpret a statement. The right chunking pattern depends on document structure and query style. Headings, sections, metadata, and semantic boundaries can all help create retrievable units that are meaningful enough to support a complete answer without dragging irrelevant text into the prompt.

Enterprise evaluation should measure more than whether the final wording sounds good. Review retrieval precision, source freshness, permission enforcement, citation usefulness, answer completeness, and behavior when no authoritative evidence exists. If a test reveals that the correct document is never retrieved, tune the retrieval layer. If the document is retrieved but misinterpreted, inspect prompt and model behavior. If unauthorized content appears, stop and fix the access model before further quality tuning.

Grounding architecture should also include an ownership path for corrections. If a user finds a wrong answer that traces back to an outdated source, someone must know whether to update the source document, the search index, metadata, or the agent configuration. Without that ownership, teams often patch individual prompts while the bad source remains available. Reliable enterprise grounding improves the information system itself rather than accumulating prompt-level workarounds. Review source ownership and update frequency during retrieval-quality investigations before compensating with increasingly elaborate prompt instructions.