Microsoft DP-700: Real-Time and Streaming Data in Fabric

Streaming architecture is easy to overuse because “real time” sounds automatically better than batch. DP-700 requires a more disciplined view. The candidate must decide when low latency creates business value, choose an appropriate Fabric streaming engine, handle event time correctly, and design processing that remains accurate when events arrive late, out of order, or more than once.

Microsoft Fabric’s Real-Time Intelligence capabilities sit alongside Spark structured streaming, KQL, Eventstream, Eventhouse, and OneLake. The exam tests how these parts fit the workload rather than treating them as interchangeable names.

Define the latency requirement before selecting technology

Ask how quickly the consumer must act. Fraud detection, operational alerting, telemetry monitoring, and live customer experiences may justify seconds or sub-seconds of delay. A finance report consumed the next morning usually does not. Every reduction in latency can increase infrastructure, monitoring, and recovery complexity.

The requirement should be measurable: “95% of events available for query within two minutes,” for example, is more useful than “near real time.” It also clarifies what happens when the target cannot be met.

The Microsoft DP-700 exam expects candidates to translate this kind of requirement into an appropriate Fabric pattern.

Use Eventstream when event routing and low-code processing fit

Eventstream can provide a managed path for ingesting, routing, filtering, and transforming streams before they reach analytical destinations. It is useful when the workload benefits from an event-oriented experience without requiring a large custom streaming application.

The design still needs decisions about source connectivity, event schema, partitioning, retention, transformation, and destinations. A low-code surface does not remove the need to understand the event contract.

Events should carry stable identifiers and useful timestamps where possible. Without them, deduplication and event-time analysis become harder later.

Use Spark structured streaming for code-centric processing

Spark structured streaming is appropriate when teams need programmable transformations, stateful processing, integration with Spark libraries, or engineering patterns that already use DataFrames and Delta tables. The same distributed-computing considerations that matter in batch Spark also matter in streaming, but continuous processing adds state and checkpoint requirements.

A streaming job should persist enough progress information to restart without losing or multiplying data. Checkpoints, sink semantics, source offsets, and idempotent writes all contribute to reliable recovery.

Scaling should respond to actual throughput and processing cost. Adding compute cannot solve a fundamentally skewed partitioning scheme or a downstream system that cannot accept the write rate.

Use KQL and Eventhouse for event-oriented analytics

KQL is designed for fast exploration and analysis of telemetry, logs, and other time-oriented data. Eventhouse provides a natural destination for Real-Time Intelligence workloads where analysts need interactive queries over high-volume event data.

The design choice is not just “streaming equals KQL.” Consider the consumers and transformations. If the primary requirement is operational analytics over event data, KQL can be a strong fit. If events need extensive engineering and integration with broader lakehouse processing, Spark or another path may be more appropriate.

DP-700 explicitly includes KQL among the transformation skills, so candidates should understand where it is strongest rather than trying to translate every SQL habit directly.

Reason about event time, not only arrival time

An event can occur at 10:00 and arrive at 10:07 because of network delay, device buffering, or upstream retry. If a five-minute window groups data by arrival time, the event may be counted in the wrong period. Event-time processing uses the time the business event actually occurred, but then the system must decide how long to wait for late arrivals.

Windowing functions define how continuous events are grouped for analysis. Tumbling windows create non-overlapping intervals, while other window strategies can overlap or follow activity patterns. The correct window matches the question the business is asking.

Late-event policy should be explicit. Accepting late data forever makes results unstable; dropping it immediately can sacrifice accuracy. The trade-off belongs in the architecture.

Design deduplication and ordering explicitly

Many streaming sources provide at-least-once delivery, meaning retry can produce duplicate events. The consumer must use an event identifier, sequence value, or business key to recognize duplicates when they would distort the result.

Global ordering is often expensive or impossible at scale. Instead, systems preserve ordering within a partition or entity when the business requires it. A device’s events may need sequence order even if different devices can be processed independently.

The same principle appears in batch ingestion: reliability comes from knowing which unit of work can be replayed. Streaming simply reduces the time between those decisions.

Choose storage and shortcuts for the analytical path

Streaming data may serve immediate operational queries and later historical analytics. That can lead to multiple destinations or a layered architecture in which raw events are retained while curated structures support reporting. OneLake can provide a shared foundation, and shortcuts can reduce copying where source and governance requirements support them.

The comparison between data warehouse and data lake patterns remains relevant because the real-time path eventually needs a serving model. Event data that is useful for live alerting may also need daily aggregation for BI.

The architecture should explain which store is authoritative for replay, which is optimized for live analysis, and how retention differs between them.

Monitor lag, throughput, and correctness together

A streaming job can remain “running” while falling minutes behind. Operational monitoring therefore needs input rate, processing rate, consumer lag, checkpoint health, error rate, event rejection, and end-to-end latency. Capacity should be scaled before lag becomes business-visible.

Correctness metrics matter too. A sudden fall in event count, unexpected schema change, or duplicate spike can be more damaging than a process crash because incorrect data may still flow to dashboards.

The broader data engineering role includes this responsibility: real-time systems are not finished when events move; they are finished when the data is timely, explainable, and recoverable.

Backpressure is another streaming concern. If events arrive faster than the processing stage can handle them, the system needs buffering, scaling, throttling, or a controlled degradation strategy. Simply adding workers may increase pressure on the destination. Capacity planning therefore has to consider the whole path from producer to sink.

Schema evolution is especially risky in streaming because there may be no natural pause for coordinated deployment. Producers and consumers should use compatible change rules, version fields where appropriate, and validation that can quarantine unexpected payloads without stopping every partition. A new optional property is very different from changing the type or meaning of an existing key.

Replay requirements should influence retention. If an event processing bug is discovered after two days, the team can correct historical output only if the original events or an authoritative raw representation are still available. Retaining replayable data for an appropriate period creates insurance against logic defects, not just source outages.

Real-time architecture also needs a plan for downstream consumers that do not operate in real time. A live Eventhouse query and a daily executive dashboard may derive from the same event stream but have different freshness, aggregation, and retention needs. Designing those products separately prevents the lowest-latency requirement from forcing unnecessary complexity into every use case.

Cost should be tied to business latency. Continuous compute, high retention, and duplicated serving layers can be justified when rapid decisions create value, but not when consumers refresh once a day. A sound DP-700 answer can explain not only how to achieve low latency but why the selected latency is appropriate.

Testing streaming systems requires more than sending a few sample events. Useful tests include out-of-order data, duplicates, bursts, malformed payloads, long idle periods, restart after checkpoint loss, and events arriving beyond the normal lateness window. These cases reveal whether the design is truly stateful and recoverable.

Time zones are another common source of error. Event timestamps should have a clear standard, usually with UTC retained in the processing layer and local interpretation applied for reporting where needed. Daylight-saving changes can otherwise distort window boundaries and daily aggregates.

Data contracts are particularly valuable for event streams because producers and consumers are loosely coupled in time. A contract should define event identity, required fields, timestamp meaning, versioning rules, and whether a producer can resend or reorder events. Without that agreement, every consumer invents its own assumptions.

For historical analytics, raw event retention should be separated from derived aggregates. If only five-minute summaries are retained, a future requirement for a different window cannot be reconstructed. The appropriate raw retention period depends on cost, regulation, and how likely the business is to need replay.

Operational runbooks should explain how to pause, drain, and resume a streaming path without losing state. Maintenance is part of production life; a design that can only be stopped by abandoning checkpoints creates unnecessary recovery risk.

Microsoft’s October 19, 2026 DP-700 skills map explicitly includes loading patterns for streaming data, Eventstream, Spark structured streaming, KQL, native tables, OneLake shortcuts, query acceleration, and windowing. Those objectives make streaming a design topic, not an isolated feature checklist.

For scenario questions, begin with the latency and correctness requirement. Then choose the processing engine, event-time policy, recovery model, storage path, and monitoring signals that satisfy it with the least operational complexity.