Glossary

Concise definitions of the terms used across the StoatFlow docs — lanes, commit barriers, epochs, watermarks, event time, changelog, state stores, serdes, repartitioning, and exactly-once.

Definitions of the vocabulary used throughout the StoatFlow documentation. Each entry links to the page that covers the concept in depth.

A delivery guarantee under which every record is processed at least once, but a crash may cause some records to be processed again and re-emitted downstream. StoatFlow's at-least-once mode drops the Kafka transaction: the runtime uses a non-transactional producer and commits consumer offsets directly, on a faster cadence. Trades duplicate-tolerance for a lower latency floor. See Exactly-once and at-least-once.

Changelog

A compacted Kafka topic that records every write to a state store. Because the topic is compacted by key, the latest value for every key is preserved indefinitely without unbounded growth. On restart, the runtime rebuilds local state by reading the changelog. See State stores and the Architecture page.

Commit barrier

A marker — not a data record — that the runtime periodically injects into the processing lanes. As records flow through the topology the barrier flows with them; when every lane has reached it, the runtime executes one Kafka transaction that atomically commits the epoch's state writes, output records, and input offsets. The mechanism belongs to the Chandy-Lamport family of distributed-snapshot algorithms (the same lineage as Flink's checkpoint barriers), scoped to a single process. See Exactly-once and at-least-once.

Dead-letter queue (DLQ)

A Kafka topic that receives records the pipeline could not handle — deserialization failures, processing exceptions, or production errors — when the corresponding exception handler is configured with one. The failed record is preserved together with error-context headers (the __stoatflow.errors.* namespace) for offline inspection or replay, while the topology keeps processing. See Error handling and DLQ.

Epoch

The unit of work bounded by two consecutive commit barriers — all the records processed, state written, and output produced between one commit and the next. Either an epoch commits atomically or, on a crash, its partial work is aborted and reprocessed. Epoch size and commit cadence self-tune within configured bounds; see Tuning.

Event time

The time at which an event actually occurred, carried on the record itself — the Kafka record timestamp by default, or whatever a custom TimestampExtractor returns. Stateful operators that reason about time (windowed aggregations, session windows, time-bounded joins) use event time, not arrival time. Contrast with processing time. See Event time and watermarks.

Exactly-once (EOS)

A guarantee that each input record affects state and produces output exactly once, even across crashes. StoatFlow achieves it with the commit barrier: state writes, sink output, and consumer offsets commit together in a single Kafka transaction, or not at all. Downstream consumers reading with read_committed isolation never see duplicates. Contrast with at-least-once. See Exactly-once and at-least-once.

Grace period

A configurable interval, set on a window specification (for example TimeWindows.ofSizeAndGrace(...)), during which late records are still folded into a window after the watermark has passed the window's end. Once the grace period elapses the window closes and later records that would have belonged to it are dropped. See Windowing.

Hot standby

An opt-in high-availability mode (stoatflow.ha.mode: active-standby) that runs the application as a cluster of one active instance plus one or more passive warm standbys: the active processes and commits while each standby continuously follows the changelog and stays warm; on failover a lag-aware election promotes the freshest, ready in seconds. It adds redundancy without changing the single-active-instance model. See High availability.

Key-affinity lane

A unit of concurrent processing inside the single JVM. Each lane runs the full topology independently against the shared state stores. Records are routed to lanes by key affinity — the same key always goes to the same lane — which guarantees per-key ordering while letting different keys process in parallel. Lane count is configured independently of Kafka partition count and scales with CPU cores. See Lanes and parallelism.

Keyed timer (TimerService)

A one-shot, per-key callback registered from a custom Processor through TimerService — a Flink-inspired extension. A timer fires onTimer(...) for a specific key at a specific instant, in event time (when the watermark passes it) or processing time, in the same per-key serialized context as process(...) and with full state access. With the persistent (RocksDB) timer backend, pending timers survive restarts. See Processor API → Timers.

Late record

A record whose event time is older than the current watermark. While its window remains within its grace period, a late record still updates the window's result; after the window has closed it is dropped. See Event time and watermarks.

Materialized

The configuration object (Materialized) that tells a stateful DSL operator to back its result with a named, queryable state store — controlling the store name, store type, serdes, caching, and changelog. Operators like count, reduce, aggregate, and table transforms accept a Materialized to expose their result as a store. See State stores.

Processing time

The wall-clock time at which a record is processed by the runtime, independent of when the underlying event occurred. Used by processing-time timers and as the default clock when no event-time semantics are configured. Contrast with event time. See Event time and watermarks.

Promotion token

A single record on the hot-standby coordination topic, claimed by compare-and-set, that authorizes exactly one standby to promote during a failover election. A standby that loses the claim stands down and remains a standby — no fence, no restart. Under exactly-once the token is the election layer and Kafka's producer-epoch fence remains the safety backstop; under at-least-once the token itself is the fence. See High availability.

Punctuator

A periodic callback scheduled by a processor via context.schedule(...), firing on stream time (watermark advance) or wall-clock time — useful for heartbeats, metrics, or flushing buffered state. Punctuators run on a dedicated punctuation lane per sub-topology with full state access, concurrently with record processing; for per-key work in the record-serialized context, use a keyed timer instead. See Processor API → Punctuators.

Repartition

Re-routing a record to a different lane because an operation changed its key — selectKey, groupBy, or a key-changing map/flatMap/join. In standard Kafka Streams this requires writing to and re-reading from an internal repartition topic; in StoatFlow it happens in-memory between lanes, with no broker round-trip and no extra serialization. See Lanes and parallelism.

Scheduled source

A topology-level source that emits records on a clock instead of consuming from Kafka — StreamsBuilder.scheduled(...) returns a normal KStream for downstream processing. Supports fixed intervals (stream-time or wall-clock) and cron expressions (CronExpression.unix/quartz/spring, always wall-clock). A StoatFlow extension with no Kafka Streams equivalent. See Scheduled sources.

Serde

A serializer/deserializer pair that converts between typed keys/values and the byte form Kafka stores. Source topics, sink topics, and state stores each need serdes for their key and value types; the DSL supplies defaults and per-operator overrides (Consumed, Produced, Materialized, Grouped). See Serdes.

State store

A keyed store that stateful operators read and write. StoatFlow ships several types — key-value, window, session, versioned (timestamped lookups), and timer — each in a RocksDB-backed (persistent) or in-memory variant. State is global: every store lives in the one JVM and any lane can read or write any key, with correctness guaranteed by key affinity. Durability comes from the changelog. See State stores and State and thread safety.

Sub-topology

A segment of a topology bounded by a re-keying operation. A key-changing operator (selectKey, groupBy, a key-changing map/flatMap, or a repartition) marks a sub-topology boundary, because records cross it under a new key. A topology with no re-keying is a single sub-topology; each additional re-key adds one. The boundary is what determines where repartitioning occurs. See Aggregations for where the boundary appears in practice.

Suppression

Holding a windowed aggregation's intermediate updates and emitting only the final result — declaratively via EmitStrategy.onWindowClose(), or with explicit buffer control via KTable.suppress(Suppressed.untilWindowCloses(...)) / untilTimeLimit(...). Suppressed results flush when the watermark closes the window. See Windowing → Suppression.

Topology

The directed graph of source nodes, processors, and sink nodes that defines a stream-processing application — built with StreamsBuilder (high-level DSL) or Topology directly (Processor API). The runtime executes the same topology independently in every lane. See StreamsBuilder and Architecture.

Watermark

A claim made by the runtime that no further records earlier than a given time are expected: "I do not expect any record older than T." Watermarks are tracked per source partition and combined into a single global watermark for the application. When the global watermark passes a window's end (plus its grace period), the window closes. The watermark advances together with the commit barrier so windowed results commit alongside the watermark progress that produced them. See Event time and watermarks.