Kafka Streams compatibility matrix

Operator-by-operator parity between the StoatFlow DSL and the Kafka Streams DSL โ€” implemented, partial, not yet implemented, deprecated, incompatible, and StoatFlow extensions.

The engineering view of Kafka Streams DSL parity in StoatFlow: every operator, helper class, and store method, with its current status and notes where behaviour differs. Targets Kafka Streams 4.4.x: StoatFlow reimplements the Kafka Streams DSL surface under its own package, so most KS applications port by swapping the dependency and rewriting imports โ€” see Migrating from Kafka Streams, and let the automated port recipe do it for you.

For the high-level, feature-by-feature comparison against Kafka Streams and Flink, see Comparison matrix. For how-to guidance on the operators below, see Building.

This matrix is transcribed from the in-repository compatibility reference and curated for the public docs. Status reflects the current release (1.0.0-rc.4). A small number of rows differ in signature from Kafka Streams (marked โš ๏ธ) or are intentionally absent (marked ๐Ÿ›‘) โ€” read the notes.

Legend

SymbolMeaning
โœ…Implemented
โš ๏ธPartial โ€” stub, limited, or differing signature
โŒNot implemented yet
๐Ÿ›‘Incompatible, or won't be implemented (not needed in the single-instance model)
โ†˜๏ธDeprecated in Kafka Streams
๐Ÿš€StoatFlow extension โ€” not in Kafka Streams
๐Ÿ†•StoatFlow extension on an otherwise KS-compatible interface
๐ŸŸกRecognised no-op โ€” accepted and warned about, never an error (configuration keys only)
๐ŸงฉClient pass-through โ€” forwarded to the Kafka clients the key is valid for (configuration keys only)

StreamsBuilder

MethodStatusNotes
StreamsBuilder()โœ…No-arg
StreamsBuilder(TopologyConfig)โœ…KSC-77 โ€” seeds build-time defaults (DSL store format, FK reverse-index cache, explicit naming) and the KIP-1112 ProcessorWrapper from a TopologyConfig projection of StreamsConfig. It carries no serde defaults: default.key.serde / default.value.serde on the runtime StreamsConfig decide every serde, as in Kafka Streams. Named-topology / per-topology overrides N/A (single-instance)
stream(topic, consumed?)โœ…Single topic subscription; single-arg form Java-callable
stream(Collection<String>)โœ…Multiple topics subscription
stream(Pattern)โœ…Pattern-based subscription โ€” N patterns combine into one subscription; a topic two patterns accept is rejected at assignment; literals are quoted into the combined regex (KSC-101); no own-topic filtering. Deleting a matched topic is survived (in-place engine restart, ~0.5-1 s, no pod restart), matching KS; a deleted literal source topic stays fatal, also matching KS โ€” see Streams builder
table(topic, consumed?)โœ…
table(topic, materialized)โœ…Explicit skip-Consumed overload (KS parity). The Materialized serdes also read the source topic, as in Kafka Streams
table(topic, consumed?, materialized)โœ…With state store
globalTable(topic, consumed?)โœ…Wraps KTable
globalTable(topic, materialized)โœ…Without Consumed parameter. โš ๏ธ A versioned Materialized is accepted and materialises a versioned store โ€” Kafka Streams throws GlobalTables cannot be versioned. (KSC-125, more permissive)
build()โœ…Returns Topology
build(Properties)โš ๏ธThe Properties are ignored โ€” logs a warning and delegates to build(). Config carries over via StreamsConfig.fromProperties(...) / the KS-shaped StreamsConfig(Properties) ctor instead (ADR-124)
addStateStore(storeBuilder)โœ…Register standalone store
addStateStore(storeBuilder, keySerde, valueSerde)โœ…With changelog Serdes
addStateStore(storeSupplier, keySerde, valueSerde)โœ…Convenience overload
addGlobalStore(storeBuilder, topic, consumed, processor)โœ…Custom update processor
addGlobalStore(storeBuilder, topic, consumed)โœ…Default put/delete semantics, written in the store's shape: a timestamped builder holds ValueAndTimestamp(value, record timestamp), a versioned one a version at the record timestamp. ๐Ÿ†• StoatFlow convenience โ€” Kafka Streams always takes a processor
addReadOnlyStateStore(storeBuilder, topic, consumed, processor)โœ…KIP-813: custom processor
addReadOnlyStateStore(storeBuilder, topic, consumed)โœ…KIP-813: Default put/delete semantics, written in the store's shape: a timestamped builder holds ValueAndTimestamp(value, record timestamp), a versioned one a version at the record timestamp. ๐Ÿ†• StoatFlow convenience โ€” Kafka Streams always takes a processor
scheduled(interval, type, emitter)๐Ÿš€Interval-based periodic record source
scheduled(interval, type, named, emitter)๐Ÿš€With explicit naming
scheduled(interval, type, keySerde, emitter)๐Ÿš€With explicit key serde for lane assignment
scheduled(interval, type, keySerde, options, emitter)๐Ÿš€With per-source ScheduledSourceOptions โ€” when the source fires first (ScheduledStart.IMMEDIATELY / NEXT_INTERVAL); default from dsl.scheduled-source-start (A2.19)
scheduled(named, interval, type, keySerde, options, emitter)๐Ÿš€Named, with per-source ScheduledSourceOptions (A2.19)
scheduled(cron, emitter)๐Ÿš€Cron-based calendar scheduling
scheduled(cron, named, emitter)๐Ÿš€Cron with explicit naming
scheduled(cron, keySerde, emitter)๐Ÿš€Cron with explicit key serde for lane assignment

Topology (Processor API)

Both the high-level DSL (StreamsBuilder) and the low-level Topology builder are supported.

MethodStatusNotes
describe()โœ…Returns the Kafka Streams TopologyDescription (nested Source/Processor/Sink/Subtopology/GlobalStore node graph). Breaking vs older StoatFlow: previously returned the native model โ€” that view moved to describeStoatFlow()
describeStoatFlow()๐Ÿš€StoatFlow-native topology model (the former describe() shape)
addSource(name, topics...)โœ…16 Kafka Streams-shaped overloads: offset reset, timestamp extractor, deserializers, pattern (same rules as stream(Pattern)), and ๐Ÿ†• watermark strategy (no KS equivalent โ€” the Processor API route to a per-source generator, withIdleness and alignment). The deserializers, timestamp extractor and offset reset passed in are used; null means the configured default (KS parity).
addSource(name, consumed, topics... | pattern)๐Ÿ†•Takes a Consumed, mapped exactly as StreamsBuilder.stream() maps it: key/value serdes, timestamp extractor or watermark strategy, offset reset. Unlike a key deserializer, it declares the key serde, so Processor API timers and in-memory sub-topology boundaries behind the source use it instead of default.key.serde. A Consumed.as(...) name must match the node name; withMaterializeFromSourceTopic is refused
addProcessor(name, supplier, parents...)โœ…User-defined processor with ProcessorSupplier
addScheduledSource(name, interval, type, emitter)๐Ÿš€Processor API route to an interval scheduled source โ€” StoatFlow extension (ADR-039)
addScheduledSource(name, interval, type, options, emitter)๐Ÿš€With per-source ScheduledSourceOptions (A2.19)
addScheduledSource(name, cron, emitter)๐Ÿš€Cron-based scheduled source
addSink(name, topic, parents...)โœ…8 overloads: serializers, partitioner, TopicNameExtractor. The serializers and partitioner passed in are used; null means the configured default (KS parity). Difference (KSC-148): a static topic must exist when the application starts โ€” a missing one refuses the start; a TopicNameExtractor sink is not checked.
addStateStore(builder, processors...)โœ…Register store and optionally connect to processors. Several processors may connect the same builder instance and share the store; a different builder under an existing name is rejected
addGlobalStore(...)โœ…2 variants: with/without TimestampExtractor. The deserializers and timestamp extractor passed in are used; null means the configured default (KS parity).
addGlobalStore(storeBuilder, sourceName, consumed, topic, processorName, supplier)๐Ÿ†•The Consumed form, mapped as addSource(name, consumed, โ€ฆ). An offset reset other than earliest is refused: a store's source always reads from the beginning
addReadOnlyStateStore(...)โœ…2 variants (KIP-813): source topic is changelog. The deserializers and timestamp extractor passed in are used; null means the configured default (KS parity).
addReadOnlyStateStore(storeBuilder, sourceName, consumed, topic, processorName, supplier)๐Ÿ†•The Consumed form, mapped as addSource(name, consumed, โ€ฆ). An offset reset other than earliest is refused
connectProcessorAndStateStores(...)โœ…Connect existing store to processor

TopologyDescription

describe() returns the Kafka Streams TopologyDescription โ€” a graph of nested node interfaces.

MemberStatusNotes
subtopologies() / globalStores()โœ…Sets of Subtopology / GlobalStore
Subtopology.id() / nodes()โœ…
Node.name() / predecessors() / successors()โœ…
Source.topicSet() / topicPattern()โœ…The matched topics of a pattern source are a runtime fact (stoatflow.source.pattern.matched.topics), not part of the description
Processor.stores()โœ…Connected store names
Sink.topic() / topicNameExtractor()โœ…Static or dynamic routing
GlobalStore.id() / source() / processor()โœ…

KStream โ€” stateless transformations

MethodStatusNotes
filter(predicate)โœ…
filterNot(predicate)โœ…
map(mapper)โœ…Key-changing; the sub-topology boundary opens downstream, where an operator needs the new key's affinity (KS repartitionRequired model, ADR-138)
mapValues(ValueMapper)โœ…
mapValues(ValueMapperWithKey)โœ…Key-aware value mapper variant
flatMap(mapper)โœ…Key-changing; the sub-topology boundary opens downstream, where an operator needs the new key's affinity (KS repartitionRequired model, ADR-138)
flatMapValues(ValueMapper)โœ…
flatMapValues(ValueMapperWithKey)โœ…Key-aware value mapper variant
selectKey(keySelector)โœ…Key-changing; the sub-topology boundary opens downstream, where an operator needs the new key's affinity (KS repartitionRequired model, ADR-138)
peek(action)โœ…Java BiConsumer overload
merge(other)โœ…
split()โœ…Returns BranchedKStream
repartition()โœ…In-memory only
repartition(Repartitioned)โœ…Custom partitioner, naming
print()โœ…Configurable via Printed

KStream โ€” terminal operations

MethodStatusNotes
forEach(action)โœ…Java BiConsumer overload
to(topic, produced?)โœ…Static topic routing. Difference (KSC-148): the topic must exist when the application starts โ€” a missing one refuses the start with MissingOutputTopicException, where Kafka Streams finds it at the first send (opt-out validation.validate-output-topics)
to(TopicNameExtractor, produced?)โœ…Dynamic topic routing. Not checked at start โ€” its topics are chosen per record
toTable()โœ…Auto-generates store name; the store takes the stream's serdes, windowed keys included
toTable(named?, materialized?)โœ…With naming and materialization; Materialized serdes first, then the stream's (KS parity)

KStream โ€” Processor API

MethodStatusNotes
process(processorSupplier, named?, storeNames...)โœ…Key-changing; ProcessorSupplier. Connects the named stores (and any declared via supplier.stores()). Like KS, it does not open a sub-topology boundary of its own โ€” see the note below. A node connected to a store another part of the topology writes is placed with that store's writer, as KS puts processors sharing a store into one subtopology; nodes in different sub-topologies connecting one store warn (presumed-cross-sub-topology-write) โ€” see Sharing a store between processors
processValues(processorSupplier, named?, storeNames...)โœ…Non-key-changing; FixedKeyProcessorSupplier + named/declared stores. Like KS, it does not open a sub-topology boundary of its own โ€” see the note below. Store placement as for process
transform()๐Ÿ›‘Removed from KStream in KS 4.x; use process()
transformValues()๐Ÿ›‘Removed from KStream in KS 4.x; use processValues()
Key affinity at a Processor API node. Kafka Streams never materialises a repartition before process() / processValues(), even with connected stores, and since 1.0.0 neither does StoatFlow (topology.processor-api-key-affinity: off โ€” presumed restores the pre-1.0.0 boundary). After a many-to-one re-key that means per-key state at such a node is fragmented across lanes, exactly as it is across tasks in KS. StoatFlow adds a guard KS does not have: it refuses to compile the provable case โ€” a proven re-key into a node with a writable store (topology.validation.papi-key-affinity-presumed, default error) โ€” and warns where the feeder is itself a Processor API node, whose key change is only a conservative default (papi-key-affinity-presumed-chain, default warn). See Key affinity after a re-key.

KStream โ€” grouping

MethodStatusNotes
groupByKey()โœ…Returns KGroupedStream
groupByKey(Grouped)โœ…With Serdes configuration
groupBy(keySelector)โœ…Key-changing; the sub-topology boundary opens downstream, where an operator needs the new key's affinity (KS repartitionRequired model, ADR-138)
groupBy(keySelector, Grouped)โœ…With Serdes configuration

KStream โ€” joins

MethodStatusNotes
join(KStream, ValueJoiner, windows)โœ…Stream-stream windowed join
join(KStream, ValueJoiner, windows, streamJoined)โœ…With StreamJoined config
join(KStream, ValueJoinerWithKey, windows)โœ…Key-aware joiner variant
join(KStream, ValueJoinerWithKey, windows, streamJoined)โœ…Key-aware joiner with config
leftJoin(KStream, ValueJoiner, windows)โœ…Stream-stream left join. An unmatched record whose window has already closed when it arrives is emitted at once with a null side, under its own timestamp, never buffered โ€” KS's late branch, judged on the watermark instead of stream time (KSC-132)
leftJoin(KStream, ValueJoiner, windows, streamJoined)โœ…With StreamJoined config
leftJoin(KStream, ValueJoinerWithKey, windows)โœ…Key-aware joiner variant
leftJoin(KStream, ValueJoinerWithKey, windows, streamJoined)โœ…Key-aware joiner with config
outerJoin(KStream, ValueJoiner, windows)โœ…Stream-stream outer join. Late unmatched records on either side are emitted at once with a null side, as for leftJoin
outerJoin(KStream, ValueJoiner, windows, streamJoined)โœ…With StreamJoined config
outerJoin(KStream, ValueJoinerWithKey, windows)โœ…Key-aware joiner variant
outerJoin(KStream, ValueJoinerWithKey, windows, streamJoined)โœ…Key-aware joiner with config
join(KTable, ValueJoiner)โœ…Stream-table inner join. After a re-key the join reads the table from another lane: under exactly_once as of the stream record's commit, and an lruMap or in-memory versioned table is refused there (KSC-126, also for leftJoin); with no re-key it reads on its own lane
join(KTable, ValueJoinerWithKey)โœ…Key-aware joiner variant
leftJoin(KTable, ValueJoiner)โœ…Stream-table left join
leftJoin(KTable, ValueJoinerWithKey)โœ…Key-aware joiner variant
join(GlobalKTable, keySelector, ValueJoiner)โœ…Stream-GlobalKTable inner join. Kotlin: also a (V, GV) -> VR lambda overload, optionally with a Named before the joiner. Under exactly_once every global-table join (join and leftJoin, all overloads) reads the table as of the stream record's commit โ€” stronger than KS, whose global thread gives no cut against the stream tasks; same API. An lruMap or in-memory versioned global table is refused there (KSC-126)
join(GlobalKTable, keySelector, ValueJoinerWithKey)โœ…Key-aware joiner variant; deprecated since Kafka Streams 4.4 (KIP-1340): the joiner receives the stream key, not the join key โ€” use ValueJoinerWithStreamAndMappedKey
join(GlobalKTable, keySelector, ValueJoiner|ValueJoinerWithKey, Named)โœ…Named overloads; the ValueJoinerWithKey form is deprecated as above
join(GlobalKTable, keySelector, ValueJoinerWithStreamAndMappedKey, Named)โœ…KIP-1340 (Kafka Streams 4.4): the joiner receives (streamKey, mappedKey, value1, value2). Kotlin: also a (K, GK, V, GV) -> VR lambda overload, optionally with a Named before the joiner
leftJoin(GlobalKTable, keySelector, ValueJoiner)โœ…Stream-GlobalKTable left join. Kotlin: also a (V, GV?) -> VR lambda overload, optionally with a Named before the joiner
leftJoin(GlobalKTable, keySelector, ValueJoinerWithKey)โœ…Key-aware joiner variant; deprecated since Kafka Streams 4.4 (KIP-1340): the joiner receives the stream key, not the join key โ€” use ValueJoinerWithStreamAndMappedKey
leftJoin(GlobalKTable, keySelector, ValueJoiner|ValueJoinerWithKey, Named)โœ…Named overloads; the ValueJoinerWithKey form is deprecated as above
leftJoin(GlobalKTable, keySelector, ValueJoinerWithStreamAndMappedKey, Named)โœ…KIP-1340 (Kafka Streams 4.4): the joiner receives (streamKey, mappedKey, value1, value2); mappedKey is null when the selector returns null (the table value is then null too). Kotlin: also a (K, GK?, V, GV?) -> VR lambda overload, optionally with a Named before the joiner
Temporal join semantics (KIP-914). When the backing table uses a versioned store (KIP-889), stream-table joins perform a temporal lookup against the stream record's timestamp โ€” returning the table value that was valid at the stream event time, so results stay correct when records arrive out of order. Stream-table joins are otherwise point-in-time lookups; a table update does not re-trigger the join.

KStream โ€” branching (BranchedKStream)

MethodStatusNotes
branch(predicate)โœ…
branch(predicate, branched)โœ…With Branched config
defaultBranch()โœ…
defaultBranch(branched)โœ…With Branched config
noDefaultBranch()โœ…

KGroupedStream

MethodStatusNotes
count()โœ…With Materialized overloads
reduce(reducer)โœ…With Materialized overloads
aggregate(initializer, aggregator)โœ…With Materialized overloads
windowedBy(TimeWindows)โœ…Tumbling/hopping windows โ†’ TimeWindowedKStream
windowedBy(SlidingWindows)โœ…Sliding windows (KIP-450) โ†’ returns TimeWindowedKStream, exactly like Kafka Streams (no separate SlidingWindowedKStream type)
windowedBy(SessionWindows)โœ…Session windows
cogroup()โœ…Returns CogroupedKStream

Windowed grouped streams

TimeWindowedKStream

Reached by both windowedBy(TimeWindows) (tumbling/hopping) and windowedBy(SlidingWindows) (KIP-450 sliding) โ€” sliding windows flow through this type, exactly as in Kafka Streams. All overloads take a plain-key Materialized<K, V, โ€ฆ>. For small event times โ€” below size โˆ’ advance for hopping windows, below the time difference for sliding ones โ€” the windows differ from Kafka Streams' โ€” see Windows near time 0 under the architectural differences.

MethodStatusNotes
count()โœ…With Materialized overloads (plain key)
reduce(reducer)โœ…With Materialized overloads
aggregate(initializer, aggregator)โœ…With Materialized overloads
emitStrategy()โœ…OnWindowUpdate (default), OnWindowClose (lateness and window close follow the watermark, not per-aggregation stream time โ€” KSC-132)

SessionWindowedKStream

MethodStatusNotes
count()โœ…With session merger
reduce(reducer)โœ…Reducer doubles as merger
aggregate(initializer, aggregator, sessionMerger)โœ…With session merger
emitStrategy()โœ…OnWindowUpdate (default), OnWindowClose (lateness and window close follow the watermark, not per-aggregation stream time โ€” KSC-132)

Cogrouped streams

CogroupedKStream

MethodStatusNotes
cogroup(KGroupedStream, aggregator)โœ…Add another stream to the cogroup
aggregate(initializer)โœ…With Materialized overloads
windowedBy(TimeWindows)โœ…Returns TimeWindowedCogroupedKStream
windowedBy(SlidingWindows)โœ…Sliding windows (KIP-450) โ†’ returns TimeWindowedCogroupedKStream
windowedBy(SessionWindows)โœ…Returns SessionWindowedCogroupedKStream

TimeWindowedCogroupedKStream

MethodStatusNotes
aggregate(initializer)โœ…With Materialized overloads
emitStrategy()โœ…OnWindowUpdate (default), OnWindowClose (lateness and window close follow the watermark, not per-aggregation stream time โ€” KSC-132)

SessionWindowedCogroupedKStream

MethodStatusNotes
aggregate(initializer, sessionMerger)โœ…With Materialized overloads; (initializer, sessionMerger) argument order matches Kafka Streams
emitStrategy()โœ…OnWindowUpdate (default), OnWindowClose (lateness and window close follow the watermark, not per-aggregation stream time โ€” KSC-132)

KTable โ€” transformations

MethodStatusNotes
filter(predicate)โœ…Derived table (no state store)
filter(predicate, Named)โœ…With naming
filter(predicate, Materialized)โœ…With materialization
filter(predicate, Named, Materialized)โœ…With naming and materialization
filterNot(predicate)โœ…Inverse of filter
filterNot(predicate, Named)โœ…With naming
filterNot(predicate, Materialized)โœ…With materialization
filterNot(predicate, Named, Materialized)โœ…With naming and materialization
mapValues(ValueMapper)โœ…Derived table (no state store); joined and grouped like a materialized table, with old values, as in KS
mapValues(ValueMapper, Named)โœ…With naming
mapValues(ValueMapper, Materialized)โœ…With materialization
mapValues(ValueMapper, Named, Materialized)โœ…With naming and materialization
mapValues(ValueMapperWithKey)โœ…Key-aware value mapper variant
mapValues(ValueMapperWithKey, Named)โœ…With naming
mapValues(ValueMapperWithKey, Materialized)โœ…With materialization
mapValues(ValueMapperWithKey, Named, Materialized)โœ…With naming and materialization
toStream()โœ…Same key type
toStream(KeyValueMapper)โœ…Key-changing variant; the sub-topology boundary opens downstream, where an operator needs the new key's affinity (ADR-138)
queryableStoreName()โœ…Returns store name if materialized; null for a derived table built without Materialized and for an untilWindowCloses / untilTimeLimit suppressed table (KS parity). An untilBarrier table keeps its source's store name
groupBy(keyValueSelector)โœ…Selector returns KeyValue<KR, VR>; returns KGroupedTable<KR, VR>
groupBy(keyValueSelector, Grouped)โœ…With Serdes configuration
suppress(Suppressed)โœ…untilWindowCloses, untilTimeLimit, untilBarrier. Each emission carries the buffered record's latest headers, timestamp, topic, partition and offset, as in Kafka Streams; records leaving together leave in Kafka Streams' order (buffer time, then key), and a downstream groupBy or foreign-key join gets the value from before the key's first buffering as the old value โ€” Kafka Streams overwrites it per update, so StoatFlow retracts what downstream actually saw. Joins read the suppressed table through its value getter: a buffered key reads its pre-buffering value. The buffer falls back to default.key.serde / default.value.serde when no serde was set upstream โ€” including for windowed tables, where Kafka Streams fails (KAFKA-9259)
transformValues(ValueTransformerWithKeySupplier, named?, storeNames...)โœ…Stateful per-key value transform; returns KTable<K, VR>
transformValues(ValueTransformerWithKeySupplier, Materialized, Named, storeNames...)โœ…Materializes the result into a queryable store (tombstone-aware: a null transform deletes). Supplier parameter carries Kafka Streams' ? super K, ? super V, ? extends VR wildcards, so a supplier producing a subtype of the result value type still fits

KTable โ€” primary-key joins

Both tables share key type K. No co-partitioning required (global state model).

MethodStatusNotes
join(KTable, joiner)โœ…Inner join. Either side may be materialized, a derived filter / mapValues / transformValues, suppressed or a windowed aggregation โ€” read through its value getter, as in KS; only an unmaterialized join result is refused. Each side reads the other from another lane: under exactly_once as of its own commit, and an lruMap or in-memory versioned input table is refused (KSC-126, every primary-key join overload) โ€” a RocksDB versioned one is accepted
join(KTable, joiner, Materialized)โœ…With materialization of join result
join(KTable, joiner, Named, Materialized)โœ…With naming and materialization
leftJoin(KTable, joiner)โœ…Left table always emits
leftJoin(KTable, joiner, Materialized)โœ…With materialization of join result
leftJoin(KTable, joiner, Named, Materialized)โœ…With naming and materialization
outerJoin(KTable, joiner)โœ…Full outer join
outerJoin(KTable, joiner, Materialized)โœ…With materialization of join result
outerJoin(KTable, joiner, Named, Materialized)โœ…With naming and materialization
KIP-914 semantics. When both tables are backed by versioned stores, out-of-order records do not trigger join evaluation, and temporal lookups return the correct value from the other table โ€” preventing inconsistent results under out-of-order arrival.

KTable โ€” foreign-key joins

Foreign-key extractor: a Java Function<V, KO?> (or BiFunction<K, V, KO?> for key-aware extraction) โ€” or a Kotlin (V) -> KO? / (K, V) -> KO? lambda.

MethodStatusNotes
join(KTable, fkExtractor, joiner)โœ…Inner FK join. โš ๏ธ An lruMap or in-memory versioned input table on either side is refused under exactly_once (KSC-126, all FK join overloads); a RocksDB versioned one is accepted
join(KTable, fkExtractor, joiner, Materialized)โœ…With materialization of join result
join(KTable, fkExtractor, joiner, TableJoined, Materialized)โœ…With naming and materialization
leftJoin(KTable, fkExtractor, joiner)โœ…Left FK join; a null FK emits joiner(left, null) (KIP-962). FK-join retractions are forwarded materialized or not, as in Kafka Streams
leftJoin(KTable, fkExtractor, joiner, Materialized)โœ…With materialization of join result
leftJoin(KTable, fkExtractor, joiner, TableJoined, Materialized)โœ…With naming and materialization
outerJoin(KTable, fkExtractor, joiner)โš ๏ธNot supported in KS either

KGroupedTable

KGroupedTable is the result of KTable.groupBy() and uses changelog semantics โ€” moving a key from group A to group B decrements A and increments B. Every readable table tracks old values for groupBy() โ€” materialized, derived (filter, mapValues, transformValues), suppressed or windowed โ€” as in Kafka Streams; only an unmaterialized join result is refused.

MethodStatusNotes
count()โœ…Changelog semantics
reduce(adder, subtractor)โœ…Kafka Streams compatible; both adder and subtractor are Reducer<V>
aggregate(init, adder, subtractor)โœ…Kafka Streams compatible; both adder and subtractor are Aggregator<K, V, VR> โ€” no StoatFlow-only Subtractor type

GlobalKTable

MethodStatusNotes
queryableStoreName()โœ…Returns underlying table's store name
asKTable()๐Ÿš€StoatFlow extension to unwrap
GlobalKTable wraps KTable internally. Because all state is global, its behaviour is identical to KTable.

Processor API

ProcessorContext

MethodStatusNotes
forward(record)โœ…Forward to downstream processors
forward(record, childName)โœ…Forward to specific child
getStateStore(name)โœ…Access connected state stores โ€” declared via ProcessorSupplier.stores() or the process(supplier, "name") varargs. An undeclared store throws StoreNotConnectedException (KS parity). Global stores (addGlobalStore, globalTable) are exempt and returned read-only unless the node declares them, in which case they are writable (a StoatFlow extension โ€” KS makes declaring a global store a TopologyException). The read-only view covers every store family (plain, timestamped, versioned, headers-aware) and keeps the store's declared type, so the caller's getStateStore<TimestampedKeyValueStore<..>> cast still succeeds; read-only stores (addReadOnlyStateStore) are exempt too โ€” a deliberate divergence, KS exempts only globals
schedule(interval, type, punctuator)โœ…KS-compatible punctuator, KS fire times (STREAM_TIME aligned to 0, a grid, one fire per catch-up โ€” the first fire is at the first watermark, which in StoatFlow may be negative); inside the callback context.timestamp() is the punctuation time and headers() empty, as in KS
schedule(interval, type, mode, punctuator)๐Ÿš€StoatFlow extension with BLOCKING/NON_BLOCKING mode
schedule(interval, startTime, type, punctuator)โœ…KIP-1146: anchored punctuation with grid-aligned fire times
schedule(interval, startTime, type, mode, punctuator)โœ…KIP-1146: anchored punctuation with BLOCKING/NON_BLOCKING mode
recordMetadata()โš ๏ธNot on the interface: topic() / partition() / offset() report the record's source coordinates directly, following the callback they are called in โ€” past an in-memory repartition the source record's, where Kafka Streams reports its repartition topic's; "" / -1 / -1 where Kafka Streams' recordMetadata() is empty (init(), a punctuator), in onTimer, and for what those callbacks forward
applicationId()โœ…The configured application.id, on every context; under the TopologyTestDriver, the driver config's id
currentSystemTimeMs()โœ…Wall-clock time via System.currentTimeMillis(); under the TopologyTestDriver, the driver's wall clock, as in Kafka Streams
currentStreamTimeMs()โœ…Delegates to currentWatermarkMs() (global watermark)
currentWatermarkMs()๐Ÿ†•Global watermark in epoch millis
timerService()๐Ÿš€Key-based timer access (Flink-inspired)

Processor interfaces

TypeStatusNotes
Processor<KIn,VIn,KOut,VOut>โœ…Key-changing processor interface
FixedKeyProcessor<KIn,VIn,VOut>โœ…Key-preserving processor interface
ContextualProcessor<KIn,VIn,KOut,VOut>โœ…Abstract base managing ProcessorContext
ContextualFixedKeyProcessor<KIn,VIn,VOut>โœ…Abstract base managing FixedKeyProcessorContext
ProcessorSupplier<KIn,VIn,KOut,VOut>โœ…Factory for Processor instances
ProcessorSupplier.stores()โœ…KS 4.x pattern for declaring stores inline
FixedKeyProcessorSupplier<KIn,VIn,VOut>โœ…Factory for FixedKeyProcessor instances
ValueTransformerWithKey<K,V,VR>โœ…Consumed by KTable.transformValues; init binds StoatFlow's FixedKeyProcessorContext
ValueTransformerWithKeySupplier<K,V,VR>โœ…Factory for ValueTransformerWithKey; declares stores via stores(). As a KTable.transformValues parameter it carries Kafka Streams' ? super K, ? super V, ? extends VR wildcards
ValueTransformer / Transformer (+ suppliers)โ†˜๏ธDeprecated in KS and removed from KStream; kept as nameable types for source compatibility
init() / close() lifecycleโœ…Once per task โ€” init() at task start, before any records (application start, an HA pod's activation, TopologyTestDriver construction), close() once at shutdown (KS contract), on every Processor API route (process / processValues / transformValues / addProcessor). A punctuator scheduled in init() starts without input. A global or read-only store's update processor follows the route that added it: through Topology.addGlobalStore / addReadOnlyStateStore it initialises at task start, like addProcessor; through StreamsBuilder.addGlobalStore / addReadOnlyStateStore it initialises on each lane's first record
FixedKeyProcessor.onTimer + schedule() / timerService()โœ…Full Processor timer parity for FixedKeyProcessor (processValues): periodic punctuators via schedule() and keyed timers via timerService() โ†’ onTimer(...) (key-preserving FixedKeyTimerContext, full state access, re-registration). Record timestamp is the event time

Punctuators and timers

Type / MethodStatusNotes
PunctuationType.STREAM_TIMEโœ…Fires on event-time watermark advancement
PunctuationType.WALL_CLOCK_TIMEโœ…Fires on system clock
PunctuatorMode.BLOCKING๐Ÿš€Barriers wait for punctuator completion before commit
PunctuatorMode.NON_BLOCKING๐Ÿš€Barriers proceed independently (default)
Punctuator.punctuate(timestamp)โœ…Callback invoked at scheduled time
Cancellable.cancel()โœ…Cancel scheduled punctuator or timer
TimerService.registerEventTimeTimer(key, timestamp)๐Ÿš€Fire when watermark โ‰ฅ timestamp
TimerService.registerProcessingTimeTimer(key, timestamp)๐Ÿš€Fire at wall-clock time
TimerService.deleteEventTimeTimer(key, timestamp)๐Ÿš€Cancel event-time timer
TimerService.deleteProcessingTimeTimer(key, timestamp)๐Ÿš€Cancel processing-time timer
TimerService.registerEventTimeTimer(key, name, timestamp)๐Ÿš€Named timer: one per (key, name) across both domains โ€” registering the name again replaces its pending timer, at another time or a processing-time one
TimerService.registerProcessingTimeTimer(key, name, timestamp)๐Ÿš€Named processing-time timer โ€” replaces the name's pending timer, also an event-time one
TimerService.deleteTimer(key, name)๐Ÿš€Delete a named timer by name, whatever its domain
TimerService.deleteAllNamedTimers(key)๐Ÿš€Delete every named timer of a key; unnamed timers stay
TimerService.namedTimer(key, name)๐Ÿš€The pending NamedTimer (timeNotion(), timestamp()), or null โ€” also for the firing timer inside its own onTimer, until that call re-arms it
TimerService.registerEventTimeTimer(key, name, timestamp, headers) / registerProcessingTimeTimer(key, name, timestamp, headers)๐Ÿš€A timer with headers of its own, upserted by key into the ones it captures from the arming record (timer headers)
TimerContext.timerName() / FixedKeyTimerContext.timerName()๐Ÿš€The fired timer's name, null for an unnamed timer
TimerContext.headers() / FixedKeyTimerContext.headers()๐Ÿš€The fired timer's headers โ€” the arming record's, plus those its registration passed; a fresh copy per fire; a record onTimer forwards with forward(key, value) (fixed-key: forward(value)) carries them, a forwarded Record keeps its own headers, as in process()
TimerService.currentWatermark()๐Ÿš€Current event-time watermark
TimerService.currentProcessingTime()๐Ÿš€Current wall-clock time (the driver's wall clock under the TopologyTestDriver)
Kafka Streams punctuators are partition-scoped with writable state and always non-blocking. StoatFlow punctuators run globally (topology-wide state view) on a dedicated punctuation lane per sub-topology, with full read/write state access; a punctuation lane writes only after every record lane of its sub-topology has crossed the commit barrier, so its writes and the record lanes' are never reordered across a commit, under either guarantee. PunctuatorMode and the key-based TimerService (which fires in the key's affinity lane, also with full read/write state access) are StoatFlow extensions.

Scheduled sources (StoatFlow extension)

A ScheduledEmitter โ€” registered via StreamsBuilder.scheduled(...) โ€” periodically emits records (interval- or cron-based) using the ScheduledEmitterContext below, without consuming from Kafka.

ScheduledEmitterContext

MethodStatusNotes
forward(key, value)๐Ÿš€Emit record stamped with the fire time โ€” the watermark that fired a STREAM_TIME source, the wall-clock time otherwise
forward(key, value, timestamp)๐Ÿš€Emit record with explicit timestamp
forward(record)๐Ÿš€Emit record with key, value, timestamp, headers
getStateStore(name)๐Ÿš€Access a state store read-only โ€” writes throw UnsupportedOperationException. Every family is covered (plain, timestamped, versioned, headers-aware) and the view keeps the store's declared type, so a getStateStore<TimestampedKeyValueStore<..>> cast still succeeds
currentWatermarkMs()๐Ÿ†•Current watermark (global event-time progress)
currentStreamTimeMs()โœ…Delegates to currentWatermarkMs() (KS compat)
currentSystemTimeMs()โœ…Wall-clock time via System.currentTimeMillis() (KS compat); under the TopologyTestDriver, the driver's wall clock, the same as currentWallClockTime()
currentWallClockTime()๐Ÿš€Current wall-clock time

CronExpression

MethodStatusNotes
unix(expression)๐Ÿš€Unix cron (5 fields: min hour dom mon dow)
quartz(expression)๐Ÿš€Quartz cron (6 fields with seconds)
spring(expression)๐Ÿš€Spring cron (6 fields with seconds)
parse(expression, cronType)๐Ÿš€Parse with explicit type
nextExecutionTime(afterMs, zoneId?)๐Ÿš€Calculate next fire time
timeUntilNextExecution(fromMs, zoneId?)๐Ÿš€Duration until next fire
Cron-based scheduling always uses wall-clock time.

Functional interfaces

KS-compatible functional interfaces backing the Java-friendly DSL overloads.

InterfaceStatusUsed in
KeyValueMapper<K, V, VR>โœ…map, flatMap, groupBy, FK joins
ValueMapper<V, VR>โœ…KTable.mapValues, FK join extractors
ValueMapperWithKey<K, V, VR>โœ…mapValues with key access
Predicate<K, V>โœ…filter, filterNot
ForeachAction<K, V>โœ…peek, forEach
Initializer<VA>โœ…All aggregate()
Aggregator<K, V, VA>โœ…All aggregate() โ€” incl. both the adder and subtractor of KGroupedTable.aggregate() (KS uses Aggregator for both; no Subtractor type)
Reducer<V>โœ…All reduce()
Merger<K, V>โœ…Session window aggregate()
ValueJoiner<V1, V2, VR>โœ…All joins
ValueJoinerWithKey<K, V1, V2, VR>โœ…Key-aware joins. In a GlobalKTable join the key is the stream key, not the join key โ€” those overloads are deprecated since Kafka Streams 4.4
ValueJoinerWithStreamAndMappedKey<StreamKey, TableKey, StreamValue, TableValue, VOut>โœ…Kafka Streams 4.4 (KIP-1340): KStreamโ€“GlobalKTable joins, apply(streamKey, mappedKey, value1, value2). On a left join the mapped key and the table value may be null, so a Kotlin left-join joiner types them nullable โ€” one declared with non-null type arguments compiles on Kafka Streams (Java platform types) but not here. A null-value record is dropped before the key selector runs (Kafka Streams runs it first)
StreamPartitioner<K, V>โœ…Kafka Streams 4.4 shape (KIP-1321): the default partitions(topic, key, value, headers, numPartitions) is what StoatFlow calls, with the record's headers; the KIP-837 4-arg partitions(...) stays the single abstract method, deprecated for removal. Sink multicast matches KS; the in-memory repartition path throws on multicast (single-instance) and passes the lane count. Lives in the topology package

Configuration helper classes

Consumed

MethodStatusNotes
with(keySerde, valueSerde)โœ…
with(keySerde, valueSerde, timestampExtractor, resetPolicy)โœ…Extractor adapted to a stream-time watermark strategy
keySerde(serde)โœ…
valueSerde(serde)โœ…
as(name)โœ…
withWatermarkStrategy(strategy)๐Ÿš€StoatFlow extension (mutually exclusive with withTimestampExtractor)
withTimestampExtractor()โœ…KS TimestampExtractor adapted to a per-source watermark strategy that assigns event time and emits a monotonous watermark โ€” the highest timestamp seen less 1 ms, so the source's watermark follows its newest record as Kafka Streams' partition time does. Windows close and judge lateness on the global watermark, the minimum across partitions: as in KS for a single partition, and with several once the slowest has passed them. Use withWatermarkStrategy for bounded-out-of-orderness/idleness. The extractor receives the real partition time (restored from the committed offsets after a restart)
negative event times๐Ÿš€Every event time but Long.MIN_VALUE is valid end to end, as in Flink โ€” Kafka Streams fails the application on a negative one by default (FailOnInvalidTimestamp) and otherwise drops it. Windows, joins, onWindowClose() and the Processor API handle it; a sink writes the record without a timestamp (the producer or broker stamps the wall-clock time, counted as stoatflow.sink.records.negative.timestamp.total). A missing time โ€” Kafka's -1 on a source with no extractor or assigner, or Long.MIN_VALUE from one โ€” is skipped and counted (dropped.records{reason=no_event_time}). To keep the KS refusal, use the built-in extractors โ€” see Negative event time under the architectural differences (KSC-138)
withOffsetResetPolicy()โœ…Per-source offset reset. AutoOffsetReset: Earliest / Latest / None / byDuration(Duration) (KIP-1106 โ€” seek to the first offset at/after now โˆ’ duration)
withNumberOfLanes(int)๐Ÿ†•Sets this source chain's lane count (parallelism)
withLaneQueueCapacity(int)๐Ÿ†•Sets this source chain's per-lane queue capacity

Event-time progress is driven by a WatermarkStrategy. A KS Consumed.withTimestampExtractor(...) is supported and adapted to one under the hood; Consumed.withWatermarkStrategy(...) is the richer StoatFlow-native form (bounded out-of-orderness, idleness, alignment).

MethodStatusNotes
forBoundedOutOfOrderness(maxOutOfOrderness)๐Ÿš€Bounded out-of-orderness
forMonotonousTimestamps()๐Ÿš€Strictly increasing timestamps
noWatermarks()๐Ÿš€Processing-time only
withTimestampAssigner(assigner)๐Ÿš€Custom event-time assignment
withIdleness(duration)๐Ÿš€Idle-source handling
withWatermarkAlignment(group, maxDrift, updateInterval)๐Ÿš€Flink FLIP-217 alignment, Flink's signature. A partition whose watermark drifts beyond the group cap is held, never paused: it keeps being fetched, its records go only in timestamp order, and while a lagging partition has nothing buffered the whole application waits (the drift bound applies against empty laggards; strict order otherwise). Requires idleness on the group's sources. See Watermark alignment

The TimestampExtractor SAM is present with the KS-exact, non-generic signature long extract(ConsumerRecord<Object, Object>, long), and Consumed.withTimestampExtractor(...) wires it to event-time + stream-time watermarks.

KS-optional builder arguments (serdes, grace, partitioner, offset-reset, name) across Consumed/Grouped/Produced/Repartitioned/Joined/StreamJoined/Materialized/Branched accept null (= "use the default"), matching Kafka Streams; Named itself still requires a non-null name.

Built-in timestamp extractors

Kafka Streams' built-in extractors ship in io.stoatflow.core.processor under their Kafka Streams names, so a ported default.timestamp.extractor or Consumed.withTimestampExtractor(...) keeps its meaning after the import swap. default.timestamp.extractor also accepts their Kafka Streams class names (org.apache.kafka.streams.processor.*, a string or a Class) and loads StoatFlow's, so a ported Properties works unchanged. As in Kafka Streams, the first three treat every negative record timestamp as invalid โ€” they are how a port keeps Kafka Streams' refusal of negative times:

ExtractorStatusNotes
FailOnInvalidTimestampโœ…Fails the application on a negative record timestamp โ€” Kafka Streams' default default.timestamp.extractor
LogAndSkipOnInvalidTimestampโœ…Skips a record with a negative timestamp, with a WARN; counted as dropped.records{reason=no_event_time} and its offset committed. Its onInvalidTimestamp returns Long.MIN_VALUE (StoatFlow's "no event time") where Kafka Streams returns the negative timestamp for its record queue to drop โ€” the record is skipped either way
UsePartitionTimeOnInvalidTimestampโœ…Replaces a negative record timestamp with the partition time; fails while the partition time is unknown, as in Kafka Streams
WallclockTimestampExtractorโœ…The current wall-clock time as event time

Every extractor receives the real partition time: the highest event time extracted from the record's partition before it, restored from the committed offsets after a restart.

Grouped

MethodStatusNotes
as(name)โœ…
keySerde(serde)โœ…
valueSerde(serde)โœ…
with(keySerde, valueSerde)โœ…
with(name, keySerde, valueSerde)โœ…
withName(name)โœ…
withKeySerde(serde)โœ…
withValueSerde(serde)โœ…
withNumberOfLanes(int)๐Ÿ†•Sets the grouped sub-topology's lane count
withLaneQueueCapacity(int)๐Ÿ†•Sets the grouped sub-topology's per-lane queue capacity

Produced

MethodStatusNotes
with(keySerde, valueSerde)โœ…
keySerde(serde)โœ…
valueSerde(serde)โœ…
as(name)โœ…
streamPartitioner(partitioner)โœ…

Materialized

MethodStatusNotes
as(storeName)โœ…
as(storeSupplier)โœ…
as(DslStoreSuppliers)โœ…Store type supplier. Applies only when no explicit supplier is set โ€” an explicit as(supplier) wins (Kafka Streams' order since KSC-100, 2026-09-09)
with(keySerde, valueSerde)โœ…
keySerde(serde)โœ…
valueSerde(serde)โœ…
withCachingEnabled()โœ…Sets the store's cache caps (default caps, or EmissionSuppression limits). Caching is always on and never changes downstream emissions โ€” every update is forwarded (KSC-88; consolidate with Suppressed.untilBarrier())
withCachingDisabled()โœ…Accepted, changes nothing: caching cannot be turned off and the store keeps the default caps; every update is forwarded anyway (KSC-88)
withLoggingEnabled()โœ…Enable changelog with optional topic config
withLoggingDisabled()โœ…Disable changelog for this store
withRetention()โœ…Override computed retention for windowed stores. It is a minimum โ€” a persistent store keeps up to retention + segmentInterval; see window and session retention
withStoreType(DslStoreSuppliers)โœ…Store type selection. Keeps an explicit supplier, which wins over it (Kafka Streams' order since KSC-100); both set โ†’ one WARN per store at build
withRecordHeaders()๐Ÿ†•Persist the record's headers alongside key/value in the store (KIP-1271 / KIP-1285). Its serdes then see headers as in Kafka Streams 4.4 โ€” a write's key and value with the value's headers, lookups with the current record's, iterators with each entry's stored headers โ€” always on a copy, so the record's own headers never gain a store's schema id. A store without record headers calls its serdes with empty headers (Kafka Streams: the current record's), so a schema-registry serde in header mode needs this option
withoutRecordHeaders()๐Ÿ†•Disable header persistence (default)
withChangelogNumPartitions(n)๐Ÿ†•Per-store changelog partition count. No KS equivalent โ€” KS co-partitioning forces the count to equal the source topic's, so it is never a decision there. Create-time immutable, and bidirectional: with the default at 3, pinning a small store back to 1 is the common use. On a foreign-key join the internal -subscription-store inherits it too

Branched

MethodStatusNotes
as(name)โœ…
withName(name)โœ…
withFunction(chain)โœ…
withConsumer(consumer)โœ…Java Consumer overload

Named

Named is open/subclassable, so KSML's name-validation subclasses work.

MethodStatusNotes
as(name)โœ…
withName(name)โœ…Fluent builder form
name()โœ…Read the configured name

Printed

MethodStatusNotes
toSysOut()โœ…Default stdout
toFile(filePath)โœ…File output
withLabel(label)โœ…Prefix label
withKeyValueMapper(mapper)โœ…Custom formatting; takes a KS KeyValueMapper<K, V, String>
withName(processorName)โœ…Set processor name

Joined

Joined<K, VLeft, VRight> is the Kafka Streams generic form. Its serdes and grace period are advisory โ€” stored and round-tripped for source compatibility, but StoatFlow resolves the effective serdes/grace from the upstream Consumed/Grouped configuration.

MethodStatusNotes
as(name)โœ…
create()โœ…Create with defaults
with(keySerde, valueSerde, otherValueSerde)โœ…Static factory; serdes advisory
withName(name)โœ…
withGracePeriod(duration)โœ…Advisory grace period
withKeySerde(serde)โœ…Advisory
withValueSerde(serde)โœ…Advisory
withOtherValueSerde(serde)โœ…Advisory
name()โœ…Read the configured name
gracePeriod()โœ…Round-trips the advisory grace period
keySerde() / valueSerde() / otherValueSerde()โœ…Round-trip the advisory serdes

TableJoined

MethodStatusNotes
as(name)โœ…
with()โœ…Defaults
with(partitioner, otherPartitioner)โœ…Accepted no-op stub (partitioning is N/A single-instance)
withName(name)โœ…
withPartitioner()โœ…Accepted no-op stub for KS source compatibility
withOtherPartitioner()โœ…Accepted no-op stub for KS source compatibility

JoinWindows

MethodStatusNotes
of(timeDifference)โ†˜๏ธDeprecated; use ofTimeDifferenceWithNoGrace
ofTimeDifferenceWithNoGrace(timeDifference)โœ…Symmetric window, no grace
ofTimeDifferenceAndGrace(timeDifference, grace)โœ…Symmetric window with grace
before(timeDifference)โœ…Asymmetric window (look back)
after(timeDifference)โœ…Asymmetric window (look forward)
grace(duration)โ†˜๏ธDeprecated; use ofTimeDifferenceAndGrace
size()โœ…Window size in ms
gracePeriodMs()โœ…Grace period in ms
beforeMs / afterMs are exposed as read-only properties (getBeforeMs() / getAfterMs() from Java; before / after return Duration). A KS port that reads the public jw.beforeMs field replaces it with the accessor.

StreamJoined

MethodStatusNotes
as(name)โœ…Create with processor name
with(keySerde, leftValueSerde, rightValueSerde)โœ…Create with Serdes
create()โœ…Create with defaults
withName(name)โœ…Set processor name
withKeySerde(serde)โœ…Set key Serde
withLeftValueSerde(serde)โœ…Set left stream value Serde
withRightValueSerde(serde)โœ…Set right stream value Serde
withThisStoreSupplier(storeSupplier)โœ…Custom left window store. Must retain duplicates โ€” a non-duplicate supplier is rejected when join() is called (KSC-99), as Kafka Streams rejects it. Its retention must cover the join (before + after + grace), or it is rejected there too; Kafka Streams also requires that exact retention and a window size of before + after, where StoatFlow accepts a longer retention and any window size. Any Stores window supplier is accepted, as in Kafka Streams: a timestamped or headers-aware one builds a plain window store with the same backend, name, retention and window size, since the join keeps its serializers' headers inside the stored values (KSC-129). Wins over withDslStoreSuppliers for its side and no longer clears it (Kafka Streams' order since KSC-100)
withOtherStoreSupplier(storeSupplier)โœ…Custom right window store; same KSC-99 guard, same supplier handling, same KSC-100 precedence
withLoggingEnabled()โœ…Enable changelog logging
withLoggingEnabled(config)โœ…Enable changelog logging with topic config
withLoggingDisabled()โœ…Disable changelog logging
withDslStoreSuppliers(suppliers)โœ…Each join store without an explicit supplier uses this type โ€” an explicit supplier wins for its side and neither setter clears the other (Kafka Streams' order since KSC-100, 2026-09-09). Both retain duplicates (fixed 2026-09-03 โ€” before that this path silently built a non-duplicate join store); a custom DslStoreSuppliers that ignores DslWindowParams.retainDuplicates is rejected at DSL build (KSC-99)
withStoreName(name)โœ…Set base store name
withChangelogNumPartitions(n)๐Ÿ†•Changelog partition count for this join's stores. No KS equivalent. The number is multiplied โ€” one call sets it on 2 stores (INNER) or 3 (LEFT/OUTER, adding -outer-store), so (6) is 12 or 18 partitions

Repartitioned

MethodStatusNotes
as(name)โœ…
streamPartitioner(partitioner)โœ…numPartitions = numLanes
withStreamPartitioner(partitioner)โœ…
withName(name)โœ…
with(keySerde, valueSerde)โœ…keySerde used for lane assignment hashing
withKeySerde()โœ…Used for lane assignment hashing at sub-topology boundary
withValueSerde()โœ…Reserved for future use
withNumberOfLanes(int)๐Ÿ†•Sets the repartition sub-topology's lane count
numberOfLanes(int)๐Ÿ†•Static factory; sets the lane count
withLaneQueueCapacity(int)๐Ÿ†•Sets the per-lane queue capacity
withNumberOfPartitions()๐Ÿ†•KS-compat alias of withNumberOfLanes (sets the lane count)
numberOfPartitions(int)๐Ÿ†•KS-compat static-factory alias of numberOfLanes

EmitStrategy

MethodStatusNotes
onWindowUpdate()โœ…Default; emit on every update
onWindowClose()โœ…Emit only when window closes; driven by the global event-time watermark
OnWindowClose emission is watermark-driven: a window's final result is emitted when the global watermark passes the window end plus any configured grace period. See Event time and watermarks.

Suppressed

MethodStatusNotes
untilWindowCloses(bufferConfig)โœ…For windowed KTables; requires a StrictBufferConfig (KS parity); flush driven by the global event-time watermark. A final result keeps its latest headers and timestamp; a null final result is dropped (KS parity). The grace period comes from the window definition upstream, through any operators in between (the largest across a merge); with no windowed ancestor the build fails with Kafka Streams' "Window close time is only defined for windowed computations"
untilTimeLimit(duration, bufferConfig)โš ๏ธRate-limits per-key updates with KS semantics โ€” every update buffered, the wait counted from a key's first update. โš ๏ธ The wait is measured on the watermark, not the task's stream time: Duration.ZERO emits on the next watermark advance, and a release trails by the out-of-orderness bound
withName(name)โœ…KS NamedOperation; names the suppression node
Suppression buffers are flushed on watermark advancement โ€” untilWindowCloses releases a key's final value once the window closes, and untilTimeLimit releases the latest buffered value once the per-key time limit elapses in event time. See Event time and watermarks. Suppress buffers (and versioned stores, which the migration tool supports as experimental) are the two store families with state-carrying migration caveats.
KIP-914:suppress() throws a validation exception for tables backed by versioned stores โ€” their temporal semantics conflict with suppression.

BufferConfig

MethodStatusNotes
unbounded()โœ…No limits (use with caution)
maxRecords(count)โœ…Bounded by record count
maxBytes(bytes)โœ…Bounded by the buffer's serialized size, headers included (KS estimates memory); checked after each update, as in KS
withMaxRecords(count)โœ…Fluent record bound
withMaxBytes(bytes)โœ…Fluent byte bound
withNoBound()โœ…Remove bounds
emitEarlyWhenFull()โœ…Emit instead of evict on overflow
shutDownWhenFull()โœ…Shut down on buffer overflow
withLoggingEnabled(config) / withLoggingDisabled()โœ…Changelog control for the suppress buffer
withChangelogNumPartitions(n) / changelogNumPartitions()๐Ÿ†•Changelog partition count for the buffer store. No KS equivalent; returns BC, preserving the self-type. The buffer follows the global changelog.num-partitions by default, like every other store; set this only on a measurement

WindowedSerdes

MethodStatusNotes
timeWindowedSerdeFrom(...)โš ๏ธTakes Serde<T> instead of Class<T> (no serde registry)
timeWindowedSerdeFrom(..., windowSize)โš ๏ธTakes Serde<T> only; window size encoded in serialized bytes
sessionWindowedSerdeFrom(...)โš ๏ธTakes Serde<T> instead of Class<T>
TimeWindowedSerde(inner)โœ…WindowedSerdes.TimeWindowedSerde(serde)
TimeWindowedSerde(inner, windowSize)โš ๏ธWindow size not required (encoded in serialized bytes)
SessionWindowedSerde(inner)โœ…WindowedSerdes.SessionWindowedSerde(serde)
StoatFlow's factory methods take Serde<T> rather than KS's Class<T> (there is no serde registry), and omit the window-size parameter (window bounds are encoded in the serialized bytes). TimeWindowedSerde.forChangelog() is not needed โ€” StoatFlow uses identical encoding for client and changelog.

State stores

Record types & iterators

TypeStatusNotes
Windowed<K>โœ…Windowed-key wrapper (key + Window)
ValueAndTimestamp<V>โœ…Timestamped-store value wrapper (KIP-258). Null semantics as in Kafka Streams: make(null, ts) returns null, makeAllowNullable an instance; a null value is written as a delete and a null-deserialized read returns null
VersionedRecord<V>โœ…Versioned-store record; validTo() returns Optional<Long>
KeyValueIterator / WindowStoreIteratorโœ…Closeable iterators for range/scan queries; use after close() throws InvalidStateStoreException, as in Kafka Streams โ€” and so does any read on a closed store
ReadOnlyRecord<K, V>โœ…KIP-1356: the read-only record a headers-aware query returns (key, value, timestamp, headers), in io.stoatflow.core.processor โ€” the migration recipe rewrites Kafka Streams' processor.api import. Record implements it, and so does FixedKeyRecord (Kafka Streams' does not). Its timestamp may be negative (StoatFlow accepts negative event times)
ReadOnlyRecordIterator<K, V>โœ…KIP-1356: a closeable iterator of ReadOnlyRecords; remove() unsupported

State store metrics

MetricStatusNotes
num-keys (KIP-1250)โœ…Entry-count gauge for in-memory KV/window/session stores
uncommitted-bytes (KIP-892)โœ…stoatflow.store.uncommitted.bytes, INFO level, as in Kafka Streams. Emitted for every store family, where Kafka Streams registers it on key-value stores only (KSC-118); the application-wide total is stoatflow.barrier.epoch.uncommitted.bytes

StateStore

MethodStatusNotes
approximateNumUncommittedBytes()โœ…KIP-892 (Kafka Streams 4.4); defaults to 0 for a custom store. Counts the bytes held in memory for epochs not yet written to RocksDB, including the committed epoch still being written, which READ_COMMITTED can already read โ€” memory, not visibility (KSC-117). A custom store a builder of your own returns is not counted towards the memory limit, whatever it reports: no commit reaches it, so its bytes could never drain

KeyValueStore

MethodStatusNotes
get(key)โœ…
put(key, value)โœ…
putIfAbsent(key, value)โœ…
putAll(entries)โœ…
delete(key)โœ…Reads and returns the previous value in both engines โ€” a read-then-write. Prefer put(key, null) when you do not need it: Deleting a key
compute(key, remappingFunction)๐Ÿš€Atomic read-compute-write; Kotlin lambda + Java BiFunction overloads
merge(key, value, remappingFunction)๐Ÿš€Atomic merge; Kotlin lambda + Java BiFunction overloads
all()โœ…Read-your-writes semantics
range(from, to)โœ…Read-your-writes semantics. Null bounds = open-ended (KIP-763, KSC-89); from > to returns empty + WARN (KS-lenient)
reverseAll()โœ…Read-your-writes semantics
reverseRange(from, to)โœ…Read-your-writes semantics. Null bounds = open-ended (KIP-763, KSC-89); from > to returns empty + WARN (KS-lenient); (low, high) bounds order, KS-exact (KSC-90 โ€” was (high, low) pre-2026-07-31)
approximateNumEntries()โœ…
prefixScan()โœ…KIP-614; read-your-writes semantics

ReadOnlyKeyValueStore

MethodStatusNotes
get(key)โœ…
containsKey(key)๐Ÿš€StoatFlow extension; bloom filter optimization for RocksDB
all()โœ…
range(from, to)โœ…Null bounds = open-ended (KIP-763, KSC-89); from > to returns empty + WARN (KS-lenient)
reverseAll()โœ…
reverseRange(from, to)โœ…Null bounds = open-ended (KIP-763, KSC-89); from > to returns empty + WARN (KS-lenient); (low, high) bounds order, KS-exact (KSC-90 โ€” was (high, low) pre-2026-07-31)
approximateNumEntries()โœ…
prefixScan(prefix, serializer)โœ…KIP-614
readOnly(isolationLevel)โœ…KIP-892 (Kafka Streams 4.4). READ_COMMITTED reads the state of the last commit barrier; READ_UNCOMMITTED returns the store itself. Iterators over an in-memory store are weakly consistent: never an uncommitted row, but an iterator held open can see later commits. An lruMap, an in-memory versioned store, or a custom store with no caching layer keeps no commit buffer and answers live, with a warning (KSC-116, KSC-106). Isolation level

WindowStore

Key-range bounds are compared as serialized bytes (unsigned lexicographic); null bounds are open-ended (KIP-763, KSC-91). All Instant variants delegate to the Long variants.

MethodStatusNotes
put(key, value, windowStartTime)โœ…
fetch(key, windowStartTime)โœ…Point lookup
fetch(key, timeFrom, timeTo)โœ…Single key time range; Long and Instant variants
backwardFetch(key, timeFrom, timeTo)โœ…Reverse iteration; Long and Instant variants
fetch(keyFrom, keyTo, timeFrom, timeTo)โœ…Key range + time range; Long and Instant variants. Null key bounds = open-ended (KIP-763, KSC-91); keyFrom > keyTo returns empty + WARN (KS-lenient)
backwardFetch(keyFrom, keyTo, timeFrom, timeTo)โœ…Reverse key range; bounds stay (low, high) like fetch (KS-exact); Long and Instant variants. Null key bounds = open-ended (KIP-763, KSC-91)
fetchAll(timeFrom, timeTo)โœ…All keys time range; Long and Instant variants
backwardFetchAll(timeFrom, timeTo)โœ…Reverse all keys; Long and Instant variants
all()โœ…All entries
backwardAll()โœ…Reverse iteration
windowSizeMs๐Ÿš€StoatFlow extension for window size access
retentionMs๐Ÿš€StoatFlow extension for retention access
expireWindows(watermark)๐Ÿš€StoatFlow extension for manual expiration
approximateNumEntries()๐Ÿš€StoatFlow extension for entry count

ReadOnlyWindowStore

MethodStatusNotes
All query methods from WindowStoreโœ…16 query methods total
containsKey(key, windowStartTime)๐Ÿš€StoatFlow extension; Long and Instant variants
readOnly(isolationLevel)โœ…KIP-892 (Kafka Streams 4.4). READ_COMMITTED reads the state of the last commit barrier; READ_UNCOMMITTED returns the store itself. Iterators over the default segmented store, or an in-memory one, are weakly consistent: never an uncommitted row, but an iterator held open can see later commits. An lruMap, an in-memory versioned store, or a custom store with no caching layer keeps no commit buffer and answers live, with a warning (KSC-116, KSC-106). Isolation level

SessionStore

MethodStatusNotes
put(sessionKey, aggregate)โœ…Windowed key
remove(sessionKey)โœ…Windowed key
fetchSession(key, startTime, endTime)โœ…Long and Instant variants
findSessions(key, earliestEndTime, latestStartTime)โœ…Long and Instant variants
backwardFindSessions(key, earliestEndTime, latestStartTime)โœ…Long and Instant variants
findSessions(keyFrom, keyTo, earliestEndTime, latestStartTime)โœ…Key range; Long and Instant variants. Null key bounds = open-ended (KIP-763, KSC-91); keyFrom > keyTo returns empty + WARN (KS-lenient)
backwardFindSessions(keyFrom, keyTo, earliestEndTime, latestStartTime)โœ…Key range; Long and Instant variants. Null key bounds = open-ended (KIP-763, KSC-91)
fetch(key)โœ…All sessions for single key
backwardFetch(key)โœ…Reverse order
fetch(keyFrom, keyTo)โœ…Key range queries. Null key bounds = open-ended (KIP-763, KSC-91); keyFrom > keyTo returns empty + WARN (KS-lenient)
backwardFetch(keyFrom, keyTo)โœ…Reverse order; bounds stay (low, high) (KS-exact). Null key bounds = open-ended (KIP-763, KSC-91)
findAll()๐Ÿš€StoatFlow extension
expireSessions(watermark)๐Ÿš€StoatFlow extension
approximateNumEntries()๐Ÿš€StoatFlow extension

ReadOnlySessionStore

MethodStatusNotes
All query methods from SessionStoreโœ…Query methods for IQ access
containsSession(key, startTime, endTime)๐Ÿš€StoatFlow extension; Long and Instant variants
readOnly(isolationLevel)โœ…KIP-892 (Kafka Streams 4.4). READ_COMMITTED reads the state of the last commit barrier; READ_UNCOMMITTED returns the store itself. Iterators over the default segmented store, or an in-memory one, are weakly consistent: never an uncommitted row, but an iterator held open can see later commits. An lruMap, an in-memory versioned store, or a custom store with no caching layer keeps no commit buffer and answers live, with a warning (KSC-116, KSC-106). Isolation level

Stores factory

StoatFlow's suppliers implement the KS *BytesStoreSupplier interfaces and the *StoreBuilder(...) factories return a KS StoreBuilder<T>. Some signatures take a Serde<T> where KS takes Class<T> (no serde registry); StoreBuilder.withCaching*/withLogging* are honored, and build() throws (StoatFlow builds stores via the registry). A custom StateStoreBuilder whose build() returns a persistent supplier's get() is refused at startup (KSC-124): register the supplier through Stores.keyValueStoreBuilder(supplier, โ€ฆ) (or the window/session equivalent) or Materialized.as(supplier).

MethodStatusNotes
persistentKeyValueStore(name)โœ…RocksDB key-value store
inMemoryKeyValueStore(name)โœ…In-memory key-value store
persistentWindowStore(name, retention, windowSize, retainDuplicates)โœ…RocksDB window store. Segments are sized max(retention / 3, 60s, windowSize) (KS: max(retention / 2, 60s)), with no way to configure it, and expiry drops whole segments โ€” so effective retention is up to 4/3ร— what you configure where the divisor governs (retention โ‰ฅ 180s), up to retention + 60s where the 60s floor governs (see the windowing docs), and up to 2ร— where windowSize governs on the segmented backend, which is the common case since ofSizeWithNoGrace makes retention == windowSize (retention is a minimum bound, as in Kafka Streams, whose own slack is up to 3/2ร—; in-memory stores retain exactly 1ร—). Changing retention on an existing segmented store is safe โ€” each segment records its own time range on disk, so the change remaps rather than being reinterpreted; Kafka Streams requires a wipe. Only a hand-built non-segmented store (no Stores factory builds one) keeps the interval in every key, where a retention change that moves it needs a restore. Changing the window size, switching plainโ†”timestamped, or flipping retainDuplicates is detected at startup: a store whose changelog is its recovery source refuses to start, since its changelog holds the old layout and a rebuild would misread it (see Changes a changelog cannot rebuild), and a store with no changelog refuses too, or starts empty under stoatflow.state.layout-mismatch: wipe-and-restore. A stream-stream join buffer's window size is not compared: a join reads its buffers by record timestamp. retainDuplicates=true (the stream-stream join-buffer setting) uses the same segmented backend since 2026-09-03, so its expiry reclaims disk by dropping whole segments too โ€” before that it ran a non-segmented backend whose deleteRange expiry reclaimed nothing until a later background compaction happened to cover the tombstone
inMemoryWindowStore(name, retention, windowSize, retainDuplicates)โœ…In-memory window store. Not segmented: expires per record at exactly retention, and its supplier reports segmentIntervalMs() == 1, as KS's does.
persistentSessionStore(name, retentionPeriod)โœ…RocksDB session store. Segments are sized max(retention / 3, 60s) โ€” a session has no window size, so KIP-1366's >= windowSize term does not apply โ€” not configurable, and effective retention is up to retention + segmentInterval. On a uniform layout StoatFlow never retains more than Kafka Streams here: its divisor is /3 against KS's /2 and both are floored at 60s, so its interval is never the larger of the two โ€” equal while both sit on the floor, tighter once KS leaves it. (Through a retention narrowing that ordering is suspended, since the older, wider segments below are still live.) Bounds assume the default onWindowUpdate(). See the windowing docs for the regimes and worked examples. A retention change remaps rather than requiring a wipe. After a retention narrowing, existing segments keep their original span, so the store frees disk gradually rather than at once
inMemorySessionStore(name, retentionPeriod, expiryIndex)โœ…In-memory session store. Not segmented: expires per record at exactly retention, and its supplier reports segmentIntervalMs() == 1, as KS's does. ๐Ÿ†• The third parameter (default false) opts into an end-time expiry index โ€” no KS equivalent; the two-argument call is unchanged. See expiring sessions from an in-memory store.
inMemorySessionStoreWithHeaders(name, retentionPeriod, expiryIndex)โœ…In-memory headers-aware session store (KIP-1271). ๐Ÿ†• Same expiryIndex third parameter as inMemorySessionStore โ€” it builds the same store class.
persistentTimestampedKeyValueStore(name)โœ…Timestamped KV store (KIP-258)
persistentTimestampedWindowStore(name, retention, windowSize, retainDuplicates)โœ…Timestamped window store (KIP-258); 4-arg retainDuplicates overload supported. Same segment sizing as persistentWindowStore (segmented whatever the retainDuplicates value, since 2026-09-03) and the same effective-retention bounds. Switching a populated store between this and persistentWindowStore changes the value format: a store whose changelog is its recovery source refuses to start (see Changes a changelog cannot rebuild)
inMemoryTimestampedKeyValueStore(name)โœ…In-memory timestamped KV store (KIP-258)
inMemoryTimestampedWindowStore(name, ...)โœ…In-memory timestamped window store (KIP-258). Not segmented: expires per record at exactly retention, and segmentIntervalMs() reports 1, as KS's does โ€” the segmented persistentTimestampedWindowStore keeps up to retention + segmentInterval.
persistentTimestampedWindowStoreWithHeaders(name, retention, windowSize, retainDuplicates)โœ…Headers-aware timestamped window store (KIP-1271), with Kafka Streams 4.4's 4-arg retainDuplicates form; the 3-arg form (false) is StoatFlow's own. A duplicate-retaining one is accepted as a stream-stream join store
inMemoryTimestampedWindowStoreWithHeaders(name, retention, windowSize, retainDuplicates)๐Ÿ†•In-memory headers-aware timestamped window store โ€” no Kafka Streams equivalent. Same retainDuplicates parameter
persistentVersionedKeyValueStore(name, historyRetention)โœ…Versioned store โ€” RocksDB (KIP-889). Point-in-time reads below the retention floor return the latest version if it satisfies the bound, else null โ€” KS-exact. Divergence: the stream time behind the grace window is shared by every lane writing the store, where KS keeps one per task; under exactly_once a write and a read are judged on it as of the lane's own commit. Divergence: retention is a minimum bound. KS drops whole segments; StoatFlow reclaims with a budgeted background sweep (stoatflow.state.versioned.sweep-period-ms), so the on-disk footprint is bounded rather than exact, and a standby applies the active's reclamation tombstones, so its footprint tracks the active within replication lag. Under exactly_once every lane reads it as of its own commit, so any join can read it
persistentVersionedKeyValueStore(name, historyRetention, segmentInterval)โœ…3-arg KS overload; segmentInterval accepted but ignored (not segmented) โ€” logs a warning
WindowBytesStoreSupplier.segmentIntervalMs() / SessionBytesStoreSupplier.segmentIntervalMs()โœ…Reports the real interval for the backend the supplier builds โ€” max(retention / 3, 60s, windowSize) for a segmented window store, max(retention / 3, 60s) for a session store, and 1 for an in-memory one, exactly as Kafka Streams' suppliers do
inMemoryVersionedKeyValueStore(name, historyRetention)โœ…Versioned store โ€” in-memory (KIP-889). ๐Ÿ†• no KS equivalent. Same read contract and sweep as the persistent backend; its observed stream time is not durable, so it restarts unset. No per-commit write buffer: exactly-once through an engine rebuild of an aborted commit, but its reads are live, so it answers READ_COMMITTED live, with a warning (KSC-106), and under exactly_once it is refused as a table a join reads from another lane (KSC-126) โ€” the TopologyTestDriver judges the declared store, so it refuses this one and accepts persistentVersionedKeyValueStore
lruMap(name, maxCacheSize, tombstoneOnEviction)๐Ÿ†•StoatFlow extension: with true, each eviction also writes a changelog delete (needs logging; refused with withLoggingDisabled()). The two-argument form keeps Kafka Streams' behaviour (evictions are local only)
lruMap(name, maxCacheSize)โœ…LRU cache store with bounded entries. No per-commit write buffer (exactly-once through an engine rebuild of an aborted commit); reads are always live, so it answers READ_COMMITTED live, with a warning (KSC-116); refused under exactly_once as a table a join reads from another lane โ€” either side of a primary-key or foreign-key join, a re-keyed stream-table join, a global-table join (KSC-126)
keyValueStoreBuilder(supplier, keySerde, valueSerde)โœ…Returns a KS StoreBuilder
timestampedKeyValueStoreBuilder(supplier, keySerde, valueSerde)โœ…KIP-258
A store switched plain โ†” timestamped, or to another store type, window size or retainDuplicates, under one nameโš ๏ธKafka Streams upgrades a plain RocksDB key-value store reopened as timestamped in place, and checks nothing else: a resized windowed aggregation reads its old windows with the new size. A StoatFlow store whose changelog is its recovery source refuses to start when its value format flips (plain โ†” timestamped), its type changes (key-value โ†” versioned, window โ†” session), its window size changes or its retainDuplicates flips: its changelog holds the old layout, so it cannot rebuild the store. The store is left untouched. Only Processor API stores can flip their value format or retainDuplicates โ€” every DSL store is timestamped whatever its supplier; a DSL windowed aggregation's new window size refuses too, and a stream-stream join's JoinWindows change is adopted, since a join reads its buffers by record timestamp. See Changes a changelog cannot rebuild
windowStoreBuilder(supplier, keySerde, valueSerde)โœ…Returns a KS StoreBuilder
timestampedWindowStoreBuilder(supplier, keySerde, valueSerde)โœ…KIP-258
sessionStoreBuilder(supplier, keySerde, aggSerde)โœ…Returns a KS StoreBuilder
versionedKeyValueStoreBuilder(supplier, keySerde, valueSerde)โœ…KIP-889
timestamped{KeyValue,Window} / sessionStoreWithHeadersBuilder(...)โœ…Headers-aware builders (KIP-1271); pass a *WithHeaders supplier
KeyValue/Window/Session/VersionedBytesStoreSupplier, StoreBuilderโœ…KS supplier/builder interfaces are present

KeyLockManager (StoatFlow extension)

Striped lock utility for multi-key atomic sections in user-defined Processors. Not in Kafka Streams.

MethodStatusNotes
withLock(key, block)๐Ÿš€Single-key lock scope
withLocks(keys, block)๐Ÿš€Multi-key lock scope with deadlock-free ordering

Record headers in state stores (KIP-1271 / KIP-1285)

Opt-in persistence of a record's Headers alongside key/value, across the timestamped KV, versioned, timestamped-window, and session store families.

SurfaceStatusNotes
Materialized.withRecordHeaders()๐Ÿ†•Per-store opt-in (DSL)
dsl.store.format = HEADERSโœ…Global default to headers-aware stores
Stores.*WithHeaders(...) suppliers + *WithHeadersBuilder(...)โœ…Processor-API headers stores (all four families)
*WithHeadersBuilder(...) over a plain or timestamped supplier๐Ÿ›‘Refused when the topology is built (KSC-145); Kafka Streams accepts it and keeps no headers. Pass a Stores.*WithHeaders(...) supplier
QueryableStoreTypes.*WithHeaders()โœ…Read stored headers back via IQ
Headers-aware IQv2 queries (KIP-1356)โœ…Read stored headers back through query() โ€” key-value, window and session stores; see Interactive Queries v2 below
SessionStoreWithHeaders.expireSessions(watermark)๐Ÿš€StoatFlow extension (abstract). StoatFlow expires sessions from the aggregation, not inside the store, so a custom implementation must implement it. Kafka Streams' interface has no such method
In-place upgrade (existing store โ†’ headers-aware)โœ…Flip on, restart โ€” no wipe, no changelog replay
Downgrade refusal (headers-aware โ†’ plain)โœ…Like KS, a downgrade is refused with a diagnosed, actionable message rather than a raw RocksDB error โ€” it is a declared format change, not corruption
state.format.downgrade = wipe-and-restore๐Ÿš€StoatFlow-only: acknowledge the downgrade in config and it rebuilds that store from the changelog. KS offers no equivalent โ€” its only remedy is deleting local state by hand
Empty-keyspace downgrade (nothing written in headers mode)๐Ÿš€Dropped in place, free. KS refuses this case too
Reserved header names (__stoatflow.v, __stoatflow.v.*) and the stoatflow.epoch stampโš ๏ธA headers-aware store does not keep them (KSC-149): the reserved names, kept for StoatFlow's changelog format version, are dropped from the stored headers with one WARN per store naming the header, and the engine's stoatflow.epoch stamp is dropped silently. The restore drops the same names, so a live store and a restored one, a table restored from its source topic included, hold the same headers. Every other header stays, __stoatflow.vendor and the dead-letter __stoatflow.errors.* included. Kafka Streams keeps every header
StreamJoined headersโœ… (differs)KS 4.3+ side buffers follow dsl.store.format: under HEADERS (or in memory) they keep the record's headers plus the serializers' and a header-mode serde works; under the default they drop them and it fails. StoatFlow's side buffers always keep what their serializers wrote, never the record's, so a header-mode serde works on every store type (KSC-129)

Interactive Queries

StoatFlow (entry point) methods

MethodStatusNotes
store(StoreQueryParameters)โœ…Returns a read-only view, as in Kafka Streams, in one of two ways. When the store holds the value shape the matcher promises, the view keeps the store's interface and every write throws UnsupportedOperationException โ€” including through sessionStoreWithHeaders() and versionedKeyValueStoreWithHeaders(). When the matcher converts (keyValueStore() over a timestamped store, the headers-aware twins, sessionStore() over a headers-aware session store), the view implements only the ReadOnly* interface and has no write methods: a cast to a writable store type fails with ClassCastException. Either way the handle looks the store up on every read, so it survives an in-place engine restart; QueryableStoreType.create(StateStoreProvider, String) has Kafka Streams' signature
storeNames()โœ…List available store names
query(StateQueryRequest)โœ…Interactive Queries v2 (KIP-796) โ€” see Interactive Queries v2 below. Gates by Kafka Streams' exclusion list, so it answers on a PAUSED/DRAINING application where store() throws; NOT_ACTIVE while stores restore (KSC-109)
queryMetadataForKey(store, key, serializer)โœ…Returns KeyQueryMetadata โ€” single-instance: the local host, no standbys, the key's faithful Kafka partition
queryMetadataForKey(store, key, headers, serializer)โœ…KIP-1321: the key serialized with the headers, as at the sink
queryMetadataForKey(store, key, headers, partitioner)โœ…KIP-1321: the partition your StreamPartitioner returns; anything but a single partition throws IllegalArgumentException. โš ๏ธ Optional.empty() also throws (KS returns metadata without a usable partition)
streamsMetadataForStore(store)โœ…The single local host materializing the store
metadataForAllStreamsClients()โœ…The one local instance
setUncaughtExceptionHandler(handler)โœ…KS StreamsUncaughtExceptionHandler; REPLACE_THREAD triggers an in-place engine restart (fault-storm-guarded โ€” repeated faults escalate to shutdown), under hot-standby HA as well as without it: the fault is absorbed in place and the pod keeps the active role; SHUTDOWN_CLIENT/SHUTDOWN_APPLICATION shut the single instance down
pause()๐Ÿš€Pause processing with drain semantics
unpause()๐Ÿš€Resume processing after pause
awaitState(state, timeout)๐Ÿš€Wait for target state with timeout
stateTransitionHistory()๐Ÿš€Recent state transitions with timestamps

StoreQueryParameters

MethodStatusNotes
fromNameAndType(name, type)โœ…Factory method
enableStaleStores()โœ…Allow queries during RESTORING
withPartition(partition)๐Ÿ›‘N/A for single-instance

QueryableStoreTypes

MethodStatusNotes
keyValueStore()โœ…Returns ReadOnlyKeyValueStore. Over a timestamped store (the default for grouped and windowed aggregations) or its headers-aware twin it answers with the plain value, as in Kafka Streams
windowStore()โœ…Returns ReadOnlyWindowStore. Over a timestamped window store or its headers-aware twin it answers with the plain value, as in Kafka Streams
sessionStore()โœ…Returns ReadOnlySessionStore. Also accepts a headers-aware session store and answers with the plain aggregate, as in Kafka Streams
timestampedKeyValueStore()โœ…Returns ReadOnlyKeyValueStore<K, ValueAndTimestamp<V>> (KIP-258). Also accepts the headers-aware twin and drops the headers, as in Kafka Streams. Answers every DSL key-value store โ€” table(), toTable(), globalTable(), KTable operators and cogroup aggregates included
timestampedWindowStore()โœ…Returns ReadOnlyWindowStore<K, ValueAndTimestamp<V>> (KIP-258). Also accepts the headers-aware twin and drops the headers, as in Kafka Streams
versionedKeyValueStore()โœ…Returns ReadOnlyVersionedKeyValueStore<K, V> (KIP-889). readOnly(isolationLevel) on the versioned store (KIP-892): READ_COMMITTED answers as of the last commit, validTo included; an in-memory versioned store answers live, with a warning (KSC-106)
StateStoreProvider.defaultIsolationLevel()โœ…Kafka Streams 4.4: the level store() applies โ€” the configured default.interactive.query.isolation.level โ€” for a custom QueryableStoreType to apply with readOnly(level)
The global state model eliminates partition routing โ€” all state is locally accessible from a single process, so withPartition() and inter-instance RPC are unnecessary.

Interactive Queries v2

The org.apache.kafka.streams.query package, name for name, in io.stoatflow.core.query โ€” the import swap ports it. The TopologyTestDriver has query() too, which Kafka Streams' driver does not.

Type / memberStatusNotes
StateQueryRequestโœ…inStore/withQuery, withPositionBound, withAllPartitions, withPartitions, enableExecutionInfo, requireActive, withIsolationLevel (KIP-892) โ€” immutable, as in Kafka Streams
StateQueryResultโœ…One store partition, 0 (KSC-107): the store answers under partition 0, and a request naming another partition gets a DOES_NOT_EXIST entry under that partition; getGlobalResult() is always null. Read it with getPartitionResults().get(StateQueryResult.SINGLE_PARTITION)
StateQueryResult.SINGLE_PARTITION๐Ÿ†•0 โ€” the one store partition
getOnlyPartitionResult()โœ…As in Kafka Streams: null for a successful null answer; throws next to a DOES_NOT_EXIST entry
withPartitions(S) with a partition other than 0โš ๏ธDOES_NOT_EXIST for it โ€” the answer Kafka Streams documents; its code returns NOT_PRESENT, which StoatFlow never returns (KSC-107)
QueryResultโœ…Factories and accessors as in Kafka Streams; an UNKNOWN_QUERY_TYPE message names the store, not its class (KSC-110)
Position / PositionBoundโš ๏ธSame API. Moves once per commit, not per record (KSC-108): the last commit's source-topic offsets, the same for every store, with no repartition topics โ€” except a store with logging disabled and no source topic, which reports only what this instance committed since it started (as Kafka Streams' position for such a store restarts empty). A bound fails on any partition the position does not hold, and query() never waits for it
QueryConfig, FailureReason, ResultOrderโœ…As in Kafka Streams
KeyQuery, RangeQueryโœ…The bare value from timestamped and headers-aware key-value stores, as in Kafka Streams. skipCache() is accepted and changes nothing (KSC-111; Kafka Streams ignores it too)
TimestampedKeyQuery, TimestampedRangeQueryโœ…ValueAndTimestamp from timestamped and headers-aware stores; UNKNOWN_QUERY_TYPE on a plain store
WindowKeyQuery, WindowRangeQueryโœ…Value shapes as in Kafka Streams: a DSL windowed aggregation answers ValueAndTimestamp elements; a session store the aggregate. Final classes, where Kafka Streams leaves them open (KSC-110)
VersionedKeyQuery, MultiVersionedKeyQueryโœ…As in Kafka Streams, including the exceptions for a KeyQuery on a versioned store and an inverted time range; a RocksDB store reads one consistent snapshot, lazily
ReadOnlyVersionedKeyValueStore.get(key, fromTime, toTime, order)๐Ÿ†•The versioned range read behind MultiVersionedKeyQuery, public on the store (Kafka Streams keeps it internal)
When query() answersโš ๏ธKSC-109: an unknown store throws UnknownStateStoreException first, in every state. Not started or stopped: IllegalStateException (Kafka Streams has its own exception types). An HA standby throws, as store() does. While stores restore: NOT_ACTIVE, with or without requireActive(). A retriable InvalidStateStoreException from the read answers NOT_ACTIVE (Kafka Streams: STORE_EXCEPTION)
TimestampedKeyWithHeadersQueryโœ…KIP-1356: a ReadOnlyRecord from a headers-aware timestamped key-value store โ€” key, value, timestamp and frozen headers; null for a missing key or tombstone. A negative stored timestamp is returned (Kafka Streams fails it)
TimestampedKeyWithHeadersQuery.skipCache()โš ๏ธAccepted, changes nothing (KSC-111) โ€” Kafka Streams honours it; use the isolation level
TimestampedRangeWithHeadersQueryโœ…KIP-1356: a ReadOnlyRecordIterator over a key range of a headers-aware key-value store, tombstones omitted
TimestampedWindowKeyWithHeadersQueryโœ…KIP-1356: one key's windows in a closed start range, from a headers-aware window store
TimestampedWindowRangeWithHeadersQueryโœ…KIP-1356: withWindowStartRange from a headers-aware window store, withKey from a headers-aware session store (timestamp = the session's end); each store refuses the other form, as in Kafka Streams
StateStore.query(...) / custom Query types๐Ÿ›‘No per-store query hook; an unknown query type answers UNKNOWN_QUERY_TYPE

IQ metadata (single-instance: always local)

Kafka Streams' host/partition-distribution metadata types exist so cross-instance "find the host, then query" code compiles and runs. Under StoatFlow's single-instance model the answer is always "this host, locally."

TypeStatusNotes
HostInfo(host, port)โœ…The local host; sourced from application.server (synthetic localhost:0 if unset)
StreamsMetadataโœ…hostInfo(), stateStoreNames(), topicPartitions(); no standbys
KeyQueryMetadataโœ…activeHost() = local, no standby hosts, partition() = the key's faithful Kafka partition

Exception handlers

HandlerStatusNotes
Source deserialisation with headersโœ…Source deserializers receive the record headers (deserialize(topic, headers, data)), as in Kafka Streams โ€” header-aware serdes work at the source
DeserializationExceptionHandlerโœ…KS-ordered handle(ErrorHandlerContext, ConsumerRecord, Exception); LogAndFail (default), LogAndContinue, DLQ
ProcessingExceptionHandlerโœ…KS-ordered handle(ErrorHandlerContext, Record, Exception); LogAndFail (default), LogAndContinue, DLQ. A FAIL verdict a processor catches and rethrows wrapped is not offered to the handler again โ€” Kafka Streams re-offers it at the wrapping node
ProductionExceptionHandlerโœ…KS-ordered handle(ErrorHandlerContext, ProducerRecord, Exception); intelligent retry. A send failure no handler answered CONTINUE fails the commit before the offsets, under both guarantees, and a handler that throws is a FAIL, as in Kafka Streams. Difference (KSC-147): Kafka Streams also asks the handler about a failed changelog record or dead letter; StoatFlow treats those as FAIL without asking
deadLetterTopics() (on all three handler interfaces)๐Ÿ†•StoatFlow extension, default: none, inherited from one super-interface, DeadLetterTopicDeclaring โ€” so one class implementing several handler interfaces still compiles. The topics a handler writes dead letters to; the start checks that each exists (KSC-148; a deserialization handler's only when the topology reads a Kafka topic, a production handler's only when it has a sink). The built-in DLQ handlers return theirs; a handler that wraps one returns its delegate's. A ported handler inherits the empty default
DeadLetterTopicDeclaring๐Ÿš€The interface that declares deadLetterTopics(); the three handler interfaces extend it (KSC-148). A ported handler never names it
ErrorHandlerContextโœ…KS interface: topic()/partition()/offset()/timestamp()/headers()/processorNodeId()/taskId(). A processing failure's headers(), the same instance as its record.headers(), is a copy of the failing record's headers (Kafka Streams hands over the record's own); a timer failure's are the timer's; a punctuator's are empty, as in Kafka Streams
StreamsExceptionโœ…Extends KafkaException; StoatFlow's fatal failures extend it, so catch (StreamsException) works
MissingOutputTopicException๐Ÿš€Thrown by start() when a sink topic or a declared dead-letter topic does not exist (KSC-148); extends StreamsException, missingTopics maps each missing topic to what writes to it

Lifecycle

FeatureStatusNotes
State enumโœ…StoatFlow keeps a richer set than KS's 7 โ€” CREATED, STARTING, VALIDATING_STATE, RESTORING, RUNNING, DRAINING, PAUSED, STOPPING, STOPPED, ERROR. For a KS-shaped KafkaStreams.State, use the KafkaStreamsState mirror + State.toKafkaStreamsState() / kafkaStreamsState() (KSC-81)
StateListenerโœ…KS-exact single setStateListener(StateListener) โ€” a Java method reference / lambda resolves with no cast (KSC-82); an existing BiConsumer<State,State> migrates via StateListener.of(...)
StateRestoreListenerโœ…Full KS-compatible: onRestoreStart / onBatchRestored / onRestoreEnd
StreamsUncaughtExceptionHandlerโœ…setUncaughtExceptionHandler(); REPLACE_THREAD = in-place engine restart (fault-storm-guarded; works under hot-standby HA too โ€” absorbed in place, no failover until the budget is spent), SHUTDOWN_* = single-instance shutdown
cleanUp()โœ…Deletes all local state for the application id, lock files included; valid only in CREATED/STOPPED; rebuilt from changelog on next start. Unlike Kafka Streams 4.4 no process-metadata file is retained, so a full reset needs no manual delete
close()โœ…Graceful shutdown; blocks until terminal. Every concurrent close caller now blocks until completion (a losing second close() no longer returns early mid-shutdown) โ€” KSC-83
close(Duration)โœ…The timeout bounds only the caller's wait, never the shutdown work (KS-exact). Duration.ZERO = async signal-and-return; negative โ†’ IllegalArgumentException; shutdown continues after a false return. Returns true on any terminal state incl. ERROR (divergence โ€” KS only NOT_RUNNING) โ€” KSC-83
close(CloseOptions)โœ…Mirrors the top-level KIP-1153 io.stoatflow.core.CloseOptions (KS-exact defaults + mutate-and-return-this fluents + @JvmStatic factories). GroupMembershipOperation has the Kafka Streams 4.4 shape โ€” DEFAULT, LEAVE_GROUP, REMAIN_IN_GROUP, with DEFAULT as the default and what close() passes. An explicit LEAVE_GROUP/REMAIN_IN_GROUP always wins over the leaveGroupOnClose config; DEFAULT follows it (default: remain). DEFAULT never leaves the group unless leaveGroupOnClose is set, because StoatFlow does not use the streams group protocol โ€” KSC-112. No-op under HA assign(). The deprecated nested KIP-812 KafkaStreams.CloseOptions is not mirrored โ€” a port rewrites one line to the factories โ€” KSC-83
Streams group protocol (KIP-1071 family): group.protocol=streams, the topology-description push (KIP-1331), Admin.describeStreamsGroups๐Ÿ›‘Not applicable โ€” StoatFlow is a classic consumer group, so describeStreamsGroups never lists it. Pull the topology description from GET /topology/ks instead โ€” KSC-113

Configuration

AspectStatusNotes
StreamsConfig builderโœ…Type-safe builder pattern (StreamsConfig.builder(appId, bootstrap).โ€ฆbuild()) โ€” the primary config model
Properties / *_CONFIG constantsโœ…Canonical KS adapter (ADR-124). StreamsConfig.fromProperties(props) / fromMap(map) adapt a KS-keyed Properties/Map onto the typed builder with complete coverage of the KS-4.4.0 overlap surface โ€” all 79 keys, listed one by one in KS config-key mapping below: typed mappings, class-name pluggables (serdes, the three exception handlers, default.client.supplier, rocksdb.config.setter), client prefixes (consumer./main.consumer./restore.consumer./producer./admin.), unprefixed client keys routed by per-client validity exactly as Kafka Streams does (security.protocol reaches consumer + producer + admin, acks only the producer; a prefixed spelling wins, and StoatFlow's forced overrides win over both โ€” main.consumer. is a main-consumer-only scope, as in KS, not an alias for consumer.), and a lenient 3-tier unknown-key policy (mapped / recognised-no-op / unknown) + opt-in stoatflow.config.strict, which throws on truly-unknown keys instead of forwarding them to every client as custom props. The typed builder/YAML stays the recommended model; the adapter is the canonical KS-compat surface, not a stepping stone to a Map-backed swap (ADR-124). The KS *_CONFIG symbol constants and prefix helpers (consumerPrefix(...) โ€ฆ adminClientPrefix(...)) are also shipped for symbol-level source compatibility (KSC-80).
StreamsConfig(Map) / StreamsConfig(Properties) ctorโœ…KSC-79. The KS-faithful new StreamsConfig(props) / new StreamsConfig(map) bootstrap idiom compiles after the import-swap (Properties binds the single Map<*,*> ctor); it delegates to the same fromMap(...).build() adapter as fromProperties. A missing application.id throws at construction (IllegalArgumentException, where KS throws ConfigException). The typed builder(...) / fromProperties(...) paths stay recommended.
toProperties()๐Ÿ†•StreamsConfig.toProperties() emits the config back to KS-keyed Properties (resolved-view, masked) โ€” for /config, diffing, inspect/reconstruct
default.timestamp.extractorโœ…StreamsConfig.defaultTimestampExtractor โ€” a global fallback TimestampExtractor for sources with no per-Consumed WatermarkStrategy (event-time basis only; the watermark default is unaffected). Unset, the record timestamp is the event time and a negative one is accepted; Kafka Streams' built-in extractors (FailOnInvalidTimestamp, its default, LogAndSkipOnInvalidTimestamp, UsePartitionTimeOnInvalidTimestamp, WallclockTimestampExtractor) ship and keep their meaning โ€” see Built-in timestamp extractors
Admin client configโœ…StreamsConfig.adminConfig / admin.* prefix โ€” passthrough merged at all AdminClient sites so a secured cluster authenticates
KafkaClientSupplierโœ…Custom client factory for instrumentation/testing
DefaultKafkaClientSupplierโœ…Standard KafkaConsumer/KafkaProducer/Admin factory
RocksDB configโœ…Bounded memory by default
Serdesโœ…Compatible
Default key/value Serdesโœ…defaultKeySerde/defaultValueSerde propagate through the DSL
Consumer/Producer configโœ…Pass-through maps
bootstrap.resolve.timeout.ms (KIP-909)โœ…Forced to 0 (synchronous bootstrap DNS resolution) on every client StoatFlow creates; a user value is replaced with a warning. Kafka Streams 4.4 does the same: an asynchronously resolving client that fails to resolve stays broken until rebuilt
application.serverโœ…host:port (or protocol://host:port) for the local host reported by interactive-query metadata; defaults from the runtime HTTP host:port when unset. A malformed value fails at startup, as in Kafka Streams 4.4
state.cleanup.dir.max.age.ms (KIP-1259)โœ…Age-gated purge of stale local state at startup (rebuilt from changelog); default disabled
state.cleanup.delay.msโœ…Orphaned-store cleanup interval
validation.validate-output-topics๐Ÿš€StoatFlow-only, default true: the start refuses, naming each missing topic, when a static sink topic or a handler's declared dead-letter topic does not exist (KSC-148). false restores Kafka Streams' behaviour, for clusters that rely on broker auto-creation

KS config-key mapping

Full Kafka Streams 4.4.0 StreamsConfig ConfigDef coverage โ€” all 79 keys โ€” as accepted by StreamsConfig.fromProperties / fromMap (ADR-124). A build-time drift test pins this set against the real Kafka Streams ConfigDef, so a key added upstream cannot go unnoticed. Some rows below cover a family of keys.

Status in this table: โœ… mapped ยท ๐Ÿ†• added (ADR-124) ยท ๐ŸŸก recognised-no-op (warns, never throws โ€” even under stoatflow.config.strict) ยท ๐Ÿงฉ client pass-through (routed to every client whose own configNames() contains it, exactly as Kafka Streams does; a consumer. / producer. / admin. prefix still wins, and StoatFlow's forced overrides win over both).

A port that references the symbol rather than the raw string is covered too (KSC-80): StreamsConfig.TOPOLOGY_OPTIMIZATION_CONFIG, StreamsConfig.METRIC_REPORTER_CLASSES_CONFIG, the six client-prefix constants (CONSUMER_PREFIX / MAIN_CONSUMER_PREFIX / RESTORE_CONSUMER_PREFIX / GLOBAL_CONSUMER_PREFIX / PRODUCER_PREFIX / ADMIN_CLIENT_PREFIX) and the prefix helpers (StreamsConfig.consumerPrefix(prop) โ€ฆ adminClientPrefix(prop)) all ship, and the keys they build route or no-op exactly as the raw strings do. TOPIC_PREFIX and the topology-optimization value constants are not shipped.

KS keyStatusStoatFlow mapping / reason
application.idโœ…applicationId
bootstrap.serversโœ…bootstrapServers
state.dirโœ…stateDir (StoatFlow default /var/stoatflow/state)
default.key.serdeโœ…defaultKeySerde (Class or class-name)
default.value.serdeโœ…defaultValueSerde (Class or class-name)
num.stream.threadsโœ…โ†’ numLanes (closest single-instance analogue). The lanes' thread type is the StoatFlow-only key lanes.thread-type (PLATFORM default, VIRTUAL) โ€” not a KS key
processing.guaranteeโœ…processingGuarantee (plus the exactly_once_beta alias; StoatFlow defaults to EXACTLY_ONCE where KS defaults to at_least_once โ€” KSC-84)
application.serverโœ…applicationServer (interactive-query metadata host:port โ€” KSC-47)
deserialization.exception.handler (+ deprecated default.deserialization.exception.handler)๐Ÿ†•deserializationExceptionHandler (class-name)
production.exception.handler (+ deprecated default.production.exception.handler)๐Ÿ†•productionExceptionHandler (class-name)
processing.exception.handler๐Ÿ†•processingExceptionHandler (class-name)
default.timestamp.extractor๐Ÿ†•defaultTimestampExtractor (global fallback, event-time basis). Kafka Streams' built-in extractors ship in io.stoatflow.core.processor under their Kafka Streams names; their Kafka Streams class names load StoatFlow's
default.client.supplier๐Ÿ†•kafkaClientSupplier (class-name)
rocksdb.config.setter๐Ÿ†•rocksDbConfigSetter (class-name)
dsl.store.format๐Ÿ†•dslStoreFormat (DslStoreFormat.fromConfigString)
commit.interval.ms๐Ÿ†•โ†’ barrierIntervalMs (a semantic seed; the commit cadence is adaptive)
buffered.records.per.partition๐Ÿ†•bufferedRecordsPerPartition โ€” a pause threshold, as in Kafka Streams (any positive value)
max.task.idle.ms๐Ÿ†•maxTaskIdleMs (1:1; both allow -1)
replication.factor๐Ÿ†•changelogReplicationFactor
state.cleanup.delay.ms / state.cleanup.dir.max.age.ms๐Ÿ†•stateCleanupDelayMs / stateCleanupDirMaxAgeMs (KIP-1259)
ensure.explicit.internal.resource.naming๐Ÿ†•ensureExplicitNaming (StoatFlow defaults to true, KS to false; the mapping applies only when the key is present)
processor.wrapper.classโš ๏ธKIP-1112 โ€” supported. One ProcessorWrapper wraps every node, DSL and Processor API alike, at topology-compile time. Full KS fidelity for Processor API nodes; DSL-internal nodes get the record path only, and replacing an internal node is rejected at build time. A wrapper set on the runtime config alone is honoured, where KS ignores that case as too late
num.standby.replicas๐ŸŸกSingle-instance: no standby replicas
acceptable.recovery.lag / max.warmup.replicas / task.assignor.class / probing.rebalance.interval.ms๐ŸŸกSingle-instance: no standby, warmup, assignor or probing
rack.aware.assignment.* / upgrade.from๐ŸŸกSingle-instance: no rack-aware assignment, no rolling-upgrade protocol
group.protocol๐ŸŸกStoatFlow manages consumer-group membership internally
topology.description.push.enabled (KS 4.4)๐ŸŸกIgnored with a warning: StoatFlow is not a streams group, so there is no push. The same topology description is available as a pull from GET /topology/ks โ€” KSC-113
enable.transactional.statestores (KS 4.4)๐ŸŸกAccepted and ignored: StoatFlow's state stores are always transactional. true is silent; false logs a warning that it cannot be honoured (KSC-104)
statestore.uncommitted.max.bytes (KS 4.4)โœ…Sets stoatflow.state.uncommitted-max-bytes, honoured as given. The default stays 256 MiB (Kafka Streams: 64 MiB), the budget is application-wide, under exactly-once it also counts the output held back during a commit, and -1 (disable) is refused: the budget also caps the epoch size (KSC-105)
default.interactive.query.isolation.level (KS 4.4)โœ…Sets stoatflow.state.interactive-query-isolation-level (READ_UNCOMMITTED default, or READ_COMMITTED; case-insensitive). store() applies it to every store, global stores included, where Kafka Streams reads a global store uncommitted (KSC-144)
statestore.cache.max.bytes / cache.max.bytes.buffering๐ŸŸกStoatFlow's cache caps are per-store and commit-triggering (caching.max-entries / caching.max-estimated-bytes), not a global evict budget; global memory is bounded by state.uncommitted-max-bytes
client.id๐ŸŸกClient ids are derived from application.id; override with consumer.client.id / producer.client.id
task.timeout.ms๐ŸŸกCommit-path waits are bounded by barrierTimeoutMs instead
topology.optimization๐ŸŸกRe-keying is in-memory; KTable source reuse is autoReuseKTableSourceTopics โ€” on by default for a compacted source topic. A table whose local state shows it restores from its source topic is never switched to a changelog in place (the start fails, explaining how to switch; a table with no such state gets a warning instead); a versioned table always restores from its changelog
repartition.purge.interval.ms๐ŸŸกIn-memory re-keying: there are no repartition topics to purge
processing.exception.handler.global.enabled๐ŸŸกThe processing handler is already applied globally
errors.dead.letter.queue.topic.name๐ŸŸกThe DLQ is configured per handler (DeadLetterQueue*ExceptionHandler)
default.dsl.store / dsl.store.suppliers.class๐ŸŸกChoose the store with Materialized.withStoreType
default.list.{key,value}.serde.{inner,type}๐ŸŸกNo ListSerde default-type config; use explicit serdes
windowstore.changelog.additional.retention.ms๐ŸŸกWindow changelogs use compaction; no extra retention
window.size.ms / windowed.inner.class.serde๐ŸŸกDeprecated in KS; use the typed WindowedSerdes
built.in.metrics.version / metric.reporters / metrics.* / enable.metrics.push๐ŸŸกMetrics go through Micrometer, not the Kafka metrics subsystem
config.providers๐ŸŸกResolve externalised config before constructing StreamsConfig
allow.os.group.write.access๐ŸŸกStoatFlow does not relax state-dir permissions via config
log.summary.interval.ms๐ŸŸกNo periodic summary-log toggle
poll.ms๐ŸŸกUse consumerPollTimeoutMs
security.protocol / connections.max.idle.ms / metadata.max.age.ms / {receive,send}.buffer.bytes / request.timeout.ms / reconnect.backoff.* / retry.backoff.ms / metadata.recovery.* / metadata.cluster.check.enable (KS 4.4)๐ŸงฉClient properties, routed unprefixed to every client they are valid for (KSC-92, KS parity): security.protocol reaches consumer, producer and admin; the rest go by their own ConfigDef. A consumer. / main.consumer. / producer. / admin. prefix overrides the bare form. The same routing applies to bare ssl.* and sasl.*, and to producer-only keys such as acks

Key architectural differences

The runtime-model deltas โ€” single instance vs. rebalancing cluster, lanes vs. partition-bound tasks, in-memory re-keying vs. repartition topics, barrier-based exactly-once, global vs. partition-scoped state โ€” are summarized in the canonical deltas-at-a-glance table, with the conceptual treatment on the same page. The rows below are the operational and time-semantics differences that surface while reading this matrix:

AspectKafka StreamsStoatFlow
RocksDB memoryUnbounded defaultBounded default (256 MiB)
Window closureStream-time basedWatermark-based
Grace periodAccepts a late record while windowEnd + grace is ahead of the task's stream time (max observed timestamp, KIP-328); the same clock closes the windowOne clock, the watermark (min across non-idle partitions โˆ’ out-of-orderness): a window accepts records until it passes windowEnd + grace (a session: last event + gap + grace), and then closes โ€” final emission, suppress release, join null side, event-time timers (KSC-132)
Negative event timeNever reaches the topology: the default extractor (FailOnInvalidTimestamp) fails the application on one, any other extractor's negative result is dropped with a WARNA valid time, end to end (Flink is the reference): every Long but Long.MIN_VALUE. Windows are floor-aligned with no clamp at 0, streamโ€“stream joins match across 0, onWindowClose() emits windows and sessions ending below 0, and the Processor API, timers and punctuators accept negative times. A sink writes the record without a timestamp; the producer or broker stamps the wall-clock time, counted as stoatflow.sink.records.negative.timestamp.total (Flink DataStream KafkaSink parity). Record, FixedKeyRecord and TestRecord refuse only Long.MIN_VALUE. To keep the KS behaviour for negative times, set default.timestamp.extractor to FailOnInvalidTimestamp or LogAndSkipOnInvalidTimestamp (KSC-138). Near Long.MAX_VALUE โ€” within a session gap or a join window of it โ€” Kafka Streams' sums wrap: its session aggregation fails the record and its join finds no partner, where StoatFlow's saturate, so the session is counted and the join emits; no setting reproduces the wrap
Missing event timeKafka's -1 fails the application (default extractor) or is droppedSkipped and counted as dropped.records{reason=no_event_time}, its offset committed: Kafka's -1 on a source with no extractor or assigner, or Long.MIN_VALUE from one. Failing is opt-in via FailOnInvalidTimestamp
Windows near time 0Hopping windows start no earlier than 0; sliding-window records below the time difference D share one [0, D] windowFloor-aligned: a hopping-window record below size โˆ’ advance also joins the windows that start before 0, and each sliding-window record below D gets its own [t โˆ’ D, t] window. Identical above those times โ€” see Windows near time 0. A Kafka Streams consumer of the windowed output fails on a negative window start
Schema Registry subjects of internal serializationThe internal topics' subjects: <app>-<name>-repartition-key, <app>-<store>-changelog-key / -value, a foreign-key join's โ€ฆ-subscription-registration-topic-fk / -pkThe same names, plus app-scoped subjects where no topic exists. A boundary subject per key stream and input, named as in Kafka Streams only where you named the repartition and Kafka Streams creates it, after a re-key (a streamโ€“stream join's -left / -right included), else after the merge, join or cogroup it leads into or the key's origin (<cogroup>-windowed out of a windowed cogroup); every store under its changelogFor subject (a backing source topic, else its changelog); timers <app>-<processor>-timer; scheduled sources <app>-<source>-scheduled; an outer join's right values apart. Unrelated key streams on one name are refused. Subjects StoatFlow registers
Watermark strategyN/AFlink-style WatermarkStrategy
PunctuatorsTask-scoped, on the stream threadDedicated punctuation lane per sub-topology, full state access
Key-based timersN/AFlink-inspired TimerService, with named timers that a new registration replaces
Scheduled sourcesN/AInterval & cron-based emission
Runtime controlN/Apause()/unpause()

Test utilities

TopologyTestDriver drives a topology in-memory (no broker) for unit tests. All state stores โ€” including RocksDB-Materialized ones โ€” run in-memory in the test driver, but the exactly-once store checks judge the store you declared (KSC-126: a RocksDB versioned join input passes, an in-memory versioned or lruMap one is refused, as in production); real-engine + RocksDB end-to-end coverage uses integration tests with an embedded broker.

TopologyTestDriver

MemberStatusNotes
new TopologyTestDriver(topology) / (topology, Properties) / (topology, Instant) / (topology, Properties, Instant)โœ…KS-shaped constructors; Properties via StreamsConfig.fromProperties
createInputTopic(name, Serde/Serializer, โ€ฆ)โœ…Serde and Serializer overloads + 5-arg (โ€ฆ, Instant, Duration) start-time/auto-advance form (serdes accepted, ignored)
createOutputTopic(name, Serde/Deserializer, โ€ฆ)โœ…Serde and Deserializer overloads
getAllStateStores() / producedTopicNames()โœ…All stores by name; the set of produced topics
advanceWallClockTime(Duration) / get*Store(name) / close()โœ…advanceWallClockTime also moves currentSystemTimeMs() and TimerService.currentProcessingTime()

TestRecord

KS positional order (key, value, headers, timestamp); ctors (K,V), (K,V,Headers[,Long]), (K,V,Instant), (K,V,Headers,Instant), value-only (V), (ConsumerRecord), (ProducerRecord), all callable from Java (a null timestamp means unset); bean getters getKey()/getValue()/getHeaders()/getRecordTime() + KS accessors key()/value()/headers()/timestamp(). The timestamp is unset by default, as in Kafka Streams: pipeInput(TestRecord) stamps an unset record with the input topic's current time. Any timestamp but Long.MIN_VALUE is accepted, negative included, where Kafka Streams' TestRecord rejects every negative one. The driver applies the runtime's event-time rules: a record with no event time is skipped, an extractor's exception fails the pipe, and a sink stamps a negative-time record with the driver's wall clock โ€” see Testing.

TestInputTopic

MethodStatusNotes
pipeInput(key, value , timestamp)โœ…timestamp as long or Instant
pipeInput(value) / pipeInput(value, Instant)โœ…Value-only (null key) โ€” for value-centric / Avro tests
pipeKeyValueList / pipeValueList / pipeRecordListโœ…Bulk piping; key-value lists use KeyValue; timed (โ€ฆ, Instant, Duration) overloads; pipeRecordList is out-variant
advanceTime(Duration)โœ…Advance the input topic's record clock

TestOutputTopic

MethodStatusNotes
readKeyValue() / readKeyValuesToList()โœ…Return KeyValue<K, V> (.key / .value), not Pair
readKeyValuesToMap()โœ…
readValue() / readRecord()โœ…

KeyValue is StoatFlow's KS-shaped key-value type (the same one the DSL uses); a KS port rewrites the KeyValue import to match.

High availability (hot-standby)

Not a KS-parity surface โ€” Kafka Streams scales out with standby tasks; StoatFlow's opt-in hot-standby pairs one active with one or more warm standbys. The surface, and the two deliberate gaps:

SurfaceStatusNotes
stoatflow.ha.mode = off / active-standby๐Ÿš€Deploy-time opt-in; off (default) is byte-identical to single-instance
Active/passive failover๐Ÿš€Crash failover in seconds; graceful deploys / /ha/switch hand off cleanly
Split-brain fencing๐Ÿš€EOS: broker-enforced producer-epoch fence; ALO: metadata-topic lease
/ha/{status,switch,promote,demote} endpoints๐Ÿš€Operator shims; SWITCH/PROMOTE/DEMOTE return 409 unless a caught-up target exists (?force=true override)
ha.acceptable-recovery-lag๐Ÿš€A single total lag sum (default 50000) โ€” not per-task like KS acceptable.recovery.lag (10000)
Continuous changelog replication๐Ÿš€The standby streams committed changelog deltas into local state in real time โ€” like KS standby tasks' continuous restore, for the single-active pair
Restore-before-process on promotion๐Ÿš€A promoting pod restores local state to the committed changelog end before processing
Queryable standby (KIP-535 IQ from standby)โŒStandbys are non-queryable; store queries on a standby are rejected
KS standby tasks / num.standby.replicasโŒ (different model)Single-active with warm standbys โ€” not N-instance standby tasks

See High availability for the operational guide.