Core concepts
These pages explain the model behind the StoatFlow runtime: how one process runs your whole topology, what guarantees it makes, and where the knobs are. They build on each other — read them top to bottom the first time, then come back to any one on its own.
Recommended reading order
Each page assumes the ones before it. The architecture page is the anchor — it states the conceptual model, and the rest go deeper on one facet each without re-deriving it.
1. Architecture
The single-instance model, the data path from source to sink, and the operational surface. The foundation for every other concept page.
2. Exactly-once
The commit barrier: how state, output, and offsets commit atomically in one Kafka transaction — and what at-least-once mode trades away.
3. Lanes and parallelism
How key affinity routes records to lanes, why lane count is decoupled from partition count, and why blocking I/O is cheap.
4. State and thread-safety
Global state stores, why concurrent access across lanes is safe, the store types available, and changelog-backed durability.
5. Event time and watermarks
Event time versus arrival time, how watermarks advance, when windows close, and the policies for late records.
6. Configuration model
How configuration is layered and resolved — what you set in YAML, what the engine derives, and what stays automatic.
7. Error-handling model
The failure policies the runtime applies — skip, fail, or dead-letter — and what each one means for your data and your topology.
8. How StoatFlow differs from Kafka Streams
The conceptual deltas if you already know Kafka Streams: single instance, in-memory repartitioning, lanes instead of tasks.
Where these pages stop
The concept pages describe behaviour and design — the model, the guarantees, and what you can observe at runtime. They do not document the internal algorithms, data structures, or protocols that implement them; the source code is the source of truth for those. The Architecture page sets that boundary, and the rest of this section holds to it.
Where to go next
- Building topologies — the DSL and the Processor API, once the model is clear.
- Your first app — a complete word-count app on the runtime, if you haven't run one yet.
- Features and Motivation — the full capability list and the reasoning behind the design.
Project setup
Apply StoatFlow's build conventions with Gradle or Maven — the JDK 25 toolchain, the required JVM flags, a runnable fat JAR, and opt-in reproducible Docker and GraalVM native images, from a few lines of config.
Architecture
How StoatFlow runs your Kafka Streams topology as a single replica — the conceptual model, processing lanes, commit barriers, state, and operational surface.