Benchmarks

These numbers compare a StoatFlow build (EOS Phase 6b, ALO Phase 8) against a stable Apache Kafka Streams 4.1.1 baseline on identical hardware, identical broker, identical workload. The intent is to be useful, not promotional — full topology, load profile, methodology, and the cases where StoatFlow is only at parity or measurably behind Kafka Streams are all in scope below.

All runs originate from the internal benchmark report dated 2026-04-22. The numbers below transcribe that report verbatim.

Setup

Hardware

Kafka & runtime

What's controlled, what's varied

Topology, input data, load profile, partition count, broker, hardware, and run duration are identical across the two runtimes for any given comparison. The only variable is the runtime itself (StoatFlow vs Kafka Streams) and the processing mode (EOS vs ALO).

Each run executes for 20 minutes total: a 10-minute warmup (JIT, GC stabilisation, lag catch-up) followed by a 10-minute measurement window at steady state. Topics are recreated between runs; state is empty at start.

Methodology

Latency

True E2E P50 / P95 / P99 is measured from the downstream-consumer's point of view — the latency a real service reading the output topic would actually observe. A dedicated latency-consumer pod simulates that downstream application: it subscribes to the output topic with isolation.level=read_committed (so a record only becomes visible after the producing transaction commits) and, for every consumed record, computes consumption time − record timestamp. Percentiles are derived over the entire measurement window, not from averages.

The read_committed isolation is deliberate. Under EOS, downstream consumers can't see records before the producing transaction commits — which means the runtime's commit cadence is part of the user-observable latency (StoatFlow's 130 ms minimum barrier interval — adaptively scheduled — and Kafka Streams' default 100 ms commit.interval.ms). An internal produce-time measurement would hide that cost.

The latency-consumer adds ~10–15 % measurement overhead through the extra consume-and-timestamp cycle. Percentile relative comparisons between runtimes hold; absolute latency on a runtime without a latency-consumer would be slightly lower.

Resource utilisation

CPU, JVM heap, container working set, GC pause durations, and thread counts are sampled from Prometheus at 10-second resolution and averaged over the measurement window. GC pause is the per-event JVM Stop-the-World duration, not the cumulative GC time.

Repeatability

Throughput is defined and controlled by the load generators — produced rec/s mirrors the configured input rate, predictable and repeatable across runs. Latency, CPU, memory, and GC numbers in the result tables are sampled from each run and should be read as point estimates rather than statistical means.

Scenario A — Stateless (simple)

Topology

A minimal stateless pipeline: one source topic → mapValues (uppercase) → filter (length > 5) → one sink topic. No state, no repartitioning, no joins. The simplest possible topology — used to measure pure framework overhead.

Data & load profile

Results — EOS

This is the scenario where StoatFlow's EOS commit cadence is most exposed: with no state store cost and trivial per-record work, StoatFlow's 130 ms minimum barrier interval runs a touch slower than KS's 100 ms commit.interval.ms. The ~30 ms cadence delta accounts for essentially all of the observed ~24 ms True E2E P50 gap. ALO drops the barrier cadence entirely and the gap inverts (see below).

Results — ALO

In ALO, StoatFlow recovers a small latency lead but uses substantially more container memory on this scenario — the in-memory buffer and virtual-thread stack overhead dominate when there's no state store to amortise over. The GC pause delta (−66 %) goes the other way.

Scenario B — Word count

Topology

Canonical stateful aggregation: source → flatMapValues (tokenise on whitespace) → groupBy(word) → count (RocksDB) → toStream → output. Implicit repartitioning between groupBy and count: Kafka Streams writes to a Kafka-hosted repartition topic; StoatFlow re-keys in memory.

Data & load profile

Results — EOS

Results — ALO

Scenario C — Stateless (advanced)

Topology

Two independent processing paths over four Protobuf input streams:

• Path A: user-events + product-updates → merge → flatMap (tag expansion, 1 → N fan-out) → repartition → split by priority → 3 output topics.
• Path B: orders + click-streams → filter/transform → merge → repartition by customerId → 1 output topic.

Two explicit repartitions, four inputs, four outputs, no state.

Data & load profile

Results — EOS

Results — ALO

Scenario D — Stateful joins

Topology

Five-table enrichment chain: customers + products + orders + shipments + reviews. One primary-key join (orders ⋈ shipments) plus four foreign-key joins. Output: fully enriched record per order. Ten-plus RocksDB state stores (one per input table plus join intermediates). Avro with Schema Registry.

Data & load profile

Results — EOS

Results — ALO

State restoration

Separate benchmark — cold-start recovery from pre-populated changelog topics, no live processing. Measures the wall-clock time and resource cost of materialising a full state set.

Setup

StoatFlow offers two restoration modes:

• WriteBatch HIGH_PERF — RocksDB WriteBatch ingestion. Lower memory overhead, recommended default.
• SST HP buf128 — direct SST file ingestion with a 128 MB per-worker sort buffer. Lower wall-clock time, substantially higher memory ceiling.

Results

When to use which mode

• WriteBatch (default) — broadly applicable. Wall-clock 1.45× faster than KS, lower memory than KS, smaller on-disk footprint. The right starting point unless restoration time is the binding production constraint.
• SST ingestion — when first-byte-after-restart latency matters and the deployment has memory headroom. ~3.2× the container memory of WriteBatch during the restoration window; the same memory profile during steady-state.

Where StoatFlow is at parity or behind

A short list of cases where the numbers don't favour StoatFlow — included so the page reads as data, not as advocacy.

EOS latency on the simplest topology

On Stateless (simple) in EOS mode, Kafka Streams beats StoatFlow on every latency percentile: P50 by 24 %, P95 by 17 %, P99 by 12 %. Root cause: StoatFlow's commit pipeline runs at a 130 ms minimum barrier interval (commit-barrier.min-interval-ms; the adaptive scheduler clamps barrier-to-barrier between the configured min and max via post-commit feedback). KS's default commit.interval.ms=100 produces a slightly tighter cadence on workloads where the commit itself dominates the latency budget. The ~30 ms cadence delta accounts for essentially the entire ~24 ms P50 gap. In ALO mode (no transactions, no barrier cadence) StoatFlow recovers and shows a small latency lead, so the gap is specific to EOS-on-simple-topologies. The minimum barrier interval is a tunable; lowering it further narrows the EOS gap at the cost of commit-pipeline throughput.

Container memory on Stateless (simple) ALO

In ALO mode on the simplest topology, StoatFlow uses +53 % container memory (657 MB vs 430 MB). GC pause time is 2.9× lower in the same comparison — a different point on the same trade-off — but the working-set figure is larger, and a memory-constrained deployment should know that. The delta does not appear on more substantial workloads, where state-store memory dominates.

GC pause on Stateless (advanced)

GC pause averages run +71 % in EOS and +136 % in ALO vs Kafka Streams on this scenario (2.9 ms vs 1.7 ms; 2.6 ms vs 1.1 ms). Absolute pause times remain under 3 ms in both runtimes, so the relative figure overstates the practical impact, but it's a genuine regression on this workload shape (high-fanout flatMap on relatively large records).

Not yet benchmarked

The following are explicitly out of the current measurement set. Numbers here are not just unfavourable — they are absent.

• Scaling beyond 8 vCPU. All numbers above are on a single Hetzner ccx33. Larger boxes (16+ vCPU, 64+ GB) are extrapolated from microbenchmarks of the lane dispatcher and have not been validated end-to-end at scale.
• Custom processor DAGs with deep fan-out. Only the four scenarios above were measured. Topologies with many sub-topologies (10+) may behave differently.

StoatFlow at the time of these runs is in active development (EOS Phase 6b, ALO Phase 8); Kafka Streams 4.1.1 is a stable production release. Later StoatFlow phases may improve or regress against these numbers.

What's next

Two benchmark expansions are planned for upcoming runs.

Higher-load runs on AWS EKS

16-core compute-optimized nodes for both the application tier and the Kafka broker tier, with substantially higher network and disk-IO bandwidth than the current Hetzner ccx33 baseline. Goal: stress sustained throughput beyond the single-machine measurements above, validate the vertical-scale assumption past 8 vCPU end-to-end (currently extrapolated from microbenchmarks), and push past the ~380 MB/s broker-fetch ceiling that bounds State restoration today.

Additional benchmark apps

Coverage of all window types (tumbling, hopping, sliding, session) and stream-stream joins — neither is exercised today; the current set covers stateless transforms, key changes, aggregations, and primary-key + foreign-key table joins. Plus more complex topologies modelling real-world use cases, driven by production-like test-data workloads — record-size distributions, key cardinality, arrival patterns, and foreign-key relationships that actually resolve at join time (not random keys that mostly miss) — instead of the synthetic load profiles used in the current runs.