StoatFlow: Kafka Streams compatible engine built to scale up — not out
The first alpha is here. Kafka Streams DSL on a single-replica runtime with virtual-thread parallelism — measurably less CPU, memory, and latency on the same hardware.
Blog
Engineering notes from the StoatFlow team — launch announcements, design choices, benchmarks, and product updates.
KIP-1035: Why Kafka Streams 4.3 lets state stores own their offsets
Kafka Streams kept each task's changelog offsets in a separate checkpoint file that could drift from the state it described. KIP-1035, in Kafka 4.3, moves them inside the state store — atomic with the data, and the keystone for transactional state stores.
Compiling StoatFlow to a GraalVM native image: G1, PGO, and why JNI beats FFM under AOT
Native image is viable for a throughput-first stream processor — but only with Oracle G1 + PGO. The wrong turn (we blamed FFM; it was the GC) and the reversal (FFM is faster on the JVM, slower under AOT) are the most interesting parts.
KIP-1271: record headers in state stores, and the cost of a value format
KIP-1271/1285 let a state store keep a record's headers next to its value — shipped in Apache Kafka 4.3.0. What the KIP is, where Kafka Streams has got to (and what is still in flight), how StoatFlow 1.0.0 compares — and why a one-varint format change becomes an engine-wide one.
Hot standby for StoatFlow: failover in seconds, not a cold restore
StoatFlow recovery has always been fast restart — but restart time scales with state. Hot standby is an opt-in active/passive pair that fails over in seconds regardless of state size, with exactly-once preserved across the handoff. Redundancy, not scale-out.