Tuning under load

StoatFlow ships with defaults that work for most topologies, and the parts that need to adapt to your workload — commit cadence and epoch size — self-tune within the bounds you set. This page covers the knobs you do set: lane count, the commit-cadence bounds, the memory and uncommitted-state limits, and RocksDB sizing. Each section starts from a symptom so you know when to reach for it.

For the why behind these mechanics, see Lanes and parallelism, Exactly-once, and State and thread safety. For where each value comes from and how it's merged, see the Configuration model and Defaults and presets.

Before you tune

Tune one thing at a time, and tune against a representative load, not a synthetic burst. The defaults are deliberately conservative; most workloads need at most one or two of the knobs below. Always confirm the symptom first — reach for /metrics and the /debug/barriers endpoint (see REST API) rather than guessing.

Lane count vs cores

Knob: stoatflow.lanes.count — default max(2, CPU cores).

Lanes are the unit of per-key parallelism: every record is routed to a lane by its key, and each lane is a virtual thread that processes its keys in order. More lanes means more keys processed concurrently, at the cost of more queues and a little more dispatch overhead.

stoatflow:
  lanes:
    count: 16

How to size it:

• CPU-bound processing (pure transforms, local aggregation): start at the number of CPU cores. Going much higher just adds scheduling overhead without more throughput, because the work is already saturating the cores.
• I/O-bound processing (enrichment calls, blocking lookups, external APIs): go well above core count — virtual threads park cheaply while blocked, so 4×–8× cores keeps cores busy while many lanes wait on I/O. This is the case where raising the lane count pays off most.
• Few distinct keys: lanes only help if keys spread across them. If your traffic is dominated by a handful of hot keys, extra lanes sit idle — the hot keys still serialize onto their assigned lanes. Fix the key distribution first.

Lane queue capacity

Knob: stoatflow.lanes.queue-capacity — default 300.

Each lane has a bounded queue; when a lane fills, the dispatcher applies backpressure. Raise it if you see bursty input stalling dispatch and you have memory headroom; the default is sufficient for steady-state load.

stoatflow:
  lanes:
    queue-capacity: 500

Commit cadence

StoatFlow commits in epochs — the work between two commit barriers becomes one Kafka transaction plus one state-store flush (see Exactly-once). The cadence — how often a barrier fires — is self-tuning: the engine adapts it within the bounds you configure, so you set the envelope, not the exact interval. The algorithm itself stays in the source; what you control is the floor and ceiling.

The bounds you set:

Defaults and StreamsConfig field names for every commit-barrier.* knob are in Engine configuration → Commit-barrier cadence.

stoatflow:
  commit-barrier:
    min-interval-ms: 150
    max-interval-ms: 5000
    interval-factor: 1.5

The trade-off is the classic latency-vs-throughput one:

• Tighter, more frequent commits (lower max-interval-ms, lower interval-factor) reduce end-to-end latency and bound how much work a crash replays — at the cost of more transaction and flush overhead per record.
• Looser, less frequent commits (higher bounds) amortize commit overhead over more records, raising throughput — at the cost of higher latency and a larger replay window after a restart.

Epoch size

You normally don't touch epoch size — it self-tunes within the bounds above. Two knobs exist for the edges:

stoatflow:
  commit-barrier:
    initial-max-epoch-records: 1024   # conservative startup for heavy-state topologies
    # max-epoch-records: 25000        # optional deterministic ceiling

Memory and uncommitted state

Between commits, a stateful topology accumulates uncommitted state in memory — the writes buffered since the last barrier, held until the transaction commits (KIP-892 transactional state stores). On topologies where one input record fans out into many state writes (multi-way joins, foreign-key joins, high-cardinality aggregations), this is the value most likely to threaten container memory.

Knob: stoatflow.state.uncommitted-max-bytes — default 268435456 (256 MB).

When buffered uncommitted bytes approach this limit, the engine triggers an early commit barrier to flush state and bound peak memory. This is the backstop that keeps a high-fan-out epoch from running the container out of memory.

stoatflow:
  state:
    uncommitted-max-bytes: 268435456   # 256 MB (default)

The global ceiling has two per-store companions under stoatflow.caching: max-estimated-bytes (default 256 MB per caching store) and max-entries (default unbounded). Exceeding a per-store cap fires the same early commit barrier — tagged CACHE_PRESSURE on the barrier-trigger metric — so one hot store can't monopolise the buffer. In practice uncommitted-max-bytes measures a superset and fires first; leave max-entries unbounded unless a specific store must bound its per-epoch key cardinality.

Sizing it against the container:

Segmented-store checkpoint cadence

Time-windowed and session stores are segmented, and their RocksDB segments run with the write-ahead log off — so they only become durable on a clean close(). A periodic checkpoint (flush dirty segments, then the offsets) makes them durable between commits and caps how much changelog a pod replays after an ungraceful restart, and how much a hot-standby standby drains on promotion — to at most one checkpoint interval.

Knobs: stoatflow.state.segment-checkpoint-records (default 100000) and stoatflow.state.segment-checkpoint-ms (default 30000), whichever fires first.

The defaults are deliberately coarse: a checkpoint flushes SST files, so a tight cadence trades a smaller recovery window against more SST/compaction churn in steady state. Lower them only if your recovery-time objective needs a shorter ungraceful-restart catch-up than ~30 s / 100k records of windowed writes; raise them if you see checkpoint-driven write amplification and can absorb a longer catch-up.

RocksDB for large state

Persistent state lives in RocksDB. Unlike Kafka Streams' unbounded default, StoatFlow caps RocksDB memory by default to keep the process within its container. You pick a preset; the framework wires the block cache and memtable budget for you.

Knob: stoatflow.rocks-db.preset — default DEFAULT.

stoatflow:
  rocks-db:
    preset: HIGH_PERFORMANCE

When state grows large (millions of keys, hot read paths), DEFAULT's 256 MB block cache becomes a bottleneck: working-set reads start missing the cache and hit disk. HIGH_PERFORMANCE widens both the block cache and the memtable budget. Budget for it — it is real resident memory on top of the JVM heap and uncommitted-max-bytes, so size the container accordingly.

Confirm the diagnosis before you resize. Enable the RocksDB metrics (rocks-db.metrics.enabled + statistics-enabled) and watch stoatflow_rocksdb_block_cache_hit_ratio under load: a ratio falling toward the floor while stoatflow_rocksdb_shared_block_cache_usage_bytes sits at capacity is the signal that the cache is too small — the metric that turns the symptom below from a guess into a measurement.

Backend selection

Knob: stoatflow.rocks-db.backend — default AUTO.

AUTO picks each runtime's faster path: the FFM (Foreign Function & Memory API) backend on the JVM — lower per-call overhead than JNI and no virtual-thread pinning during native RocksDB calls — and JNI under a native image, where FFM's critical-downcall advantage doesn't apply. Leave it on AUTO unless you have a specific reason to pin a backend; if you force JNI on the JVM, also set stoatflow.lanes.thread-type: PLATFORM to avoid pinning carrier threads.

Custom RocksDB budget (programmatic)

The YAML preset is the supported path for most deployments. If you need a memory budget between the presets, set RocksDBConfig directly when building the engine config — RocksDBConfig.withTotalMemory(...) splits the budget evenly between block cache and memtables:

import io.stoatflow.core.config.StreamsConfig
import io.stoatflow.core.state.rocksdb.RocksDBConfig

val config = StreamsConfig.builder("my-app", "localhost:9092")
    .rocksDbConfig(RocksDBConfig.withTotalMemory(512L * 1024 * 1024)) // 512 MB total
    .build()

import io.stoatflow.core.config.StreamsConfig;
import io.stoatflow.core.state.rocksdb.RocksDBConfig;

var config = StreamsConfig.builder("my-app", "localhost:9092")
    .rocksDbConfig(RocksDBConfig.withTotalMemory(512L * 1024 * 1024L)) // 512 MB total
    .build();

Watching the effect

Tune against signals, not hunches. After each change, watch:

• /metrics — throughput, consumer lag, commit timing, and JVM/container memory, scraped by Prometheus. See Metrics and Observability.
• /debug/barriers — the live commit-pipeline state, which tells you whether barriers are firing on schedule and what's triggering them. See REST API.
• /health/ready — readiness flips DOWN if the engine can't keep up or a commit stalls. See Probes.

A clean tuning loop: change one knob → run representative load → read the metric that maps to the symptom → keep or revert. Resist changing several knobs at once; you won't know which one moved the needle.

Next steps

• Defaults and presets — where every default comes from and how layers merge.
• Configuration reference — every key, type, range, default, and environment override.
• Production checklist — the pre-flight list before you take a topology to production.
• Observability — wiring metrics and dashboards so the symptoms above show up before users notice.