Defaults, adaptivity, and presets

StoatFlow ships opinionated defaults so a near-empty application.yaml runs a correct, bounded-memory application out of the box. This page explains what those defaults give you, which behaviour adapts at runtime versus what you set explicitly, and the RocksDB presets and bound knobs available when you do need to tune. For the layered resolution rules behind these settings (env / system property / YAML / programmatic precedence), see Configuration model; for a workload-driven walkthrough, see Tuning.

A minimal config that works

The only required key is stoatflow.application-id. Everything else falls back to a default. A complete, runnable configuration can be as short as this:

stoatflow:
  application-id: my-app
  bootstrap-servers: localhost:9092

Add the runtime's HTTP and metrics endpoints and the license, and you have what Your first app uses:

stoatflow:
  application-id: word-count
  bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:-localhost:9092}
  license:
    key: ${STOATFLOW_LICENSE_KEY}

runtime:
  http:
    enabled: true
    port: 8080
  metrics:
    enabled: true

This runs because the defaults already make the right choices for most workloads. The ones worth knowing:

What adapts, and what you set

Some runtime behaviour is adaptive: the engine measures its own commit cost and state-write rate and adjusts within bounds. The rest is fixed to what you configure. The split matters because you tune the two kinds differently — for adaptive behaviour you set the envelope, for fixed behaviour you set the value.

Commit cadence self-tunes within an interval band

The commit barrier interval is not a fixed sleep. The runtime estimates how long a commit takes and schedules the next barrier relative to that estimate, so a topology with fast commits barriers more often and a topology with slow commits backs off — both staying inside the band you configure. You set the band; the runtime picks the cadence inside it.

stoatflow:
  commit-barrier:
    min-interval-ms: 100     # allow tighter commits for lower latency
    max-interval-ms: 2000    # but never let an epoch run longer than 2s
    interval-factor: 1.5

Lower the band for lower end-to-end latency (more frequent, smaller commits); raise it for higher throughput (fewer, larger commits). The interval-factor controls how aggressively the cadence tracks commit cost: 1.0 is back-to-back, higher values leave more idle gap between commits.

Epoch size self-tunes within memory and count bounds

An epoch is the batch of records between two commit barriers. The runtime sizes each epoch automatically, tracking the dispatch rate and the per-record state-write cost so it can keep uncommitted state bounded — small for high-fan-out stateful topologies (FK joins, multi-way joins, where one input record cascades into many state writes) and large for lightweight stateless ones. You don't set the epoch size; you set the bounds it operates within.

stoatflow:
  state:
    uncommitted-max-bytes: 268435456   # 256 MiB — raise on large containers, lower on small
  commit-barrier:
    initial-max-epoch-records: 4096    # cap startup epochs before adaptation kicks in
    max-epoch-records: 50000           # optional hard ceiling

Sizing your uncommitted-max-bytes to the container is the main lever for stateful topologies. Uncommitted state is held on the heap, so the budget to size against is the computed -Xmx, not the container limit. The container-aware entrypoint sets -Xmx to the container limit minus a non-heap reservation — the JVM baseline (25% of the limit, capped at 512 MiB) plus RocksDB's 256 MiB — so on a 4 GiB container the heap is 4096 − (512 + 256) = 3328 MiB. Nothing is left over: the reservation is what the JVM and RocksDB spend outside the heap, not spare room.

As a starting point, leave at least half the heap free for your own objects, the lane queues and GC:

Those -Xmx figures assume RocksDB's default 256 MiB reservation. Change the preset and the reservation moves with it — STOATFLOW_ROCKSDB_MB is what the entrypoint subtracts.

The /metrics endpoint exposes per-epoch counters — stoatflow.barrier.epoch.actual.records, stoatflow.barrier.epoch.memory.cap, and a stoatflow.barrier.trigger.total counter tagged by what ended each epoch (time, record count, memory pressure, or per-store cache pressure) — so you can see which bound is binding and adjust the right one. See Metrics.

Fixed knobs you set directly

These do not adapt; the value you set is the value used.

RocksDB presets and tuning

State stores default to RocksDB with a bounded ~256 MiB memory budget (block cache + memtables), as opposed to Kafka Streams' unbounded RocksDB defaults. You select a memory profile with a single key.

Presets

stoatflow:
  rocks-db:
    preset: DEFAULT   # DEFAULT | LOW_MEMORY | HIGH_PERFORMANCE
    backend: AUTO     # AUTO | FFM | JNI

The memory budget is shared across all stores — one block cache and one write-buffer manager bound total RocksDB memory regardless of how many state stores the topology declares.

Backend

backend: AUTO (default) picks each runtime's faster path: the Foreign Function & Memory API (FFM) on the JVM — lower per-call overhead, no virtual-thread pinning — and the rocksdbjni bindings (JNI) under a native image. Set FFM or JNI explicitly to force a specific backend. Both require the --enable-native-access=ALL-UNNAMED JVM flag (the io.stoatflow Gradle plugin sets it — see Installation).

Beyond the presets

The three presets cover the common cases. For finer control there are two escape hatches, neither exposed as additional YAML keys:

• Custom total-memory budget (programmatic). When constructing the config in code rather than YAML, RocksDBConfig.withTotalMemory(bytes) produces a config with cache, memtables, and file sizes scaled proportionally to any budget (minimum 32 MiB):
  KotlinJava

  import io.stoatflow.core.state.rocksdb.RocksDBConfig
  
  // 512 MiB total, scaled proportionally
  val rocks = RocksDBConfig.withTotalMemory(512L * 1024 * 1024)
  

• Per-store RocksDB options (config setter). For per-store tuning — different settings for write-heavy versus read-heavy stores, custom compression, compaction strategy — set rocks-db-config-setter to the fully qualified name of a class implementing RocksDBConfigSetter. Its configure(storeName, options) runs for every store with options pre-populated from the active preset, so you can branch on the store name and override per-store knobs such as options.cf.maxWriteBufferNumber, options.cf.writeBufferSize, or options.table.useRibbonFilter. Shared memory resources (the block cache and write-buffer manager) are framework-managed and cannot be overridden per store — set the total memory budget globally via the preset or RocksDBConfig.withTotalMemory(...).

  stoatflow:
    rocks-db-config-setter: com.example.MyRocksDBConfigSetter
  

Where the RocksDB budget sits in the pod

RocksDB's 256 MiB is one of four caps the defaults set. This is all of them against a 4 GiB container, drawn to scale, each with the knob that moves it and what fires when it fills:

Two things follow from the shape of it. The caps are absolute, so a larger container buys headroom and nothing else — which is why the uncommitted-max-bytes sizing table above raises the ceiling with the container rather than leaving it at the default. And -Xmx is derived, not set: the container-aware entrypoint subtracts the RocksDB reservation and a JVM baseline from the container's own memory limit, so you size the container and the image sizes the heap. See Docker → Container-aware heap sizing for that calculation and Tuning for which knob to reach for from which symptom.

Where to go next

• Configuration model — how env vars, system properties, YAML, and programmatic config layer and override each other.
• Tuning — workload-driven tuning walkthroughs for latency, throughput, and heavy-state topologies.
• Configuration reference — every key, default, and env override in one table.
• Metrics — the per-epoch and commit-cadence counters that show which bound is binding.