Defaults, adaptivity, and presets

StoatFlow ships opinionated defaults so a near-empty application.yaml runs a correct, bounded-memory application out of the box. This page explains what those defaults give you, which behaviour adapts at runtime versus what you set explicitly, and the RocksDB presets and bound knobs available when you do need to tune. For the layered resolution rules behind these settings (env / system property / YAML / programmatic precedence), see Configuration model; for a workload-driven walkthrough, see Tuning.

A minimal config that works

The only required key is stoatflow.application-id. Everything else falls back to a default. A complete, runnable configuration can be as short as this:

stoatflow:
  application-id: my-app
  bootstrap-servers: localhost:9092

Add the runtime's HTTP and metrics endpoints and the license, and you have what Your first app uses:

stoatflow:
  application-id: word-count
  bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:-localhost:9092}
  license:
    key: ${STOATFLOW_LICENSE_KEY}

runtime:
  http:
    enabled: true
    port: 8080
  metrics:
    enabled: true

This runs because the defaults already make the right choices for most workloads. The ones worth knowing:

What adapts, and what you set

Some runtime behaviour is adaptive: the engine measures its own commit cost and state-write rate and adjusts within bounds. The rest is fixed to what you configure. The split matters because you tune the two kinds differently — for adaptive behaviour you set the envelope, for fixed behaviour you set the value.

Commit cadence self-tunes within an interval band

The commit barrier interval is not a fixed sleep. The runtime estimates how long a commit takes and schedules the next barrier relative to that estimate, so a topology with fast commits barriers more often and a topology with slow commits backs off — both staying inside the band you configure. You set the band; the runtime picks the cadence inside it.

stoatflow:
  commit-barrier:
    min-interval-ms: 100     # allow tighter commits for lower latency
    max-interval-ms: 2000    # but never let an epoch run longer than 2s
    interval-factor: 1.5

Lower the band for lower end-to-end latency (more frequent, smaller commits); raise it for higher throughput (fewer, larger commits). The interval-factor controls how aggressively the cadence tracks commit cost: 1.0 is back-to-back, higher values leave more idle gap between commits.

Epoch size self-tunes within memory and count bounds

An epoch is the batch of records between two commit barriers. The runtime sizes each epoch automatically, tracking the dispatch rate and the per-record state-write cost so it can keep uncommitted state bounded — small for high-fan-out stateful topologies (FK joins, multi-way joins, where one input record cascades into many state writes) and large for lightweight stateless ones. You don't set the epoch size; you set the bounds it operates within.

stoatflow:
  state:
    uncommitted-max-bytes: 268435456   # 256 MB — raise on large containers, lower on small
  commit-barrier:
    initial-max-epoch-records: 4096    # cap startup epochs before adaptation kicks in
    max-epoch-records: 50000           # optional hard ceiling

Sizing your uncommitted-max-bytes to the container is the main lever for stateful topologies. As a starting point, leave roughly half the container's available memory (after JVM heap, RocksDB, and non-heap) as headroom:

The /metrics endpoint exposes per-epoch counters — stoatflow.barrier.epoch.actual.records, stoatflow.barrier.epoch.memory.cap, and a stoatflow.barrier.trigger.total counter tagged by what ended each epoch (time, record count, memory pressure, or per-store cache pressure) — so you can see which bound is binding and adjust the right one. See Metrics.

Fixed knobs you set directly

These do not adapt; the value you set is the value used.

RocksDB presets and tuning

State stores default to RocksDB with a bounded ~256 MB memory budget (block cache + memtables), as opposed to Kafka Streams' unbounded RocksDB defaults. You select a memory profile with a single key.

Presets

stoatflow:
  rocks-db:
    preset: DEFAULT   # DEFAULT | LOW_MEMORY | HIGH_PERFORMANCE
    backend: AUTO     # AUTO | FFM | JNI

The memory budget is shared across all stores — one block cache and one write-buffer manager bound total RocksDB memory regardless of how many state stores the topology declares.

Backend

backend: AUTO (default) picks each runtime's faster path: the Foreign Function & Memory API (FFM) on the JVM — lower per-call overhead, no virtual-thread pinning — and the rocksdbjni bindings (JNI) under a native image. Set FFM or JNI explicitly to force a specific backend. Both require the --enable-native-access=ALL-UNNAMED JVM flag (the io.stoatflow Gradle plugin sets it — see Installation).

Beyond the presets

The three presets cover the common cases. For finer control there are two escape hatches, neither exposed as additional YAML keys:

• Custom total-memory budget (programmatic). When constructing the config in code rather than YAML, RocksDBConfig.withTotalMemory(bytes) produces a config with cache, memtables, and file sizes scaled proportionally to any budget (minimum 32 MB):
  KotlinJava

  import io.stoatflow.core.state.rocksdb.RocksDBConfig
  
  // 512 MB total, scaled proportionally
  val rocks = RocksDBConfig.withTotalMemory(512L * 1024 * 1024)
  

• Per-store RocksDB options (config setter). For per-store tuning — different settings for write-heavy versus read-heavy stores, custom compression, compaction strategy — set rocks-db-config-setter to the fully qualified name of a class implementing RocksDBConfigSetter. Its configure(storeName, options) runs for every store with options pre-populated from the active preset, so you can branch on the store name and override per-store knobs such as options.cf.maxWriteBufferNumber, options.cf.writeBufferSize, or options.table.useRibbonFilter. Shared memory resources (the block cache and write-buffer manager) are framework-managed and cannot be overridden per store — set the total memory budget globally via the preset or RocksDBConfig.withTotalMemory(...).

  stoatflow:
    rocks-db-config-setter: com.example.MyRocksDBConfigSetter
  

Where to go next

• Configuration model — how env vars, system properties, YAML, and programmatic config layer and override each other.
• Tuning — workload-driven tuning walkthroughs for latency, throughput, and heavy-state topologies.
• Configuration reference — every key, default, and env override in one table.
• Metrics — the per-epoch and commit-cadence counters that show which bound is binding.