Engine configuration (:core)

These are the :core engine settings — the stream-processing knobs that exist whether you embed stoatflow-core directly or run on the batteries-included stoatflow-runtime. On the runtime they live under the stoatflow: block of application.yaml; in plain :core code you set the same values on StreamsConfig (or its Builder). This page groups the ones you'll actually reach for, with their defaults.

Identity and connection

Only application-id is required — it has no default, so the engine won't start without it. bootstrap-servers defaults to localhost:9092, which is fine for local development but you'll set it explicitly for any real broker.

stoatflow:
  application-id: order-processor
  bootstrap-servers: kafka-1:9092,kafka-2:9092,kafka-3:9092

Default serdes

When an operator doesn't specify a serde via Consumed, Produced, Materialized, etc., these defaults apply. Both default to ByteArray — set them to the type your topology mostly uses.

In YAML the values are fully qualified serde class names; in code you pass Serde instances:

val config = StreamsConfig.builder("order-processor", "localhost:9092")
    .defaultKeySerde(Serdes.String())
    .defaultValueSerde(Serdes.String())
    .build()

var config = StreamsConfig.builder("order-processor", "localhost:9092")
    .defaultKeySerde(Serdes.String())
    .defaultValueSerde(Serdes.String())
    .build();

stoatflow:
  default-key-serde: org.apache.kafka.common.serialization.Serdes$StringSerde
  default-value-serde: org.apache.kafka.common.serialization.Serdes$StringSerde

Lanes (parallelism)

Lanes are the engine's virtual partitions — records with the same key always route to the same lane, preserving per-key order while running many keys in parallel. See Lanes and parallelism for the why.

stoatflow:
  lanes:
    count: 32
    queue-capacity: 500

Commit-barrier cadence

The commit barrier is what makes StoatFlow exactly-once: when a barrier flows through the topology, state flushes and the Kafka transaction commits as one atomic unit. The cadence self-tunes at runtime within the bounds you set — interval-ms is the seed, and the engine adapts between min-interval-ms and max-interval-ms. You set the envelope, not the algorithm; how the engine schedules within it is an implementation concern (see Architecture).

stoatflow:
  commit-barrier:
    interval-ms: 500
    min-interval-ms: 150
    max-interval-ms: 5000
    timeout-ms: 60000

There are several finer self-tuning knobs under commit-barrier.* (e.g. commit-duration-ema-alpha, initial-max-epoch-records). They have sensible defaults and rarely need changing — the full list is in the reference.

Processing guarantee

Controls the commit protocol. Both modes use barriers for state-flush and offset-commit synchronization; the difference is the transaction boundary.

stoatflow:
  processing-guarantee: EXACTLY_ONCE

See Exactly-once for the full semantics and the at-least-once recovery contract.

State and changelog

State stores are RocksDB-backed by default and durably backed by Kafka changelog topics, so state survives restarts. See State stores and State and thread safety.

State directory and memory

stoatflow:
  state:
    dir: /var/lib/stoatflow/state
    uncommitted-max-bytes: 268435456
  rocks-db:
    preset: DEFAULT

Changelog topics

stoatflow:
  changelog:
    enabled: true
    replication-factor: 3

Event-time and watermarks

Watermarks track event-time progress for windowing, late-record handling, and event-time timers. The engine auto-detects whether your topology needs event-time tracking and bypasses it on the hot path when it doesn't. See Event time and watermarks.

stoatflow:
  watermark:
    max-out-of-orderness-ms: 10000
    idleness-timeout-ms: 300000
  processing:
    timestamp-coordination-mode: AUTO

Putting it together

A representative production stoatflow: block:

stoatflow:
  application-id: order-processor
  bootstrap-servers: kafka-1:9092,kafka-2:9092,kafka-3:9092

  default-key-serde: org.apache.kafka.common.serialization.Serdes$StringSerde
  default-value-serde: org.apache.kafka.common.serialization.Serdes$StringSerde

  processing-guarantee: EXACTLY_ONCE

  lanes:
    count: 32
    queue-capacity: 500

  commit-barrier:
    interval-ms: 500
    min-interval-ms: 150
    max-interval-ms: 5000
    timeout-ms: 60000

  state:
    dir: /var/lib/stoatflow/state
    uncommitted-max-bytes: 268435456

  changelog:
    enabled: true
    replication-factor: 3

  watermark:
    max-out-of-orderness-ms: 10000
    idleness-timeout-ms: 300000

Next steps

• Configuration reference — every :core key with type, default, and bounds.
• Runtime config () — the runtime.* block (HTTP, metrics, endpoints, health).
• Kafka client config — passthrough consumer/producer properties and the layered merge order.
• Defaults and presets — the opinionated defaults and how to override them.
• Tuning — picking lane count, barrier cadence, and memory caps from observed behaviour.