Configuration reference

Curated reference of the StoatFlow configuration keys — stoatflow.* (engine) and runtime.* (HTTP, metrics, health) — with type, default, and a one-line description per key.

Every StoatFlow application is configured from a single application.yaml (loaded by StoatFlowRuntime.fromConfig(...)), plus environment-variable and system-property overrides. This page is a curated reference of the keys you are most likely to set, grouped by area, each with its type, default, and a one-line description.

The authoritative, always-current list of every key — including bound constraints, enums, and examples — is the JSON Schema shipped in the runtime artifact: runtime/src/main/resources/schemas/stoatflow-config-schema.json. IDEs (VS Code with the Red Hat YAML extension, IntelliJ) pick it up automatically for autocompletion and validation. This page omits a handful of advanced, fine-grained tuning knobs; the schema lists them all.

For the configuration model — the four override layers, env-var naming, and how stoatflow.* maps onto the core engine — see Configuration model. For Kafka client passthrough, see Kafka client config.

How to read this page

Key — the YAML path under stoatflow: or runtime:. Nest by the dots (commit-barrier.interval-ms lives under stoatflow.commit-barrier.interval-ms).
Type — YAML scalar type or enum value set.
Default — the value used when the key is omitted. — means no default (the value must be set, or it is null/unset).
Defaults shown as expressions (e.g. max(2, CPU cores)) are resolved at startup from the host.

Secrets (the license key, Kafka SASL passwords, Schema Registry credentials) should be supplied via environment-variable interpolation (${STOATFLOW_LICENSE_KEY}) — never hard-coded in committed YAML. See License configuration.

stoatflow — top level

The required and most common engine keys.

Key	Type	Default	Description
`application-id`	string	— (required)	Unique application identifier; used for the consumer group and transactional id.
`bootstrap-servers`	string	`localhost:9092`	Kafka bootstrap servers (comma-separated).
`processing-guarantee`	enum `EXACTLY_ONCE` \| `AT_LEAST_ONCE`	`EXACTLY_ONCE`	Commit protocol. Exactly-once uses Kafka transactions; at-least-once uses non-transactional commit (lower latency, duplicates possible on crash recovery).
`default-key-serde`	string (FQCN)	—	Fully qualified class name of the default key Serde when none is given explicitly.
`default-value-serde`	string (FQCN)	—	Fully qualified class name of the default value Serde when none is given explicitly.
`schema-registry-url`	string	—	Confluent Schema Registry URL. When set, propagated to serdes and enables the Schema Registry health check.
`deserialization-exception-handler`	string (FQCN)	—	Handler class deciding CONTINUE vs FAIL on deserialization errors.
`production-exception-handler`	string (FQCN)	—	Handler class for serialization and send errors.
`processing-exception-handler`	string (FQCN)	—	Handler class for exceptions thrown by `process()`.
`rocks-db-config-setter`	string (FQCN)	—	Custom `RocksDBConfigSetter` for advanced per-store RocksDB tuning.
`application-server`	string (`host:port`)	`localhost:<http.port>`	Advertised host:port for interactive-query metadata (KS `application.server`). Single instance: identifies this instance as the active host for all stores.

See Error handling and DLQ for the built-in exception handler classes.

stoatflow.lanes

Key-affinity lane parallelism. See Lanes and parallelism.

Key	Type	Default	Description
`lanes.count`	integer	`max(2, CPU cores)`	Number of key-affinity lanes (parallel processing units).
`lanes.queue-capacity`	integer	`300`	Capacity of each lane's queue; controls backpressure.
`lanes.thread-type`	enum `VIRTUAL` \| `PLATFORM`	`VIRTUAL`	Lane thread type. Virtual (Project Loom) by default; platform threads available for the JNI RocksDB backend.

stoatflow.commit-barrier

Commit-barrier cadence and the bounds the engine self-tunes within. The interval between barriers is adaptive at runtime; you set the seed and the floor/ceiling. See Exactly-once.

Key	Type	Default	Description
`commit-barrier.interval-ms`	integer	`500`	Seed interval between commit barriers; the initial value before adaptive scheduling takes over.
`commit-barrier.min-interval-ms`	integer	`150`	Lower bound on the barrier-to-barrier interval.
`commit-barrier.max-interval-ms`	integer	`5000`	Upper bound on the barrier-to-barrier interval.
`commit-barrier.timeout-ms`	integer	`60000`	Timeout for barrier completion. Bounds the commit critical path and cascades to the producer's `transaction.timeout.ms`, `max.block.ms`, and `delivery.timeout.ms`. Must be greater than `interval-ms`.
`commit-barrier.dedicated-commit-thread`	boolean	`true`	Run commits on a dedicated thread so a lane is not stalled during the Kafka transaction commit and state flush.
`commit-barrier.max-epoch-records`	integer	— (no cap)	Optional hard upper bound on records committed per epoch. Unset means the engine sizes epochs adaptively within the interval bounds above.
`commit-barrier.max-engine-restarts`	integer	`5`	Max fault-triggered in-place engine restarts within the rolling window below before the instance escalates to a terminal shutdown. `0` disables in-place restart recovery.
`commit-barrier.engine-restart-window-ms`	integer	`300000` (5 min)	Rolling window over which fault-triggered engine restarts are counted. Bounds a recurring fault to `max-engine-restarts` restarts per window instead of looping.

The schema exposes additional fine-grained commit-barrier tuning factors (smoothing and growth parameters for the adaptive epoch sizer). These rarely need changing — leave them at their defaults unless directed by support. They are listed in full in the JSON schema.

stoatflow.state

State-store storage, caching, and restoration. See State stores and Migration (with data).

Key	Type	Default	Description
`state.dir`	string	`${java.io.tmpdir}/stoatflow`	Directory for state-store data. Put it on fast local storage (SSD); in Kubernetes, on a persistent volume.
`state.uncommitted-max-bytes`	integer (bytes)	`268435456` (256 MiB)	Global cap on uncommitted state bytes across all stores; exceeding it triggers an early commit barrier.
`state.restoration-enabled`	boolean	`true`	Restore state from changelog topics on startup when local state is missing.
`state.restoration-pool-size`	integer	`CPU cores × 2`	Restoration worker pool size; each worker uses its own consumer for parallel fetch.
`state.restoration-batch-size`	integer	`20000`	Records per RocksDB write batch during restoration.
`state.sst-restoration-enabled`	boolean	`false`	Opt-in SST-file ingestion for full cold-start KV restoration (bypasses memtable/L0).
`state.parallel-store-commits`	boolean	`true`	Commit multiple stores in parallel at barrier time.
`state.parallel-store-commit-threads`	integer	`CPU cores`	Max platform threads for parallel store commits (used only with 2+ transactional stores).
`state.cleanup-delay-ms`	integer	`600000` (10 min)	Delay before deleting RocksDB directories for stores no longer in the topology. `0` = immediate, `-1` = never delete (detect + warn only).
`state.cleanup-dir-max-age-ms`	integer (ms)	`-1` (disabled)	Age threshold for purging local per-store state directories untouched for at least this long at startup (KIP-1259); they are rebuilt from the changelog. Set ≥ the changelog's `delete.retention.ms`. Distinct from `cleanup-delay-ms` (orphaned-store cleanup).
`state.segment-checkpoint-records`	integer	`100000`	Periodic checkpoint cadence for segmented window/session stores, by record count. These stores run with the WAL off and become durable only on `close()`, so a checkpoint (flush dirty segments, then the offsets) bounds how much of the changelog an ungraceful restart — or a hot-standby promotion — has to re-apply. Whichever-first with `state.segment-checkpoint-ms`. Deliberately coarse to limit SST/compaction churn.
`state.segment-checkpoint-ms`	integer (ms)	`30000` (30 s)	The same checkpoint cadence by wall-clock time; whichever-first with `state.segment-checkpoint-records`.

Advanced SST knobs (state.sst-buffer-max-bytes, state.sst-flush-interval-files) and state.flush-min-entries-per-chunk are in the schema.

stoatflow.changelog

Changelog topics that back state stores. See State stores.

Key	Type	Default	Description
`changelog.enabled`	boolean	`true`	Write state mutations to Kafka changelog topics within the transaction.
`changelog.create-topics-if-not-exist`	boolean	`true`	Auto-create changelog topics (with `cleanup.policy=compact`) when missing.
`changelog.replication-factor`	integer	`-1` (broker default)	Replication factor for auto-created changelog topics.
`changelog.num-partitions`	integer	`1`	Partition count for auto-created changelog topics (single instance: 1 is usually sufficient).

stoatflow.watermark

Event-time watermark generation. See Event time and watermarks.

Key	Type	Default	Description
`watermark.max-out-of-orderness-ms`	integer	`10000`	Allowed out-of-orderness before an event is considered late.
`watermark.idleness-timeout-ms`	integer	`300000` (5 min)	Partitions idle for this long are excluded from the global watermark. `0` disables.
`watermark.auto-interval-ms`	integer	`200`	Periodic watermark emission interval.

stoatflow.processing

Processing behavior. The keys below are the public, commonly-set ones; the schema additionally exposes a number of low-level dispatch/yield tuning knobs that should normally be left at defaults.

Key	Type	Default	Description
`processing.mdc-enabled`	boolean	`false`	Populate logging MDC keys (`laneId`, `sourceTopic`, `sourcePartition`, `sourceOffset`) during processing.
`processing.max-task-idle-ms`	integer	`0`	Wait time for empty partition buffers to preserve cross-partition timestamp order. `-1` = process immediately, `0` = wait only while broker has lag, `>0` = extra wait.
`processing.buffered-records-per-partition`	integer	`2000`	Max records buffered per partition before pausing consumption (must be ≥ `max.poll.records`).
`processing.consumer-poll-timeout-ms`	integer	`20`	Consumer `poll()` fallback timeout.
`processing.dispatch-mode`	enum `AUTO` \| `DIRECT` \| `LANE`	`LANE`	Record dispatch mode. `LANE` routes via key-affinity lanes; `AUTO` may pick `DIRECT` for simple stateless topologies; `DIRECT` is experimental.
`processing.parallel-deser-mode`	enum `AUTO` \| `ON` \| `OFF`	`AUTO`	Parallel deserialization. `AUTO` enables it when per-record cost is high enough; `ON`/`OFF` force it.
`processing.timestamp-coordination-mode`	enum `AUTO` \| `ENABLED` \| `DISABLED`	`AUTO`	Event-time coordination. `AUTO` derives from topology traits; override only for debugging.
`processing.timestamp-ordered-dispatch`	boolean	`true`	Dispatch records in global timestamp order across partitions.
`processing.wall-clock-advance-interval-ms`	integer	`10`	Cadence for wall-clock punctuators and scheduled sources.
`processing.state-transition-history-size`	integer	`20`	Number of past app-state transitions retained for the `/state` endpoint.

stoatflow.validation

Startup and single-instance safety checks. See How StoatFlow differs from KS.

Key	Type	Default	Description
`validation.ensure-explicit-naming`	boolean	`true`	Fail startup if any topology operation uses an auto-generated name (KIP-1111).
`validation.validate-complete-assignment`	boolean	`true`	Wait for and verify that all source-topic partitions are assigned at startup.
`validation.partition-assignment-timeout-ms`	integer	`60000`	Timeout to wait for complete partition assignment after subscribe.
`validation.pre-flight-consumer-group-check`	boolean	`false`	Check for active consumer-group members before starting (prevents accidental duplicate instances).

stoatflow.shutdown

Graceful shutdown behavior. See Probes.

Key	Type	Default	Description
`shutdown.timeout-ms`	integer	`25000`	Timeout for graceful shutdown; lets in-flight barriers complete.
`shutdown.leave-group-on-close`	boolean	`false`	Whether the consumer sends a LeaveGroup on close. Disabled (with static membership) gives faster restart reassignment.
`shutdown.hard-exit-on-shutdown-hang`	boolean	`true`	If graceful shutdown stalls past its budget, force a clean process exit so Kubernetes restarts the pod.
`shutdown.hard-exit-budget-ms`	integer	`2 × shutdown.timeout-ms`	Budget before the hard-exit fallback fires. Must be ≥ `shutdown.timeout-ms`.

stoatflow.commit-stall

Commit-pipeline freeze detection. These bounds also drive the liveness probe's stall check. See Probes.

Key	Type	Default	Description
`commit-stall.threshold-ms`	integer	`45000`	If commit work stalls longer than this, a thread dump is captured, graceful shutdown begins, and `/health/live` returns DOWN. Must be less than `commit-barrier.timeout-ms`. `0` disables.
`commit-stall.poll-interval-ms`	integer	`5000`	How often the stall check runs.
`commit-stall.dump-enabled`	boolean	`true`	Emit a thread dump to logs when a stall is detected.

stoatflow.rocks-db

RocksDB preset and backend. See State stores.

Key	Type	Default	Description
`rocks-db.preset`	enum `DEFAULT` \| `LOW_MEMORY` \| `HIGH_PERFORMANCE`	`DEFAULT`	Memory preset. `DEFAULT` ≈ 256 MB, `LOW_MEMORY` ≈ 64 MB, `HIGH_PERFORMANCE` ≈ 1 GB.
`rocks-db.backend`	enum `AUTO` \| `FFM` \| `JNI`	`AUTO`	RocksDB binding. `AUTO` picks the faster path per runtime — FFM on the JVM, JNI under a GraalVM native image. `FFM` has lower per-call overhead on HotSpot; `JNI` uses the rocksdbjni bindings.

stoatflow.caching

Downstream emission suppression for KTables. See Aggregations.

Key	Type	Default	Description
`caching.emission-suppression-enabled`	boolean	`true`	Compact multiple writes to the same key within a barrier interval, emitting only the final value.
`caching.max-entries`	integer	`2147483647` (unbounded)	Per-store cached-entry cap before an early barrier. Unbounded by default so it never bites high-cardinality workloads — bytes are the default lever; set a finite value for a count-based bound.
`caching.max-estimated-bytes`	integer (bytes)	`268435456` (256 MiB)	Max estimated cached bytes per store before an early barrier.

stoatflow.timer-backend

Timer storage backend for the Processor API. See Processor API.

Key	Type	Default	Description
`timer-backend.type`	enum `HEAP` \| `PERSISTENT` \| `HYBRID`	`HYBRID`	`HEAP` = in-memory (fast, unbounded), `PERSISTENT` = RocksDB only (memory-bounded), `HYBRID` = heap cache + RocksDB.
`timer-backend.changelog-enabled`	boolean	`true`	Log timer mutations to a Kafka changelog for durability (applies to all types).
`timer-backend.cache-horizon-ms`	integer	`60000`	HYBRID only: how far ahead timers are cached in the heap. Must be > `load-interval-ms`.
`timer-backend.load-interval-ms`	integer	`10000`	HYBRID only: how often the loader pulls upcoming timers from RocksDB. Must be < `cache-horizon-ms`.

stoatflow.ktable / stoatflow.fk-join / stoatflow.dsl

Key	Type	Default	Description
`ktable.auto-reuse-source-topics`	boolean	`true`	Reuse compacted KTable source topics for restoration instead of separate changelog topics.
`fk-join.use-reverse-index-cache`	boolean	`true`	Maintain an in-memory reverse index for O(1) FK-join right-side lookups (disable to save heap on very large FK tables).
`dsl.store-format`	enum `DEFAULT` \| `HEADERS`	`DEFAULT`	Global default on-disk format for DSL-materialized stores (KIP-1285). `HEADERS` makes DSL aggregations persist record headers (KIP-1271). Per-store `Materialized.withRecordHeaders()` / `withoutRecordHeaders()` always wins.

stoatflow.kafka

Kafka client passthrough. Values under these maps are merged on top of StoatFlow's framework defaults; forced overrides (e.g. bootstrap.servers, group.id, enable.auto.commit=false) cannot be changed. Full resolution order in Kafka client config.

Key	Type	Default	Description
`kafka.consumer`	map<string,string>	`{}`	Additional consumer properties (e.g. `auto.offset.reset`, `max.poll.records`).
`kafka.producer`	map<string,string>	`{}`	Additional producer properties (e.g. `compression.type`, `linger.ms`).
`kafka.restoration-consumer`	map<string,string>	`{}`	Overrides for the restoration consumer only.

stoatflow.license

Runtime license. See License configuration.

Key	Type	Default	Description
`license.key`	string	—	License key string. Use `${STOATFLOW_LICENSE_KEY}` interpolation; do not hard-code.
`license.file`	string	—	Path to a `chmod 600` file holding the key on one line. Mutually exclusive with `key` (`key` wins).
`license.environment`	string	—	Deployment environment label feeding the machine fingerprint. Required for the Production tier.
`license.cache-dir`	string	`~/.stoatflow/license-cache/`	Offline-cache directory. In Kubernetes, mount on a persistent volume so it survives restarts.
`license.verbose-banner`	boolean	`false`	Include the customer name in the startup banner.

License env vars and -D system properties (STOATFLOW_LICENSE_*) take precedence over these YAML values.

stoatflow.ha

Opt-in hot-standby high availability. With mode: off (default) the application is a single instance. See High availability.

Key	Type	Default	Description
`ha.mode`	enum	`off`	`off` (single instance) or `active-standby` (hot-standby cluster: one active + one or more warm standbys).
`ha.pod-id`	string	`POD_NAME` → `HOSTNAME` → hostname	Stable per-pod identity distinguishing the peers.
`ha.coordination-topic`	string	`__stoatflow_<applicationId>_ha`	Single-partition, compacted, auto-created topic the cluster coordinates through (heartbeats + the promotion token).
`ha.desired-standbys`	integer	`1`	Caught-up standby spares the readiness gate protects. `1` is the classic active/passive pair; raise it (with `replicas`) for more redundancy. Drives the redundancy floor on rolling deploys.
`ha.max-standbys`	integer	`3`	Ceiling on standbys, bounding changelog fan-out (each standby is one more changelog reader). Total instances ≤ `max-standbys + 1`.
`ha.failover-priority`	integer	`0`	Per-pod tie-break hint ordering candidates within the equal-lag bucket (higher wins). Never overrides lag or the hash floor.
`ha.promotion-settle-window-ms`	integer	`500`	After claiming the promotion token, how long the winner confirms it is the sole claimant before fencing (single-promoter guarantee). Bounded `1..ha.staleness-threshold-ms`.
`ha.heartbeat-ms`	integer	`1000`	How often each pod publishes its liveness and role.
`ha.staleness-threshold-ms`	integer	`5000`	Gap without a peer heartbeat before it's considered stale. Must be ≥ `ha.heartbeat-ms`.
`ha.staleness-misses`	integer	`3`	Consecutive missed checks before a standby is elected to promote (debounce).
`ha.acceptable-recovery-lag`	integer	`50000`	Total committed-changelog lag (records, summed across all changelog partitions) at/below which a standby is caught up (`READY_STANDBY`) and promotion-eligible without the grace fallback. A single total sum, not per-task like KS's `acceptable.recovery.lag` (10000). Correctness never depends on it (restore-before-process does); mostly a raise-me lever.
`ha.promotion-grace-ms`	integer	`5000`	Grace window after which the least-behind standby auto-promotes when no standby is within `acceptable-recovery-lag` (availability-first; eligibility re-checked continuously).
`ha.promotion-restore-no-progress-timeout-ms`	integer	`60000`	Progress-aware deadline for restore-before-process — fails only on a window with zero records applied (genuinely stuck), then self-terminates → K8s restart. Not the commit-path barrier timeout.

runtime.http

The built-in HTTP server. See REST API.

Key	Type	Default	Description
`runtime.http.enabled`	boolean	`true`	Enable the HTTP server (health, metrics, info, topology, control endpoints).
`runtime.http.port`	integer	`8080`	Port to bind.
`runtime.http.host`	string	`0.0.0.0`	Bind address.
`runtime.http.backlog`	integer	`50`	Max pending connections in the socket backlog.
`runtime.http.debug.enabled`	boolean	`true`	Register the `/debug/threads` and `/debug/barriers` diagnostic endpoints. Disable to reduce attack surface in hardened deployments.

runtime.metrics

Micrometer / Prometheus metrics. See Metrics.

Key	Type	Default	Description
`runtime.metrics.enabled`	boolean	`true`	Enable metrics collection and the `/metrics` Prometheus endpoint.
`runtime.metrics.prefix`	string	`stoatflow`	Prefix for all metric names.
`runtime.metrics.recording-level`	enum `info` \| `debug` \| `trace`	`info`	Metric detail level. `info` is production-safe; higher levels add cardinality/overhead.
`runtime.metrics.common-tags`	map<string,string>	`{}`	Tags applied to all metrics (e.g. `env`, `region`).
`runtime.metrics.bind-jvm-metrics`	boolean	`true`	Bind JVM metrics (memory, GC, threads).

runtime.endpoints

Visibility of the /info and /config endpoints. See REST API.

Key	Type	Default	Description
`runtime.endpoints.config.enabled`	boolean	`true`	Enable the `/config` endpoint (merged config with sensitive values masked).
`runtime.endpoints.info.show-app`	boolean	`true`	Include application info (applicationId) in `/info`.
`runtime.endpoints.info.show-java`	boolean	`true`	Include Java runtime info in `/info`.
`runtime.endpoints.info.show-kafka`	boolean	`true`	Include Kafka config (bootstrap servers) in `/info`.
`runtime.endpoints.info.show-uptime`	boolean	`true`	Include uptime in `/info`.
`runtime.endpoints.info.show-state`	boolean	`true`	Include state + transition history in `/info`.
`runtime.endpoints.info.show-watermarks`	boolean	`true`	Include watermark info in `/info`.
`runtime.endpoints.info.show-endpoints`	boolean	`true`	Include the list of available endpoints in `/info`.

runtime.health

Health-check timeouts. See Health checks.

Key	Type	Default	Description
`runtime.health.kafka-broker.timeout-ms`	integer	`2000`	Timeout for the Kafka broker connectivity check.
`runtime.health.schema-registry.timeout-ms`	integer	`5000`	Timeout for the Schema Registry check (auto-enabled when `schema-registry-url` is set).

logging

Per-package log levels applied at startup (overrides logback.xml).

Key	Type	Default	Description
`logging.level`	map<string, enum>	`{}`	Map of logger name → level (`TRACE`/`DEBUG`/`INFO`/`WARN`/`ERROR`/`OFF`). Use `ROOT` for the root logger.

logging:
  level:
    ROOT: INFO
    io.stoatflow: DEBUG
    org.apache.kafka: WARN

Example

A representative application.yaml touching the common keys:

stoatflow:
  application-id: order-processor
  bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:-localhost:9092}
  processing-guarantee: EXACTLY_ONCE

  license:
    key: ${STOATFLOW_LICENSE_KEY}
    environment: ${DEPLOY_ENV:-dev}

  lanes:
    count: 16

  state:
    dir: /var/lib/stoatflow/state

  rocks-db:
    preset: DEFAULT

runtime:
  http:
    enabled: true
    port: ${HTTP_PORT:-8080}
  metrics:
    enabled: true
    common-tags:
      env: ${DEPLOY_ENV:-dev}

Configuration model

Override layers, env-var naming, and YAML-to-engine mapping.

Kafka client config

Consumer/producer passthrough, framework defaults, forced overrides.

Defaults and presets

What StoatFlow sets out of the box and the RocksDB presets.

Tuning

Which knobs to turn for throughput, latency, and memory.

Reference

Lookup material for StoatFlow — configuration keys, REST API, the Gradle plugin and Maven build references, the Kafka Streams compatibility matrix, the metrics catalogue, and the glossary.

REST API reference

Every HTTP endpoint the StoatFlow runtime exposes — path, method, purpose, response shape, content negotiation, status codes, and gating.