Metrics reference

The full catalogue of Prometheus meters StoatFlow exposes on /metrics — name, type, tags, and what each one measures, grouped by area.

The :runtime module exports metrics on GET /metrics in Prometheus text format, backed by a Micrometer PrometheusMeterRegistry. This page is the meter catalogue: every StoatFlow meter the current release registers, grouped by area, with its type, tags, and a one-line description. For how to enable, scrape, and configure metrics — recording levels, naming rewrites, PromQL examples — see Metrics. For what to alert on, see Observability.

Contract. This catalogue lists the stable meter set. The live /metrics scrape is authoritative for the exact tag sets and any point-release additions — when in doubt, trust the scrape.

How to read this page

Meter — the canonical dotted metric ID. Prometheus rewrites . to _ and appends the type suffix (_total for counters; _seconds_count / _seconds_sum / _seconds_max for timers), so stoatflow.barrier.commit.latency is scraped as stoatflow_barrier_commit_latency_seconds_*.
Tags — per-meter tags beyond the common ones. Every StoatFlow meter carries application_id; the registry-level application tag plus anything under runtime.metrics.common-tags is applied to every meter on the endpoint, including JVM and Kafka-client meters.
Recording level — meters marked (debug) in the description only appear with runtime.metrics.recording-level: debug; everything else is exported at the default info level.
The commit-path latency timers publish client-side P50/P95/P99 quantiles as {quantile="…"} series on the base _seconds name — there are no _bucket histograms.

Application and engine

Meter	Type	Tags	What it measures
`stoatflow.client.state`	Gauge	—	Application state ordinal: `CREATED=0`, `STARTING=1`, `VALIDATING_STATE=2`, `RESTORING=3`, `RUNNING=4`, `DRAINING=5`, `PAUSED=6`, `STOPPING=7`, `STOPPED=8`, `ERROR=9`.
`stoatflow.client.lane.alive`	Gauge	—	Currently alive/healthy lanes.
`stoatflow.client.uptime.seconds`	Gauge	—	Seconds since the application started.
`stoatflow.client.start.time.epoch.seconds`	Gauge	—	Application start time (epoch seconds).
`stoatflow.lanes`	Gauge	`sub_topology`, `chain`	Configured key-affinity lanes per sub-topology; `sum(stoatflow_lanes)` gives the cross-topology total.
`stoatflow.engine.restart.total`	Counter	`disposition`, `target`, `trigger`	Successful in-place engine restarts (fault recovery, programmatic, HA demote).
`stoatflow.topology.stateless`	Gauge	—	1 if the topology has no state stores, 0 otherwise.
`stoatflow.topology.event.time.tracking`	Gauge	—	1 if watermark tracking is active, 0 if bypassed.
`stoatflow.topology.dispatch.mode`	Gauge	—	0 for timestamp-ordered dispatch, 1 for FIFO.

Lanes and throughput

Meter	Type	Tags	What it measures
`stoatflow.lane.queue.size`	Gauge	`lane_id`, `chain`	Current lane queue depth (backpressure signal).
`stoatflow.lane.queue.capacity`	Gauge	`sub_topology`, `chain`	Configured lane queue capacity per sub-topology.
`stoatflow.lane.queue.full.total`	Counter	`lane_id`	Times a lane queue was full (backpressure events).
`stoatflow.lane.records.processed.total`	Counter	`lane_id`	Records processed by a lane.
`stoatflow.lane.bytes.processed.total`	Counter	`lane_id`	Bytes processed by a lane.
`stoatflow.lane.process.latency`	Timer	`lane_id`	Per-record processing latency.
`stoatflow.e2e.latency`	Timer	—	Per-record end-to-end latency, record timestamp to processing completion (KS `record-e2e-latency` analogue).

Lane IDs follow the {subtopology}_{lane} convention — e.g. 0_0, 1_7. Aggregate with a regex matcher: sum(stoatflow_lane_queue_size{lane_id=~"1_.*"}).

Commit barriers and transactions

Meter	Type	Tags	What it measures
`stoatflow.barrier.initiated.total`	Counter	—	Barriers initiated.
`stoatflow.barrier.completed.total`	Counter	—	Barriers successfully completed.
`stoatflow.barrier.failed.total`	Counter	`error_type`	Barrier failures, by error type.
`stoatflow.barrier.in.flight`	Gauge	—	Currently pending barriers.
`stoatflow.barrier.latency`	Timer	—	Barrier initiation to completion.
`stoatflow.barrier.alignment.latency`	Timer	—	Time for all lanes to receive the barrier.
`stoatflow.barrier.commit.latency`	Timer	—	Commit latency: producer drain + Kafka TX + state flush (KS `commit-latency` analogue).
`stoatflow.barrier.records.committed.total`	Counter	—	Records committed across barriers.
`stoatflow.barrier.bytes.committed.total`	Counter	—	Bytes committed across barriers.
`stoatflow.barrier.subtopology.received.total`	Counter	`subtopology_id`	Barriers received per sub-topology. (debug)
`stoatflow.barrier.subtopology.pending.lanes`	Gauge	`subtopology_id`	Pending lane completions per sub-topology. (debug)
`stoatflow.barrier.interval.target.ms`	Gauge	—	Current target barrier-to-barrier interval (ms).
`stoatflow.barrier.commit.duration.estimate.ms`	Gauge	—	Smoothed estimate of commit duration (ms).
`stoatflow.barrier.epoch.max.records`	Gauge	—	Current dynamic epoch record limit.
`stoatflow.barrier.epoch.actual.records`	Gauge	—	Records dispatched in the last epoch.
`stoatflow.barrier.epoch.memory.cap`	Gauge	—	Memory-derived epoch record cap; `-1` = uncapped.
`stoatflow.barrier.epoch.bytes.per.record`	Gauge	—	Smoothed uncommitted state bytes per record (feeds the memory cap).
`stoatflow.s1.epoch.records.smoothed`	Gauge	—	Smoothed epoch-records average used by the epoch-size controller.
`stoatflow.s1.cap.grow.events.total`	Counter	`reason`	Epoch-cap grow events by binding signal (`record_count`, `memory_pressure`, `stall`, `soft_binding`, `time_grow`).
`stoatflow.s1.cap.decay.events.total`	Counter	—	Epoch-cap decay events.
`stoatflow.barrier.trigger.total`	Counter	`trigger`	Barrier creation count by trigger: `TIME`, `RECORD_COUNT`, `MEMORY_PRESSURE`, `CACHE_PRESSURE`.
`stoatflow.dispatcher.stall.latency`	Timer	—	Dispatcher stall time waiting for an active commit after hitting the epoch record limit.
`stoatflow.barrier.commit.changelog.latency`	Timer	—	Aggregate changelog serialization+send latency per commit epoch.
`stoatflow.barrier.commit.state.flush.latency`	Timer	—	Aggregate state-store flush latency per commit epoch.
`stoatflow.barrier.commit.producer.flush.latency`	Timer	—	Pre-commit producer flush latency.
`stoatflow.barrier.commit.send.offsets.latency`	Timer	—	`sendOffsetsToTransaction` RPC latency.
`stoatflow.barrier.commit.queue.wait.latency`	Timer	—	Barrier wait in the commit queue between alignment and commit-thread pickup.
`stoatflow.barrier.commit.post.tx.latency`	Timer	—	Post-transaction phase latency (guard wait + epoch prepare + send-buffer drain).
`stoatflow.barrier.commit.window.duration`	Timer	—	Duration of the commit-in-progress blocking window. (debug)
`stoatflow.commit.state.flush.guard.wait`	Timer	—	Time the next transaction waited on the previous epoch's async state flush.
`stoatflow.commit.state.flush.async.latency`	Timer	—	Async (off-commit-path) state flush latency.
`stoatflow.commit.state.flush.overlap`	Counter	—	Commits that overlapped a still-running previous flush.
`stoatflow.commit.stall.detected.total`	Counter	—	Commit-pipeline stalls detected by the in-process watchdog.
`stoatflow.commit.stall.duration.ms`	Gauge	—	Age of the most recently observed commit stall (ms).
`stoatflow.transaction.begin.total`	Counter	—	Kafka transactions begun.
`stoatflow.transaction.commit.total`	Counter	—	Kafka transactions committed.
`stoatflow.transaction.abort.total`	Counter	—	Kafka transactions aborted.
`stoatflow.transaction.commit.latency`	Timer	—	Kafka transaction commit latency.
`stoatflow.alo.commit.delegation.wait`	Timer	—	Commit delegation wait (at-least-once mode only).
`stoatflow.alo.commit.sync.latency`	Timer	—	`commitSync` latency (at-least-once mode only).

State stores and caching

Most store meters are per-store (store_name) and require recording-level: debug; the changelog counters and store.num.keys are exported at info.

Meter	Type	Tags	What it measures
`stoatflow.store.get.total`	Counter	`store_name`	Get operations. (debug)
`stoatflow.store.put.total`	Counter	`store_name`	Put operations. (debug)
`stoatflow.store.delete.total`	Counter	`store_name`	Delete operations. (debug)
`stoatflow.store.range.latency`	Timer	`store_name`	Range query latency. (debug)
`stoatflow.store.all.latency`	Timer	`store_name`	Full-iteration latency. (debug)
`stoatflow.store.flush.latency`	Timer	`store_name`	Store flush latency. (debug)
`stoatflow.store.cache.hit.total`	Counter	`store_name`	Store cache hits. (debug)
`stoatflow.store.cache.miss.total`	Counter	`store_name`	Store cache misses. (debug)
`stoatflow.store.changelog.records.written.total`	Counter	`store_name`	Changelog records written.
`stoatflow.store.changelog.bytes.written.total`	Counter	`store_name`	Changelog bytes written.
`stoatflow.store.commit.changelog.latency`	Timer	`store_name`	Per-store changelog serialization+send latency during the commit epoch. (debug)
`stoatflow.store.commit.flush.latency`	Timer	`store_name`	Per-store flush latency during the commit epoch. (debug)
`stoatflow.store.flush.entries`	DistributionSummary	`store_name`	Entry count per epoch flush. (debug)
`stoatflow.store.flush.sub.batches`	DistributionSummary	`store_name`	Sub-batch count per flush (1 = sequential, N = parallel). (debug)
`stoatflow.store.num.keys`	Gauge	`store_name`, `store_type`	Committed key count of an in-memory store (KIP-1250 parity); silent for RocksDB-backed stores.

State restoration

Meter	Type	Tags	What it measures
`stoatflow.restoration.in.progress`	Gauge	—	1 while restoration is running, 0 otherwise.
`stoatflow.restoration.stores.total`	Gauge	—	Total stores to restore.
`stoatflow.restoration.stores.completed`	Gauge	—	Stores that have finished restoring.
`stoatflow.restoration.records.restored.total`	Counter	`store_name`	Records restored, per store.
`stoatflow.restoration.bytes.restored.total`	Counter	`store_name`	Bytes restored, per store.
`stoatflow.restoration.duration.seconds`	Timer	`store_name`	Per-store restoration duration.

Consumer, producer, and buffers

Meter	Type	Tags	What it measures
`stoatflow.consumer.poll.latency`	Timer	—	Consumer poll latency.
`stoatflow.consumer.poll.total`	Counter	—	Poll invocations.
`stoatflow.consumer.poll.records`	DistributionSummary	—	Records per poll.
`stoatflow.consumer.records.consumed.total`	Counter	`topic`, `partition`	Records consumed.
`stoatflow.consumer.bytes.consumed.total`	Counter	`topic`, `partition`	Bytes consumed.
`stoatflow.consumer.lag.records`	Gauge	`topic`, `partition`	Consumer lag, in records.
`stoatflow.consumer.assigned.partitions`	Gauge	—	Number of assigned partitions.
`stoatflow.consumer.partitions.paused`	Gauge	—	Currently paused partitions.
`stoatflow.consumer.partitions.delay.pending`	Gauge	—	Partitions with a pending resume delay.
`stoatflow.consumer.partition.paused.total`	Counter	`topic`, `partition`, `reason`	Partition pause events, by reason.
`stoatflow.consumer.partition.resumed.total`	Counter	`topic`, `partition`, `reason`	Partition resume events, by reason.
`stoatflow.producer.records.produced.total`	Counter	`topic`	Records produced.
`stoatflow.producer.bytes.produced.total`	Counter	`topic`	Bytes produced.
`stoatflow.producer.send.latency`	Timer	`topic`	Producer send latency. (debug)
`stoatflow.buffer.partition.size`	Gauge	`topic`, `partition`	Current partition buffer depth. (debug)
`stoatflow.buffer.partition.capacity`	Gauge	`topic`, `partition`	Partition buffer capacity. (debug)
`stoatflow.buffer.offer.total`	Gauge	`topic`, `partition`	Cumulative records offered to the partition buffer. (debug)
`stoatflow.buffer.poll.total`	Gauge	`topic`, `partition`	Cumulative records polled from the partition buffer. (debug)
`stoatflow.buffer.oldest.age.ms`	Gauge	`topic`, `partition`	Age of the oldest buffered record. (debug)
`stoatflow.buffer.scheduled.source.size`	Gauge	—	Scheduled-source dispatch buffer depth.
`stoatflow.buffer.timer.event.size`	Gauge	—	Event-time timer buffer depth.
`stoatflow.buffer.timer.processing.size`	Gauge	—	Processing-time timer buffer depth.

Dispatcher internals

Meter	Type	Tags	What it measures
`stoatflow.dispatcher.batch.size`	DistributionSummary	—	Records per dispatch batch. (debug)
`stoatflow.dispatcher.batch.held.total`	Counter	—	Batches held because a target lane lacked capacity. (debug)
`stoatflow.dispatcher.dispatch.latency`	Timer	—	Per-batch dispatch latency.
`stoatflow.parallel.deser.activated.total`	Counter	—	Times parallel deserialization was activated.
`stoatflow.queue.offer.failure.total`	Counter	`queue_type`	Internal queue offer rejections (`lane`, `partition_buffer`, `commit`).
`stoatflow.coordinator.select.latency`	Timer	—	Time spent selecting the next timestamp-ordered batch. (debug)
`stoatflow.coordinator.partition.switches`	DistributionSummary	—	Partition switches per selected batch. (debug)

Watermarks and event time

Meter	Type	Tags	What it measures
`stoatflow.watermark.current`	Gauge	—	Current global watermark (epoch ms).
`stoatflow.watermark.lag.ms`	Gauge	—	Wall-clock time minus watermark (event-time lag).
`stoatflow.watermark.advance.total`	Counter	—	Watermark advancement events.
`stoatflow.watermark.partition.current`	Gauge	`topic`, `partition`	Per-partition watermark. (debug)
`stoatflow.watermark.late.records.total`	Counter	`topic`, `partition`	Records arriving after the watermark. (debug)
`stoatflow.watermark.out.of.orderness.max`	Gauge	—	Maximum observed out-of-orderness (ms). (debug)
`stoatflow.watermark.alignment.drift.ms`	Gauge	`topic`, `partition`, `group`	Per-partition drift above the alignment group's minimum watermark. (debug)
`stoatflow.watermark.alignment.group.min`	Gauge	`group`	Minimum watermark across the alignment group (epoch ms). (debug)
`stoatflow.watermark.alignment.partitions.paused`	Gauge	`group`	Partitions currently paused for watermark alignment.

Timers, punctuators, and scheduled sources

Meter	Type	Tags	What it measures
`stoatflow.timer.registered.total`	Counter	`type`	Keyed timers registered (`event_time`, `processing_time`).
`stoatflow.timer.fired.total`	Counter	`type`	Timers fired.
`stoatflow.timer.deleted.total`	Counter	`type`	Timers explicitly deleted.
`stoatflow.timer.queue.size`	Gauge	`type`	Timer dispatch-buffer depth, by timer type.
`stoatflow.timer.fire.latency`	Timer	`type`	Delay between scheduled and actual fire time. (debug)
`stoatflow.timer.fire.late.total`	Counter	`type`	Timers that fired late. (debug)
`stoatflow.punctuator.invocation.total`	Counter	`punctuator_id`, `type`, `mode`	Punctuator invocations.
`stoatflow.punctuator.skip.total`	Counter	`punctuator_id`	Punctuator invocations skipped (previous run still active).
`stoatflow.punctuator.active`	Gauge	—	Currently registered punctuators.
`stoatflow.punctuator.execution.latency`	Timer	`punctuator_id`, `type`	Punctuator execution time. (debug)
`stoatflow.scheduled.source.trigger.total`	Counter	`source_name`, `type`	Scheduled source triggers.
`stoatflow.scheduled.source.records.emitted.total`	Counter	`source_name`	Records emitted by scheduled sources.

Error handling

Meter	Type	Tags	What it measures
`stoatflow.error.total`	Counter	`error_type`, `topic`	Errors by type (`topic` omitted when unknown).
`stoatflow.error.typed.total`	Counter	`category`, `error_type`	Categorised errors (e.g. production failures), by category and type.
`stoatflow.dropped.records.total`	Counter	`reason`	Records dropped before processing (e.g. `null_key`; KS `dropped-records` parity).
`stoatflow.sink.records.dropped.total`	Counter	`sink_name`, `topic`	Records dropped because a custom partitioner returned no partition (KIP-837).
`stoatflow.sink.records.multicast.total`	Counter	`sink_name`, `topic`	Output records produced via multicast — a custom partitioner returned multiple partitions (KIP-837).

High availability

Registered only when hot standby is enabled (ha.mode != off).

Meter	Type	Tags	What it measures
`stoatflow.ha.role`	Gauge	—	`1` when this pod is the active, `0` when standby.
`stoatflow.ha.standby.replication.lag.records`	Gauge	—	Total committed-changelog lag in records across all changelog partitions; `0` when active or caught up.
`stoatflow.ha.standby.replication.applied.records`	Counter	—	Cumulative changelog records the standby applier has applied; `rate(...)` = apply throughput.
`stoatflow.ha.standby.replication.lag.ms`	Gauge	—	Standby replication lag in milliseconds.
`stoatflow.ha.peer.freshness.ms`	Gauge	—	Time since the freshest observed peer's last heartbeat.
`stoatflow.ha.peers`	Gauge	—	Number of observed HA peers.
`stoatflow.ha.restart.required`	Gauge	—	`1` when this pod needs a restart (e.g. after a source-topic partition-count change).
`stoatflow.ha.redundancy.ready.standbys`	Gauge	—	Caught-up (`READY_STANDBY`) fresh standby count, including self.
`stoatflow.ha.redundancy.below.desired`	Gauge	—	`1` when the active has fewer ready standby spares than `ha.desired-standbys`.
`stoatflow.ha.token.epoch`	Gauge	—	Current promotion-token epoch on the metadata topic.
`stoatflow.ha.election.outcome`	Counter	`outcome`	Cumulative election outcomes: `won`, `reclaimed`, `stood_down_fresh`, `stood_down_lost`.
`stoatflow.ha.election.latency.ms`	Gauge	—	Last claim-to-ACTIVE promotion latency (ms); `-1` until the first promotion.

License

Registered once at startup; exported at every recording level. See License configuration.

Meter	Type	Tags	What it measures
`stoatflow.license.valid`	Gauge	`tier`	`1` under a VALID or GRACE_PERIOD license, else `0`.
`stoatflow.license.expiry_timestamp_seconds`	Gauge	—	Unix epoch seconds of the license entitlement deadline.
`stoatflow.license.days_remaining`	Gauge	—	Whole days until expiry; negative within the post-expiry grace window.
`stoatflow.license.grace_remaining_seconds`	Gauge	—	Seconds left in the offline-grace budget; floored at 0.
`stoatflow.license.heartbeat_consecutive_failures`	Gauge	—	Heartbeat ticks failed since the last success.
`stoatflow.license.heartbeat_last_success_timestamp_seconds`	Gauge	—	Unix epoch seconds of the last successful heartbeat.
`stoatflow.license.validations_total`	Counter	`result`	Cold-start validation outcomes, by result.
`stoatflow.license.heartbeats_total`	Counter	`result`	Heartbeat tick outcomes, by result.

JVM, system, and Kafka-client meters

Beyond the stoatflow.* meters above, the same endpoint carries:

JVM and system meters (jvm_*, system_*, process_*) — bound by default (runtime.metrics.bind-jvm-metrics: true): heap/non-heap memory, GC pauses, thread counts, classloading, CPU usage.
Native Kafka client meters (kafka_consumer_*, kafka_producer_*) — the Kafka clients' own JMX metrics surfaced through Micrometer.

These carry the registry common tags (application plus your common-tags) but not application_id. See Metrics for details.

Next steps

Metrics — enabling, scraping, recording levels, naming and tag conventions, PromQL examples.
Observability — wiring /metrics into a monitoring stack and what to alert on.
Configuration reference — the runtime.metrics.* keys.

Kafka Streams compatibility matrix

Operator-by-operator parity between the StoatFlow DSL and the Kafka Streams DSL — implemented, partial, not yet implemented, deprecated, incompatible, and StoatFlow extensions.

Glossary

Concise definitions of the terms used across the StoatFlow docs — lanes, commit barriers, epochs, watermarks, event time, changelog, state stores, serdes, repartitioning, and exactly-once.