The migration tool

stoatflow-migration-tool is a standalone, offline CLI that carries Kafka Streams state into StoatFlow: it reads your KS app's changelog topics, writes byte-translated StoatFlow changelog topics under the new application-id, seeds the emit-frontier companions where asked, and carries the input consumer-group offsets over — so the StoatFlow app's first start is a normal full restore that resumes exactly where KS stopped. When to choose this path over reprocessing is covered on Migration carrying state; this page is the how-to.

Getting the tool

Your Reposilite credentials (the same ones your build uses for StoatFlow artifacts) fetch the fat jar directly — match the tool version to the StoatFlow version you are migrating to (currently 1.0.0-rc.1):

curl -u customer-<slug>:<token> -O \
  "https://maven.stoatflow.io/releases/io/stoatflow/stoatflow-migration-tool/<version>/stoatflow-migration-tool-<version>-all.jar"

java -jar stoatflow-migration-tool-<version>-all.jar --help

Teams that prefer resolving through their build can depend on io.stoatflow:stoatflow-migration-tool:<version>:all and copy the artifact — it is a plain executable jar, JVM 17+.

The migration config

One YAML file drives all commands:

kafka:
  bootstrap.servers: broker:9092
  # + optional security client props passthrough (security.protocol, sasl.*, ssl.*)

source:                       # the Kafka Streams app being migrated FROM
  applicationId: my-ks-app
  processingGuarantee: exactly_once     # REQUIRED: exactly_once | at_least_once
  inputTopics: [orders, customers]      # topics whose group offsets carry over
  ignoreStores: []                      # KS stores deliberately left behind (e.g. a drained suppress buffer)

target:                       # the StoatFlow app being migrated TO
  applicationId: my-sf-app
  changelogNumPartitions: 1             # must match the SF app's config (default 1)
  replicationFactor: 3

stores:
  - ksName: order-totals                # KS store name (changelog: {ks-app}-order-totals-changelog)
    sfName: order-totals                # SF store name (default = ksName; set when the port renamed a store)
    type: kv-timestamped                # kv | kv-timestamped | window | window-timestamped | window-duplicates |
                                        #   session | versioned | fk-subscription | outer-join
    extraTopicConfigs: {}               # optional per-store topic-config overrides
    recordHeaders: false                # true iff the store is headers-aware (KIP-1271) — carries the
                                        #   KS changelog's native record headers through
    emitFrontier: true                  # OnWindowClose aggregations only: seed the {sfName}-emitfrontier
                                        #   companion so migrated closed windows aren't re-emitted
  - ksName: my-table-source
    type: kv                            # SF table() source stores are plain KV — see the classification table
    sourceTopic: customers              # declares a source-KTable materialization; plan resolves its path

The type values and the ksName → sfName mapping follow the classification table — get kv vs kv-timestamped right, it is byte-silent if wrong (and plan's heuristics cannot catch that particular pair; the post-start spot-check below can).

source.processingGuarantee is required because it changes what a safe quiesce means: under at_least_once, a clean shutdown is mandatory — a crashed ALO app passes the group-empty check after its session timeout with changelog state ahead of its committed input offsets, and migrating that snapshot bakes at-least-once duplication into the carried state. Exactly-once snapshots are crash-safe.

The four commands

java -jar stoatflow-migration-tool-<v>-all.jar plan         -c migration.yaml
java -jar stoatflow-migration-tool-<v>-all.jar translate    -c migration.yaml [--force]
java -jar stoatflow-migration-tool-<v>-all.jar seed-offsets -c migration.yaml
java -jar stoatflow-migration-tool-<v>-all.jar verify       -c migration.yaml

plan — verify everything, write nothing

Runs every preflight and prints the migration plan. Each check exists because the failure it catches is otherwise silent:

Run it as a dry run while the KS app is still up (expect only the "group not empty" error), and again for real inside the quiesce window.

translate — seed the StoatFlow changelogs

Per store: creates {sf-app-id}-{store}-changelog with StoatFlow's exact topic configs, consumes the KS changelog (read_committed, from earliest to the captured stable offset), applies the store type's byte-translation rule, and produces with the default partitioner — reproducing StoatFlow's own changelog placement. For every store marked emitFrontier: true it additionally seeds the {sfName}-emitfrontier companion with that store's own max changelog record timestamp.

Re-run semantics: a re-run requires an empty target topic or --force (delete + recreate + reseed). There is no partial resume.

seed-offsets — carry the input offsets

Copies the KS group's committed offsets for the configured inputTopics into the (empty) target group. Must run before the StoatFlow app's first start. Recommendation: set AutoOffsetReset.none() on migrated sources for the first start — a partial or failed seeding then fails loud instead of silently reprocessing from earliest or skipping to latest.

verify — prove the seeding is faithful

Per store: record count + an order-independent streaming checksum of the seeded topic versus a re-read of the KS source through the same translation, plus per-partition spot byte-comparisons, the partition-count assert, and the emit-frontier seed check.

Division of labor worth understanding: verify proves the seeding is a faithful, complete application of the tool's translation. Translation correctness is pinned by the tool's own test suite against the real Kafka Streams classes — and a misclassified type reproduces identically on both sides of verify's comparison, which is why the post-start spot-check below is a required gate, not a nicety.

The cutover runbook

The shape of the whole thing — and, more importantly, where it stops being reversible. The tool never writes to a Kafka Streams topic; it reads them and writes new StoatFlow ones, which is exactly why every step before the first StoatFlow start can be abandoned for free.

1. Prepare (KS still running). Port the code (automated port); name all migrated stores explicitly on both sides; write the migration config; dry-run plan.
2. Quiesce. Stop input production if the SLA requires; let KS drain its repartition topics; if suppress is in play, reach a windows-closed point first so the buffer drains (stream time only advances with records — see the caveats); cleanly stop the KS app. Under ALO, a clean shutdown is mandatory.
3. plan — all checks green; consistency point captured.
4. translate — seed all store changelogs (+ emit-frontier companions). Duration is proportional to total changelog bytes; the KS app stays down.
5. seed-offsets — carry input offsets into the target group.
6. verify — counts, checksums, partition counts green.
7. First StoatFlow start — a forced full restore of all seeded changelogs. Watch the stoatflow.restoration.* meters; /health/ready gates traffic until restoration completes; confirm /offsets. Then the required spot-check: interactive-query reads against the still-stopped KS state (counts match, a sampled key returns the expected value). This is the only check that catches a misclassified store type end-to-end.
8. Validate & switch. Bounded side-by-side output inspection where the topology allows, then point downstream at the StoatFlow outputs (or simply let it continue producing to the same sinks).
9. Retire. After sign-off: delete the KS deployment, its group, and its internal topics.

The rollback line

Before step 7, the KS side is untouched — the tool only reads KS topics; it writes only new StoatFlow topics and the new group. Rollback = restart the KS app; delete the seeded topics and group at leisure.

The point of no return is the first StoatFlow transactional commit into shared sinks (step 7 onward). Rolling back to KS after that means KS resumes from its own older committed offsets and re-produces output StoatFlow already produced — downstream duplication that exactly-once cannot dedupe across two applications. Keep the KS deployment deployable until sign-off.

Where to go next

• Migration carrying state — when to translate vs reprocess, the per-store support matrix, the classification table, and the boundary caveats.
• AI assistants — the stoatflow-port-from-ks skill walks the tool's commands and store classification with you.
• Architecture: lifecycle — the per-store restore decision the first start runs.
• Reusing your Kafka Streams dashboards — watch the restore and the cutover on the dashboards you already have.
• Suppress-heavy topology or versioned stores at scale? Get in touch — those are the shapes we still migrate hands-on.