The five maps and the signatures read schema.Migrations instead of map[schema.MigrationKey]schema.Migration, and the two derivations are methods on it (targetVersion, minVersion). No behavior change.
Steps were keyed by a bare int that meant "from": map[int]schema.Migration{16: ...}. They are
now keyed by schema.MigrationKey{From, To}, so each step says what it does:
{From: 16, To: 17}: schema.Query(...). Every key was copied, not recomputed (16 became
{16, 17}), so no step moved. A key that does not go exactly one version fails at startup,
next to the gap check. Stored versions are unchanged.
Migrate took the target version as a separate argument, kept in step with the steps by hand:
adding a step without bumping the version constant silently skipped it on existing installs,
while fresh installs got the new tables. The target is now the version after the last step (1
without steps), and the steps must be contiguous, so a forgotten one fails at startup. The
per-store version constants are gone.
Instead of on every start: the schema is defined in one place again. Postgres gets
idx_message_topic_published_id (topic, published, id), SQLite idx_topic_published_time (topic,
published, time), both with IF NOT EXISTS (ntfy.sh got the Postgres one by hand). A downgrade to
v2.31.0 or earlier now needs the schema version set back by hand, as described in the release notes;
from v2.31.1 on, a newer schema only logs a warning (#2059).
On ntfy.sh, v2.30.0's own replay (topic = $1 AND id > $2) was expensive: on (topic, published,
time, id), id > marker is checked inside the index but walks every entry of the topic, or the plan
scans the primary key from the marker across all topics. Replica-1 spent about one vCPU on it.
idx_message_topic_published_id (topic, published, id), created by hand on ntfy.sh on 2026-10-16,
made it ~50x cheaper in the lab. It is now created on every start (IF NOT EXISTS, no schema
version, so rollbacks to 2.30.0 still work).
With that index, the late-written half of the replay (time >= marker AND id < marker) picked it
and walked the topic's history (6 ms -> 104 ms in the lab). The replay is now split by time
instead: rows published in the marker's second or later (time index, from the marker), and rows
written after the marker but published before it (the new index, from the marker; time + 0 keeps
it off the time index). Same result as before, and both parts start at the marker.
The plan test now runs on prod-shaped data (200k rows, a few busy topics) with statistics, and
judges each scan by its index: a scan must start at the marker on that index, and replays must
stay within the topic. It fails on v2.31.0's query and on v2.30.0's without the index.
Each migration step is now a schema.Migration: schema.Query, schema.Func or schema.Nop for additive
steps, wrapped in schema.Breaking for a breaking one. Migrate derives the store's minimum version
(the version after the newest breaking step, or 1) instead of taking it from a per-store constant,
so a breaking migration cannot be added without raising the minimum. The existing steps are all
marked additive; older code never checks the minimum, so classifying them changes nothing.
#2059 lets older code start on a newer schema, which is only safe while migrations are additive.
For the rare breaking migration, each store now records the oldest schema version whose code still
runs on its schema (min_version in the version table). A breaking migration raises it; older code
then fails at startup with a clear error instead of misreading the data.
- schema.Migrate takes minVersion; every store passes its MinSchemaVersion constant (1: no
breaking migration recorded)
- The version table gets a min_version column (default 1), added in place to existing tables;
older code writes only the version and keeps working on the new table
Marking a claimed message published was wrong in the way a reviewer would
notice first: published is read by four other queries (since= replays, polls,
scheduled-message replacement, expiry), so a claimed-but-undelivered message
looked delivered to all of them, and a node dying after the claim lost the
push entirely.
The claim is now its own column. claimed_at holds the row off from the other
nodes, the claim statement is a single CTE that locks what it picks and skips
what another node is already claiming, and MarkPublished stays where it was,
after dispatch. So published still means delivered, and delivery is still at
least once.
The claim expires after two minutes, which is what makes a dead claimer
harmless: nothing would ever deliver its rows again if the claim were
permanent. Two minutes is far more than dispatching a claimed batch takes
(side effects are goroutine handoffs), and the claim also takes at most 1000
rows, so a backlog drains over several ticks instead of one node holding all
of it.
Schema 15 -> 16 adds the column; the migration test asserts the migrated
database is structurally identical to a fresh one, so the ALTER and the CREATE
TABLE cannot drift.
Tests: concurrent callers get disjoint sets, an abandoned claim is invisible
until it expires and claimable afterwards, and a published message never comes
back. Both halves of the reclaim test fail if the claim filter is dropped.
binwiederhier