fix(restore): a restore is an unclean event — the swap runs quiesced and the snapshot's durability stamps never survive it
All checks were successful
CI / Node 22 (push) Successful in 12m13s
CI / Node 24 (push) Successful in 12m8s
CI / Bun (latest) (push) Successful in 12m20s

Two defects with one root, found by the fold-checkpoint work's first
integration gate. (1) THE RACE: restore() never quiesced the generation
store, so a background flush could write into _system/ while the swap was
removing it — observed as ENOTEMPTY mid-swap when a checkpoint stamp landed
between readdir and rmdir. The swap now runs inside the store's exclusive
section (runStateReplacement): flush timer disarmed, pending tier and
checkpoint accumulator discarded BEFORE any directory moves. (2) THE
INHERITED ASSERTION: a snapshot carries its source brain's clean-shutdown
marker and fold checkpoint, but the restored files were bulk-copied without
per-file fsync — the inherited stamps would suppress exactly the recovery
fold that cures a post-restore power cut. reopenAfterRestore now deletes
both stamps before reopening: the open treats the store as uncleanly shut,
folds the restored log into canonical, barrier-syncs what it re-applied,
and stamps fresh — the restored state is durably founded at restore time
instead of borrowing assertions about bytes this disk never synced.

Pinned: restore under in-flight traffic completes; the pre-restore stamp
does not survive; the post-restore stamp is the reopen fold's own, at the
restored watermark.
This commit is contained in:
David Snelling 2026-08-13 09:19:14 -07:00
parent ff43de1ada
commit 9ca80667c3
3 changed files with 85 additions and 1 deletions

View file

@ -172,6 +172,41 @@ describe('fold-checkpoint bound — crash recovery folds (checkpoint, head], nev
expect(readCheckpoint(dir)).toBeNull()
}, 120000)
it('restore is an UNCLEAN event: the snapshots stamps do not survive — the reopen fold re-founds and re-stamps the restored state', async () => {
const dir = trackDir()
const brain = await openBrain(dir, { logAuthority: 'adopt' })
liveBrains.push(brain)
const idA = await brain.add({ data: 'survives the restore', type: NounType.Document, metadata: { n: 1 } })
await brain.flush()
const snapDir = join(trackDir(), 'snap')
const db = brain.now()
await (db as unknown as { persist(p: string): Promise<void> }).persist(snapDir)
await (db as unknown as { release(): Promise<void> }).release()
// Advance the live brain past the snapshot: a later write, a later flush,
// a later checkpoint stamp — none of which may survive the restore.
const idB = await brain.add({ data: 'must not survive', type: NounType.Document, metadata: { n: 2 } })
await brain.flush()
const stampBeforeRestore = readCheckpoint(dir)
expect(stampBeforeRestore).toBe(committedOf(brain))
// Unflushed traffic in flight at restore time — the quiesced swap discards
// it under the mutex instead of letting its flush timer race the swap
// (the ENOTEMPTY class).
await brain.add({ data: 'in-flight at restore', type: NounType.Document, metadata: { n: 3 } })
await brain.restore(snapDir, { confirm: true })
expect(await brain.get(idA), 'snapshot state restored').not.toBeNull()
expect(await brain.get(idB), 'post-snapshot state replaced').toBeNull()
// The stamp on disk is the REOPEN FOLD's fresh assertion about the
// restored (and now barrier-synced) bytes — at the restored watermark,
// strictly below the pre-restore stamp that must not survive.
const stampAfterRestore = readCheckpoint(dir)
expect(stampAfterRestore).toBe(committedOf(brain))
expect(stampAfterRestore!).toBeLessThan(stampBeforeRestore!)
}, 120000)
it('a delete rides the barrier: the tombstoned id is in the synced set and the stamp advances past it', async () => {
const dir = trackDir()
const brain = await openBrain(dir, { logAuthority: 'adopt' })