soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: transact durability barrier — committed transactions are durable on return

Browse source 
A committed transact() reported success while its canonical entity writes were
still only in the OS page cache (tmp+rename, not fsync'd), even though the
generation counter and manifest were fsync'd. A hard kill in that window could
leave the durable counter ahead of the persisted entity bytes — phantom
progress for any generation-based consumer resuming from the counter. Reported
from a downstream migration's crash-lifecycle forensics.

Add an optional transaction durability barrier to GenerationStorage
(beginWriteBarrier / flushWriteBarrier). FileSystemStorage implements it:
writeObjectToPath records each successful canonical write and
deleteObjectFromPath records each delete's parent dir, between begin and flush;
flush fsyncs every recorded write (contents + rename dir entries, via
syncRawObjects) and the parent dir of every delete. commitTransaction opens the
barrier before running the planned operations and flushes it after, BEFORE
persisting the counter and manifest — so the batch's entire canonical footprint
is durable before the generation stamp advances. The barrier is optional: the
generation store calls it through optional chaining, so in-memory and
durable-per-call (cloud object-PUT) adapters treat it as a no-op.

The single-op group-commit path is unchanged (deferred durability is the
Model-B design that avoids a 3-5x per-write fsync regression), but its
durability contract is now documented explicitly on commitSingleOp: transact =
durable on return; single-op = durable at the next flush()/close(), with the
counter buffered alongside the data so a crash loses both together (never a
torn counter-ahead-of-state store).

Regression (tests/integration/transact-durability-barrier.test.ts): the entity
writes fsync in an earlier syncRawObjects batch than the manifest for single-op
and multi-op (add+relate) transactions; a precommit-rejected batch opens no
barrier and advances nothing; MemoryStorage exposes no barrier (optional-chain
no-op).
This commit is contained in:
David Snelling 2026-07-12 08:54:14 -07:00
parent a9fc1f3f9b
commit 3b8fa51161
4 changed files with 270 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

18
src/db/types.ts
View file

@ -391,6 +391,24 @@ export interface GenerationStorage {
/** Durability barrier: fsync the given object paths (no-op in memory). */
syncRawObjects(paths: string[]): Promise<void>
/**
* OPTIONAL transaction durability barrier. `commitTransaction` calls
* {@link beginWriteBarrier} immediately before running the planned operations
* and {@link flushWriteBarrier} immediately after — BEFORE the generation
* counter and manifest are advanced. An adapter whose canonical writes are
* not synchronously durable (the filesystem adapter's tmp+rename lands in the
* page cache) MUST implement these so a transaction reported "committed" is
* durable before its generation stamp: otherwise a hard kill can leave the
* fsync'd counter ahead of the still-buffered entity bytes (phantom progress
* for any generation-based consumer). Adapters whose writes are already
* durable per-call (cloud object PUT) may leave these undefined — the store
* treats them as no-ops. `beginWriteBarrier` also resets any tracking left by
* an aborted prior transaction.
*/
beginWriteBarrier?(): void
/** @see beginWriteBarrier — fsync every canonical write since begin. */
flushWriteBarrier?(): Promise<void>
/** Read an entity's raw stored metadata+vector objects. */
readNounRaw(id: string): Promise<{ metadata: any | null; vector: any | null }>
/** Restore an entity's raw stored objects (`null` part ⇒ delete that file). */