feat(recovery): the catchup verdict is consumed; verb rows go live; the metadata rebuild goes online

Three cures on the JS metadata index, one seam:

- THE CATCHUP WIRING. The index computed its three-way watermark verdict at
  open and nothing consumed it — after a crash + adopt reopen, find() served
  the pre-crash index while canonical reads and counts recovered (caught by
  the lifecycle lane's first run). The open path now consumes the verdict:
  'adopt' is a no-op, 'catchup' folds the fact window (stamped, committed]
  through the index legs — nouns and verbs, remove-then-add, one mechanism
  for add and update — and 'rescan' runs the explicit rebuild, each narrated.
  The lane's Ch4–6 release-blocking marker comes off: the contract holds.
  Bonus root-cause: close() never stamped the projection watermarks (only
  flush() did), so any close without a prior flush verdicted a needless
  'rescan' on reopen — both doors now stamp.

- THE LIVE VERB PATH. Verb rows entered the metadata index only via rebuild
  walks, so every rebuilt store minted phantom/stale verb postings from its
  first live relate(). relate()/unrelate()/updateRelation() and remove()'s
  cascade now post/retract the verb's row in the same commit as the graph
  leg — transact() planners mirror identically — using the exact record
  shape the rebuild walk uses, so live and rebuilt populations agree.

- THE ONLINE REBUILD. rebuild() was clear-then-walk — every metadata read
  empty for the duration. rebuildMetadataIndexOnline builds a fresh manager
  beside the serving one (shared identity, in-memory build, dual-write via
  a shadow seam with zero call-site changes), atomically swaps the
  reference, and persists exactly once post-swap. A find() polled ~200x
  during a 2k-noun rebuild never dropped below its baseline.
  repairIndex({ rebuild: ['metadata'] }) uses it automatically.
This commit is contained in:
David Snelling 2026-08-25 10:01:56 -07:00
parent f8f64780b1
commit 18f172e098
6 changed files with 1275 additions and 117 deletions

View file

@ -286,32 +286,7 @@ describe.sequential('lifecycle — the working store', () => {
300000
)
/**
* Ch4 CRASH is a LIVE ENGINE FINDING, not a defect in this lane (see
* README.md and the project report this lane's build produced): after a
* crash (writes acked at commit but never flushed, the process abandoned
* exactly as `abandonAsCrashed` models, then reopened), canonical storage
* (`get()`), the vector index, and `getNounCount()`/`getCanonicalCounts()`
* all correctly recover every acked write but the METADATA INDEX behind
* `find({ where })` recovers NONE of the crash-window's acked writes
* (neither new adds nor metadata updates to pre-existing entities), even
* though `getIndexStatus()` reports `projections.metadata.synchronous:
* true`. `repairIndex()` cannot close the gap either: its own report names
* `provider:metadata` as `checked: false, skipped: "no
* validateInvariants/rebuild contract"`. The assertion below states the
* TRUE contract (find() must agree with get()) and is expected to fail
* against the current engine it must never be loosened to paper over
* this. Ch5/Ch6 are written in full below it and will start running the
* moment this gap is closed; they are not dead code, they are blocked code.
*/
// RELEASE-BLOCKING FINDING (the kill-matrix convention: assert the CONTRACT,
// mark `.fails`, never weaken): after a crash + adopt reopen, the JS metadata
// index computes its watermark verdict but nothing consumes 'catchup'
// (metadataIndex.ts loadWatermarkVerdict) — find() serves the pre-crash
// index while get()/counts recover. The catchup wiring is the cure; when it
// lands this `.fails` marker MUST be removed (vitest will force it: a
// passing `.fails` test is itself a failure).
it.fails(
it(
'Ch4 CRASH -> Ch5 REPAIR -> Ch6 SECOND LIFE: continues the Ch3 store',
async () => {
try {