feat: canonical enumeration mode for export — storage-walked, canon-complete, with an index-drift report

This commit is contained in:
David Snelling 2026-07-27 11:08:19 -07:00
parent 999d0ebbcf
commit 4d196af41b
6 changed files with 543 additions and 20 deletions

View file

@ -31,6 +31,37 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
---
## Unreleased (canonical enumeration mode for export — storage-walked, canon-complete)
From a fleet data-migration program's requirement for whole-brain exports that are
provably canon-complete: `export()`'s default enumeration for a whole-brain/predicate
selector is a generation-correct paginated `find()` walk — a projection query riding
the metadata index as an acceleration structure. Production has documented both of the
index's failure classes: a lost/stale posting can silently OMIT a canonical record from
an export, and a stale posting can silently INCLUDE a phantom row. Neither is visible
to the caller today.
- **New: `export(selector, { enumeration: 'canonical' })`** (default remains `'index'`
unchanged behavior on this release). Canonical mode walks every live noun/verb
directly off the storage adapter's canonical shard layout (`storage.getNouns()` /
`getVerbs()` — the same primitive `repairIndex()`'s recount and every index-heal
walk use) instead of the metadata/graph indexes, then applies the selector as a
plain predicate over the walked records. This guarantees canon-completeness — index
corruption cannot hide a live record from the export — at the cost of an O(N) walk
regardless of selector selectivity. Relations are also walked canonically in this
mode, for every selector, not just the whole-brain case. Requires the LIVE current
generation: called on a historical `asOf()` view or a speculative `with()` overlay it
throws `CanonicalEnumerationUnavailableError` rather than silently mixing generations
or missing an overlay's own entities — `enumeration: 'index'` (the default) is
unaffected and still composes with `asOf()`/`with()` as before.
- **New: `export(selector, { enumeration: 'canonical', reportIndexDrift: true })`**
also runs the index-based enumeration and diffs it against canonical ground truth,
attaching `PortableGraph.drift: { canonicalOnly: string[], indexOnly: string[] }`
(canon-present ids the index missed; index-visible ids canon-absent — phantoms).
Migration-audit evidence, not a repair: nonzero drift is reported loudly
(`console.warn` with the counts) and nothing is auto-healed — run `brain.repairIndex()`
to reconcile the metadata index once drift is confirmed.
## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers)
From a production incident: a native-provider op ground 38-40s inside a transaction,