soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: full-removal canonical deletes + family-scoped migration gate

Browse source 
Two production-reported spine fixes:

- Canonical delete is a FULL removal: remove()/removeMany() deleted the
  metadata leg but never the canonical vectors.json leg or the <id>/
  directory, leaving ghost rows that inflated enumerated counts forever,
  read as damage scars, and caused duplicate readdir entries for
  re-created VFS paths (the stale-unpost path). The delete operation now
  routes through storage.deleteNoun — both legs + the entity container —
  with a full two-leg before-image rollback; deleteVerb symmetric; the
  blind 'no metadata file' catch that masked real faults is gone. The
  generation log still holds the delete's before-image, so asOf() history
  is unchanged. repairIndex() gains a conservative, loud orphan-prune
  sweep (containers with no metadata content leg) + count recompute for
  stores damaged by earlier versions.

- Family-scoped migration gate: every read blocked on the whole-brain
  migration lock even when the migrating index was irrelevant. The gate
  now scopes to the index families a read actually consults — canonical
  reads never wait; find() waits only on its query shape's families;
  graph traversals wait only on the graph family. Writes keep the
  conservative whole-brain wait. A read that needs the migrating family
  still blocks (bounded) with the retryable MigrationInProgressError.
This commit is contained in:
David Snelling 2026-07-14 10:11:53 -07:00
parent 1d26988963
commit 366f9a91f5
8 changed files with 657 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

27
tests/unit/migration-lock.test.ts
View file

@ -2,11 +2,16 @@
* Migration LOCK (#18) — coordinated, automatic 7.x → 8.0 auto-upgrade.
*
* Exercises the real block-and-queue behavior of `awaitMigrationLock` at the
* `ensureInitialized` choke point: while ANY index provider reports
* `isMigrating() === true`, data-plane reads and writes WAIT (no operation
* touches a half-built index); observability (`getIndexStatus`, `health`) and
* the lock-clearing `stampBrainFormat` stay exempt; and a lock that outlives the
* configured window surfaces a retryable `MigrationInProgressError`.
* `ensureInitialized` choke point. The gate is FAMILY-SCOPED: a write (which
* touches every index) WAITS while ANY provider reports `isMigrating() === true`,
* and a read WAITS only when the migrating provider is one of the index families
* that read consults (so a read served from canonical storage or a healthy family
* is never blocked by an unrelated family's migration — see
* `tests/unit/brainy/migration-gate-family-scoped.test.ts`). Observability
* (`getIndexStatus`, `health`) and the lock-clearing `stampBrainFormat` stay
* exempt; a lock that outlives the configured window surfaces a retryable
* `MigrationInProgressError`. Here the graph provider holds the lock, so the
* read cases use a graph traversal (`related`) — the family that actually waits.
*
* A native (cortex) provider owns the real migration; here we simulate the lock
* by feature-detect-injecting `isMigrating()` onto a live provider, exactly as
@ -84,11 +89,12 @@ describe('Migration LOCK (#18) — coordinated 7.x→8.0 auto-upgrade', () => {
expect(id).toBeTruthy()
})
it('blocks a read while migrating, then releases when the flag clears (poll path)', async () => {
it('blocks a graph read while the graph provider migrates, then releases when the flag clears (poll path)', async () => {
const anchor = await brain.add({ data: 'anchor', type: NounType.Concept })
let migrating = true
brain.graphIndex.isMigrating = () => migrating
const p = brain.find({ type: NounType.Concept }) // a read → gated
const p = brain.related({ from: anchor }) // a GRAPH read → gated on the graph migration
let resolved = false
p.then(() => {
resolved = true
@ -109,9 +115,12 @@ describe('Migration LOCK (#18) — coordinated 7.x→8.0 auto-upgrade', () => {
migrationWaitTimeoutMs: 120
})
await shortBrain.init()
setMigrating(shortBrain, true) // never clears
const anchor = await shortBrain.add({ data: 'anchor', type: NounType.Concept })
setMigrating(shortBrain, true) // graph lock never clears
await expect(shortBrain.find({ type: NounType.Concept })).rejects.toBeInstanceOf(
// A graph read needs the migrating family → it waits out the window and
// surfaces the retryable error rather than serving a half-built traversal.
await expect(shortBrain.related({ from: anchor })).rejects.toBeInstanceOf(
MigrationInProgressError
)
try {