fix: full-removal canonical deletes + family-scoped migration gate

Two production-reported spine fixes:

- Canonical delete is a FULL removal: remove()/removeMany() deleted the
  metadata leg but never the canonical vectors.json leg or the <id>/
  directory, leaving ghost rows that inflated enumerated counts forever,
  read as damage scars, and caused duplicate readdir entries for
  re-created VFS paths (the stale-unpost path). The delete operation now
  routes through storage.deleteNoun — both legs + the entity container —
  with a full two-leg before-image rollback; deleteVerb symmetric; the
  blind 'no metadata file' catch that masked real faults is gone. The
  generation log still holds the delete's before-image, so asOf() history
  is unchanged. repairIndex() gains a conservative, loud orphan-prune
  sweep (containers with no metadata content leg) + count recompute for
  stores damaged by earlier versions.

- Family-scoped migration gate: every read blocked on the whole-brain
  migration lock even when the migrating index was irrelevant. The gate
  now scopes to the index families a read actually consults — canonical
  reads never wait; find() waits only on its query shape's families;
  graph traversals wait only on the graph family. Writes keep the
  conservative whole-brain wait. A read that needs the migrating family
  still blocks (bounded) with the retryable MigrationInProgressError.
This commit is contained in:
David Snelling 2026-07-14 10:11:53 -07:00
parent 1d26988963
commit 366f9a91f5
8 changed files with 657 additions and 66 deletions

View file

@ -253,7 +253,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
/** One-shot guard so the graph fast-path cold-load probe runs once per adapter. */
private _graphFastPathProbed = false
/**
* Registered-blob contract (ADR-004 §7): declared derived-index families, keyed
* Registered-blob contract: declared derived-index families, keyed
* by name. Members are undeletable through the blob delete seams. Loaded lazily
* from `_system/derived-artifacts.json` and re-persisted on every change.
*/
@ -842,6 +842,24 @@ export abstract class BaseStorage extends BaseStorageAdapter {
return this.deleteObjectFromPath(path)
}
/**
* @description Remove the container that held a canonical entity's leg files,
* called after both legs are deleted so a delete leaves NOTHING behind no
* orphan "scar" directory. `objectLegPath` is any one of the entity's leg
* paths (e.g. its `vectors.json`); the container is that path's parent.
*
* Default: a no-op. Key/prefix-addressed stores (in-memory, cloud object
* stores) have no empty-container concept deleting the leg keys already
* removes the entity entirely. The filesystem adapter overrides this to
* `rmdir` the emptied entity directory.
*
* @param objectLegPath - Storage-root-relative path of one of the entity's legs.
* @protected
*/
protected async removeCanonicalContainer(objectLegPath: string): Promise<void> {
void objectLegPath
}
/**
* @description List canonical objects under a storage-root-relative prefix.
*
@ -1062,7 +1080,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
// ==========================================================================
// Registered-blob contract (ADR-004 §7) — declared derived-index families are
// Registered-blob contract — declared derived-index families are
// undeletable through the blob delete seams. Shared here so every adapter that
// extends BaseStorage inherits the same enforcement; the concrete
// deleteBinaryBlob / removeRawPrefix call assertBlobKeyDeletable /
@ -1116,7 +1134,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
/**
* @description A blob key that is a transient write-scratch file (a `*.tmp.*`
* temp, a `*.rebuild-tmp`, a `*.rotate-tmp`). Its OWNER renames/removes it as
* part of an atomic write; it is never a protected family member (ADR-004 §7).
* part of an atomic write; it is never a protected family member.
*/
private isTransientBlobKey(key: string): boolean {
return /\.rebuild-tmp$|\.rotate-tmp$|\.tmp(\.|$)/.test(key)
@ -1185,7 +1203,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
/**
* @description Verify every declared family has all its members present on
* disk (ADR-004 §7 missing-on-open catch for an EXTERNAL deleter that bypasses
* disk (missing-on-open catch for an EXTERNAL deleter that bypasses
* the in-process refusal). Returns the incomplete families (name + missing
* members) the caller decides how to heal (rebuild from canonical). Loud by
* construction: a missing load-bearing blob is named, never silently tolerated.
@ -1601,16 +1619,22 @@ export abstract class BaseStorage extends BaseStorageAdapter {
public async deleteNoun(id: string): Promise<void> {
await this.ensureInitialized()
// Delete both the vector file and metadata file (2-file system)
await this.deleteNoun_internal(id)
// FULL removal (live-HEAD hygiene): remove BOTH canonical legs AND the
// entity's container, so a delete leaves nothing behind — no orphan/scar
// directory to inflate the enumerated count or ghost resolveLocator. The
// generation store holds the immutable before-image, so asOf() still
// reconstructs the deleted entity until retention expires.
await this.deleteNoun_internal(id) // vectors.json leg
// Delete metadata file (if it exists)
try {
await this.deleteNounMetadata(id)
} catch (error) {
// Ignore if metadata file doesn't exist
prodLog.debug(`No metadata file to delete for noun ${id}`)
}
// Metadata leg + count decrement. A genuine "already absent" is a no-op
// downstream (readCanonicalObject / deleteObjectFromPath both treat ENOENT as
// success); a REAL fault must surface loudly (loud errors, never quiet
// losses) and must never silently skip the count decrement — so this is NO
// LONGER wrapped in a blind catch that masked faults as "file didn't exist".
await this.deleteNounMetadata(id)
// Remove the now-empty entity container (a no-op for key/prefix stores).
await this.removeCanonicalContainer(getNounVectorPath(id))
}
/**
@ -2915,16 +2939,12 @@ export abstract class BaseStorage extends BaseStorageAdapter {
public async deleteVerb(id: string): Promise<void> {
await this.ensureInitialized()
// Delete both the vector file and metadata file (2-file system)
await this.deleteVerb_internal(id)
// Delete metadata file (if it exists)
try {
await this.deleteVerbMetadata(id)
} catch (error) {
// Ignore if metadata file doesn't exist
prodLog.debug(`No metadata file to delete for verb ${id}`)
}
// FULL removal (see deleteNoun): both canonical legs + the verb container.
// The blind catch that masked real faults as "no metadata file" is gone — a
// genuine absence is already a no-op downstream; a real fault surfaces loudly.
await this.deleteVerb_internal(id) // vectors.json leg
await this.deleteVerbMetadata(id) // metadata leg + count decrement
await this.removeCanonicalContainer(getVerbVectorPath(id))
}
/**
* Get graph index (lazy initialization with concurrent access protection)