fix(index): the flicker window dies — atomic in-place vector update; lazy open honors every provider's not-ready report; the Path Registry twin table
Some checks failed
CI / Node 22 (push) Successful in 12m10s
CI / Node 24 (push) Has been cancelled
CI / Bun (latest) (push) Has been cancelled

DP6/DP8 of the Path Registry (BRAINY-PROD-LATENCY-TRIAD, the proven flicker
mechanism): update paths staged RemoveFromVectorIndex then AddToVectorIndex
as two separately-awaited transaction ops — between them a live row was in
NEITHER index (dark to semantic recall, fine in metadata list). The native
pair widened that window to seconds in production before their side's
visibility-commit fix; the structural cure lands here:

- hnswIndex.updateItem: absent → add; SAME vector → pure no-op (the
  production shape — a type-only update re-indexed an unchanged vector,
  remove+add did pure damage); changed vector → the node NEVER leaves the
  index: synchronous vector swap first (every query from that instant sees
  correct distances), then unlink/relink at the node's existing level via
  shared internals (linkNode/unlinkNodeEdges refactored out of add/remove;
  entry point and maxLevel provably unchanged).
- ReplaceInVectorIndexOperation: ONE transaction leg; feature-detects
  provider updateItem (native seam flagged — their side ships updateItem,
  then the adjacent remove+add fallback is dead code). Both update staging
  sites swapped; delete sites untouched.
- LAZY-OPEN GATE (fleet adoption find, SELF-ENGINE-PAIR-STANDARD): under
  disableAutoRebuild, ensureIndexesLoaded assessed ONLY the vector index —
  a not-ready native METADATA provider never blocked the completion latch
  and every find() silently returned [] on a populated store. All three
  providers now vote; any not-ready report falls through to the rebuild.
- docs/path-registry.md: brainy's twin table for the 32 shared path IDs —
  service class, budgets, lifecycle, narration, and the cited pin per row;
  owed rows named (LC4 doors-open migration, MT4 yielding heals, LC7
  downgrade contract) per the lifecycle-sprint choreography.

Pins: update-item-atomic 9/9 (visibility-atomic swap, reverse-index parity
vs fresh rebuild, entry-point invariants) · lazy-notready-honor 2/2.
Gates: unit 1928/1928 (148 files) · integration 760 · conformance 27/27.
This commit is contained in:
David Snelling 2026-08-05 16:11:23 -07:00
parent 3236a01bef
commit ebe06cdf33
7 changed files with 937 additions and 57 deletions

View file

@ -151,6 +151,95 @@ export class RemoveFromVectorIndexOperation implements Operation {
}
}
/**
* Replace an item's vector in the vector index as ONE atomic transaction leg
* the row is never absent from vector search during an update.
*
* Backend-neutral: see {@link AddToVectorIndexOperation} `index` may be the
* JS HNSW fallback or a native acceleration provider; the emitted `name`
* stamps the active backend.
*
* Why this op exists: update flows historically staged a
* {@link RemoveFromVectorIndexOperation} followed by an
* {@link AddToVectorIndexOperation} as two separately-awaited operations.
* Between them the row was in NEITHER index dark to semantic recall while
* perfectly visible to metadata reads (a transient-invisibility window that
* stretched to seconds in a production deployment). The structural cure is a
* single leg that never removes without simultaneously re-inserting.
*
* Execution strategy (feature-detected, in preference order):
* 1. Provider exposes `updateItem` ONE in-place call. The provider swaps
* the vector without the row ever leaving its index, and an element-wise
* UNCHANGED vector (the type-only-update production shape) is a pure
* no-op on its side.
* 2. Provider without `updateItem` (a native provider that has not shipped
* it yet) `removeItem` + `addItem` executed ADJACENT within this single
* op. Still strictly better than the historical pair: no other transaction
* operation can interleave between the two calls. This is a temporary
* seam the native side of the pair is expected to ship its own
* `updateItem` so path 1 applies everywhere; when it does, this fallback
* becomes dead code that costs nothing.
*
* Rollback strategy (mirrors the execute branch that ran):
* - `updateItem` path `updateItem` back to `oldVector`.
* - Fallback path `removeItem` + `addItem` back to `oldVector`.
*
* Rollback semantics when the item did not exist at execute time: this op's
* contract is that the caller read the entity and its CURRENT vector
* (`oldVector`) before staging update flows only stage it for existing
* rows. If the item was somehow absent, execute() inserts it (`updateItem`
* delegates to add; the fallback's remove is a no-op before its add), and
* rollback restores `oldVector` rather than removing the same posture as
* {@link RemoveFromVectorIndexOperation}'s unconditional re-add: by
* constructing the op with `oldVector` the caller DECLARED the before-state,
* and rollback reconstructs that declared state instead of silently deciding
* the row should vanish.
*/
export class ReplaceInVectorIndexOperation implements Operation {
readonly name: string
constructor(
private readonly index: VectorIndexProvider,
private readonly id: string,
private readonly oldVector: number[], // Required for rollback
private readonly newVector: number[]
) {
this.name = `ReplaceInVectorIndex(${resolveVectorProviderId(index)})`
}
async execute(): Promise<RollbackAction> {
// Feature-detect the in-place capability — optional on the provider
// contract, like `getItem`/`setPersistMode` (Brainy's JS HNSW index
// ships it; a native provider may not have yet).
const index = this.index as VectorIndexProvider & {
updateItem?: (item: { id: string; vector: number[] }) => Promise<void>
}
if (typeof index.updateItem === 'function') {
// Atomic path: one in-place call, the row never leaves the index.
await index.updateItem({ id: this.id, vector: this.newVector })
return async () => {
// Restore the declared before-state in place (see class JSDoc for
// the item-did-not-exist posture).
await index.updateItem!({ id: this.id, vector: this.oldVector })
}
}
// Fallback seam: remove+add ADJACENT within this single op — no other
// transaction operation can interleave between them (see class JSDoc).
await this.index.removeItem(this.id)
await this.index.addItem({ id: this.id, vector: this.newVector })
return async () => {
// updateItem-style restore via the same adjacent pair, back to the
// declared before-state.
await this.index.removeItem(this.id)
await this.index.addItem({ id: this.id, vector: this.oldVector })
}
}
}
/**
* Add to metadata index with rollback support
*