fix(index): the flicker window dies — atomic in-place vector update; lazy open honors every provider's not-ready report; the Path Registry twin table
DP6/DP8 of the Path Registry (BRAINY-PROD-LATENCY-TRIAD, the proven flicker mechanism): update paths staged RemoveFromVectorIndex then AddToVectorIndex as two separately-awaited transaction ops — between them a live row was in NEITHER index (dark to semantic recall, fine in metadata list). The native pair widened that window to seconds in production before their side's visibility-commit fix; the structural cure lands here: - hnswIndex.updateItem: absent → add; SAME vector → pure no-op (the production shape — a type-only update re-indexed an unchanged vector, remove+add did pure damage); changed vector → the node NEVER leaves the index: synchronous vector swap first (every query from that instant sees correct distances), then unlink/relink at the node's existing level via shared internals (linkNode/unlinkNodeEdges refactored out of add/remove; entry point and maxLevel provably unchanged). - ReplaceInVectorIndexOperation: ONE transaction leg; feature-detects provider updateItem (native seam flagged — their side ships updateItem, then the adjacent remove+add fallback is dead code). Both update staging sites swapped; delete sites untouched. - LAZY-OPEN GATE (fleet adoption find, SELF-ENGINE-PAIR-STANDARD): under disableAutoRebuild, ensureIndexesLoaded assessed ONLY the vector index — a not-ready native METADATA provider never blocked the completion latch and every find() silently returned [] on a populated store. All three providers now vote; any not-ready report falls through to the rebuild. - docs/path-registry.md: brainy's twin table for the 32 shared path IDs — service class, budgets, lifecycle, narration, and the cited pin per row; owed rows named (LC4 doors-open migration, MT4 yielding heals, LC7 downgrade contract) per the lifecycle-sprint choreography. Pins: update-item-atomic 9/9 (visibility-atomic swap, reverse-index parity vs fresh rebuild, entry-point invariants) · lazy-notready-honor 2/2. Gates: unit 1928/1928 (148 files) · integration 760 · conformance 27/27.
This commit is contained in:
parent
3236a01bef
commit
ebe06cdf33
7 changed files with 937 additions and 57 deletions
|
|
@ -151,6 +151,95 @@ export class RemoveFromVectorIndexOperation implements Operation {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace an item's vector in the vector index as ONE atomic transaction leg —
|
||||
* the row is never absent from vector search during an update.
|
||||
*
|
||||
* Backend-neutral: see {@link AddToVectorIndexOperation} — `index` may be the
|
||||
* JS HNSW fallback or a native acceleration provider; the emitted `name`
|
||||
* stamps the active backend.
|
||||
*
|
||||
* Why this op exists: update flows historically staged a
|
||||
* {@link RemoveFromVectorIndexOperation} followed by an
|
||||
* {@link AddToVectorIndexOperation} as two separately-awaited operations.
|
||||
* Between them the row was in NEITHER index — dark to semantic recall while
|
||||
* perfectly visible to metadata reads (a transient-invisibility window that
|
||||
* stretched to seconds in a production deployment). The structural cure is a
|
||||
* single leg that never removes without simultaneously re-inserting.
|
||||
*
|
||||
* Execution strategy (feature-detected, in preference order):
|
||||
* 1. Provider exposes `updateItem` → ONE in-place call. The provider swaps
|
||||
* the vector without the row ever leaving its index, and an element-wise
|
||||
* UNCHANGED vector (the type-only-update production shape) is a pure
|
||||
* no-op on its side.
|
||||
* 2. Provider without `updateItem` (a native provider that has not shipped
|
||||
* it yet) → `removeItem` + `addItem` executed ADJACENT within this single
|
||||
* op. Still strictly better than the historical pair: no other transaction
|
||||
* operation can interleave between the two calls. This is a temporary
|
||||
* seam — the native side of the pair is expected to ship its own
|
||||
* `updateItem` so path 1 applies everywhere; when it does, this fallback
|
||||
* becomes dead code that costs nothing.
|
||||
*
|
||||
* Rollback strategy (mirrors the execute branch that ran):
|
||||
* - `updateItem` path → `updateItem` back to `oldVector`.
|
||||
* - Fallback path → `removeItem` + `addItem` back to `oldVector`.
|
||||
*
|
||||
* Rollback semantics when the item did not exist at execute time: this op's
|
||||
* contract is that the caller read the entity and its CURRENT vector
|
||||
* (`oldVector`) before staging — update flows only stage it for existing
|
||||
* rows. If the item was somehow absent, execute() inserts it (`updateItem`
|
||||
* delegates to add; the fallback's remove is a no-op before its add), and
|
||||
* rollback restores `oldVector` rather than removing — the same posture as
|
||||
* {@link RemoveFromVectorIndexOperation}'s unconditional re-add: by
|
||||
* constructing the op with `oldVector` the caller DECLARED the before-state,
|
||||
* and rollback reconstructs that declared state instead of silently deciding
|
||||
* the row should vanish.
|
||||
*/
|
||||
export class ReplaceInVectorIndexOperation implements Operation {
|
||||
readonly name: string
|
||||
|
||||
constructor(
|
||||
private readonly index: VectorIndexProvider,
|
||||
private readonly id: string,
|
||||
private readonly oldVector: number[], // Required for rollback
|
||||
private readonly newVector: number[]
|
||||
) {
|
||||
this.name = `ReplaceInVectorIndex(${resolveVectorProviderId(index)})`
|
||||
}
|
||||
|
||||
async execute(): Promise<RollbackAction> {
|
||||
// Feature-detect the in-place capability — optional on the provider
|
||||
// contract, like `getItem`/`setPersistMode` (Brainy's JS HNSW index
|
||||
// ships it; a native provider may not have yet).
|
||||
const index = this.index as VectorIndexProvider & {
|
||||
updateItem?: (item: { id: string; vector: number[] }) => Promise<void>
|
||||
}
|
||||
|
||||
if (typeof index.updateItem === 'function') {
|
||||
// Atomic path: one in-place call, the row never leaves the index.
|
||||
await index.updateItem({ id: this.id, vector: this.newVector })
|
||||
|
||||
return async () => {
|
||||
// Restore the declared before-state in place (see class JSDoc for
|
||||
// the item-did-not-exist posture).
|
||||
await index.updateItem!({ id: this.id, vector: this.oldVector })
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback seam: remove+add ADJACENT within this single op — no other
|
||||
// transaction operation can interleave between them (see class JSDoc).
|
||||
await this.index.removeItem(this.id)
|
||||
await this.index.addItem({ id: this.id, vector: this.newVector })
|
||||
|
||||
return async () => {
|
||||
// updateItem-style restore via the same adjacent pair, back to the
|
||||
// declared before-state.
|
||||
await this.index.removeItem(this.id)
|
||||
await this.index.addItem({ id: this.id, vector: this.oldVector })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Add to metadata index with rollback support
|
||||
*
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ export {
|
|||
export {
|
||||
AddToVectorIndexOperation,
|
||||
RemoveFromVectorIndexOperation,
|
||||
ReplaceInVectorIndexOperation,
|
||||
AddToMetadataIndexOperation,
|
||||
RemoveFromMetadataIndexOperation,
|
||||
AddToGraphIndexOperation,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue