fix(vfs): the VFS root never persists a zero-norm vector
Some checks failed
CI / Node 22 (push) Successful in 12m23s
CI / Node 24 (push) Successful in 12m12s
CI / Integration + conformance (Node 22) (push) Failing after 14m47s
CI / Bun (latest) (push) Successful in 12m38s

A zero-norm vector is lawful inside brainy (cosine distance scores it at
maximum, never a false top hit) but a false attractor for a downstream
engine serving squared-euclidean distance, which cannot tell a real
all-zero vector apart from a legitimate origin point.

- The VFS root now persists with vector [] (the existing "unvectored"
  shape) instead of a real all-zero 384-dim placeholder, and is never
  routed into the deferred-embed pipeline.
- A one-time migration in the root-init path detects a pre-fix store's
  all-zero placeholder root (by norm, not length) and rewrites it to []
  through a new sanctioned Brainy method that keeps the canonical
  vectored-noun ledger honest and removes the row from the vector index.
- The vector-index write seam (AddToVectorIndexOperation,
  ReplaceInVectorIndexOperation, and the generation materializer's direct
  insert) now refuses any real all-zero vector before it reaches a
  provider, loudly naming the entity, while the canonical write still
  lands.
- add()'s dimension-pinning and HNSW-insert gates, and the add-params
  validator, now treat any empty vector as carrying no dimension
  information, closing a latent trap where an explicit `vector: []`
  would have pinned dimensions to 0.
This commit is contained in:
David Snelling 2026-08-27 09:28:44 -07:00
parent aad9e2eeb1
commit c6cc0de955
10 changed files with 516 additions and 64 deletions

View file

@ -65,6 +65,29 @@ export const cosineDistance: DistanceFunction = (a: Vector, b: Vector): number =
return 1 - similarity
}
/**
* True when `vector` is a REAL (non-empty) all-zero vector the "false
* attractor" shape this engine's own cosine distance treats safely (a
* zero-norm operand always scores the MAXIMUM distance, see
* {@link cosineDistance}) but a downstream engine serving squared-euclidean
* distance cannot distinguish from a legitimate origin point. THE LAW: a
* zero-norm vector is not a vector it never crosses an engine boundary
* (never handed to a vector-index provider as a searchable item).
*
* A length-0 vector is the UNRELATED "unvectored, not yet embedded" shape
* (the deferred-embed stub, a permanently-vectorless system row) and is
* deliberately NOT zero-norm here callers checking for "nothing to index"
* should test `vector.length === 0` separately; this only flags the
* dangerous non-empty all-zero case.
*/
export function isZeroNormVector(vector: readonly number[]): boolean {
if (vector.length === 0) return false
for (let i = 0; i < vector.length; i++) {
if (vector[i] !== 0) return false
}
return true
}
/**
* Calculates the Manhattan (L1) distance between two vectors.
* Lower values indicate higher similarity.

View file

@ -588,8 +588,14 @@ export function validateAddParams(params: AddParams): void {
)
}
// Validate vector dimensions if provided
if (params.vector) {
// Validate vector dimensions if provided. A length-0 vector is the
// "unvectored" shape — an explicit `vector: []` (e.g. the VFS root's
// permanently-vectorless creation, see
// VirtualFileSystem.doInitializeRoot()'s zero-norm fix) carries no
// dimension information, exactly like an absent vector or a deferred
// embed's internal stub, so it is exempt from the dimension check rather
// than refused as a "0-dimensional vector".
if (params.vector && params.vector.length > 0) {
const config = ValidationConfig.getInstance()
if (params.vector.length !== config.maxVectorDimensions) {
throw new Error(`vector must have exactly ${config.maxVectorDimensions} dimensions`)