fix(vfs): the VFS root never persists a zero-norm vector
A zero-norm vector is lawful inside brainy (cosine distance scores it at maximum, never a false top hit) but a false attractor for a downstream engine serving squared-euclidean distance, which cannot tell a real all-zero vector apart from a legitimate origin point. - The VFS root now persists with vector [] (the existing "unvectored" shape) instead of a real all-zero 384-dim placeholder, and is never routed into the deferred-embed pipeline. - A one-time migration in the root-init path detects a pre-fix store's all-zero placeholder root (by norm, not length) and rewrites it to [] through a new sanctioned Brainy method that keeps the canonical vectored-noun ledger honest and removes the row from the vector index. - The vector-index write seam (AddToVectorIndexOperation, ReplaceInVectorIndexOperation, and the generation materializer's direct insert) now refuses any real all-zero vector before it reaches a provider, loudly naming the entity, while the canonical write still lands. - add()'s dimension-pinning and HNSW-insert gates, and the add-params validator, now treat any empty vector as carrying no dimension information, closing a latent trap where an explicit `vector: []` would have pinned dimensions to 0.
This commit is contained in:
parent
aad9e2eeb1
commit
c6cc0de955
10 changed files with 516 additions and 64 deletions
|
|
@ -176,21 +176,22 @@ describe('vector-leg open-build (two-engine gate, last red)', () => {
|
|||
})
|
||||
|
||||
it('the inverse: only deferred (never-landed) user nouns — the ledger is never inflated by them, and search over them honestly returns []', async () => {
|
||||
// ARCHITECTURAL NOTE (found while building this pin): every brainy store
|
||||
// carries ONE permanent, always-vectored noun beyond user data — the VFS
|
||||
// root (`entities/nouns/.../00000000-0000-0000-0000-000000000000`,
|
||||
// src/vfs/VirtualFileSystem.ts). It is inserted with an explicit all-zero
|
||||
// (but non-empty, length-384) vector on EVERY store's first open — never
|
||||
// deferred (a deliberate WASM-cold-compile-avoidance fix, see that
|
||||
// file's comment) — and VFS init unconditionally re-creates it if
|
||||
// missing, before the rebuild gate ever runs. A literal "0 vectored
|
||||
// nouns" store is therefore unreachable through the public API; a
|
||||
// brand-new store's `vectors.all` floor is 1, not 0. This pin verifies
|
||||
// the law the task names in the ACHIEVABLE form: nouns whose embed is
|
||||
// still deferred/unlanded contribute NOTHING to the vectored-noun ledger
|
||||
// — the coverage-gap comparison sees exactly the root (1), never
|
||||
// root+deferred — and semantic search over deferred-only user content
|
||||
// honestly returns `[]` (no error, no false "coverage restored" claim).
|
||||
// ARCHITECTURAL NOTE (updated by the zero-norm root cure): every brainy
|
||||
// store carries ONE permanent VFS root noun beyond user data
|
||||
// (`entities/nouns/.../00000000-0000-0000-0000-000000000000`,
|
||||
// src/vfs/VirtualFileSystem.ts), created (or, on a pre-fix store,
|
||||
// migrated) on every open — but it is deliberately UNVECTORED (vector
|
||||
// `[]`), never a real all-zero placeholder: a zero-norm vector is not a
|
||||
// vector and never crosses an engine boundary (see that file's
|
||||
// doInitializeRoot() comment). It therefore contributes NOTHING to the
|
||||
// vectored-noun ledger — a brand-new store's `vectors.all` floor is 0,
|
||||
// not 1. This pin verifies the law the task names in the ACHIEVABLE
|
||||
// form: nouns whose embed is still deferred/unlanded contribute NOTHING
|
||||
// to the vectored-noun ledger either — the coverage-gap comparison sees
|
||||
// exactly the baseline (the root, contributing 0), never
|
||||
// baseline+deferred — and semantic search over deferred-only user
|
||||
// content honestly returns `[]` (no error, no false "coverage restored"
|
||||
// claim).
|
||||
const dir = mkTmp()
|
||||
|
||||
const build: any = new Brainy({
|
||||
|
|
@ -202,6 +203,8 @@ describe('vector-leg open-build (two-engine gate, last red)', () => {
|
|||
})
|
||||
await build.init()
|
||||
const rootOnlyLedger = await build.storage.getCanonicalCounts()
|
||||
// THE NEW LAW: the root is unvectored — a brand-new store's floor is 0.
|
||||
expect(rootOnlyLedger.vectors.all).toBe(0)
|
||||
// Block the embedder permanently so every add below stays deferred and
|
||||
// unlanded for the rest of this test (a fast deterministic embedder
|
||||
// could otherwise land it before we ever observe the "still 0 extra"
|
||||
|
|
|
|||
Reference in a new issue