feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank
Completes brainy's #35 side. When the at-gen vector gate serves a filtered semantic read, Brainy now ships the historically-correct candidate vectors so the native provider reranks against them with zero per-vector FFI crossing — the provider need not duplicate the per-generation retention Brainy already does. - New `AtGenerationVectors { ids: BigInt64Array; vectors: Float32Array; dim }` on the VectorIndexProvider.search options bag (row-major: vectors[i*dim..] == ids[i]; invariant vectors.length === ids.length*dim; ids = the candidate set; dim must match the index dim). The built-in JS index ignores it. - buildAtGenerationVectors resolves each universe id's vector AS OF the generation (the record before-image, or live getNoun when untouched since the pin — not readNounRaw, whose canonical path is empty under lazy-vector eviction), interns the id via the shared mapper, and packs row-major. Absent/vectorless/wrong-dim ids are dropped (not vector-rankable). Bounded by the filtered universe. Shape + the four clarifications (row-major, ids-are-the-candidate-set, dim-asserted, filtered-path-only) confirmed with cor in the handoff #35 thread. Inert until cor's native at-gen rerank advertises isGenerationVisible (honesty guard). Tested: the mock versioned provider now asserts it receives atGenerationVectors with the row-major invariant, correct dim, and ids resolving back to the at-gen universe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
6991bbe3d2
commit
c9e2169415
4 changed files with 132 additions and 9 deletions
|
|
@ -14,7 +14,7 @@ import { euclideanDistance, calculateDistancesBatch } from '../utils/index.js'
|
|||
import type { BaseStorage } from '../storage/baseStorage.js'
|
||||
import { getGlobalCache, UnifiedCache } from '../utils/unifiedCache.js'
|
||||
import { prodLog } from '../utils/logger.js'
|
||||
import type { VectorIndexProvider, OpaqueIdSet } from '../plugin.js'
|
||||
import type { VectorIndexProvider, OpaqueIdSet, AtGenerationVectors } from '../plugin.js'
|
||||
import { ConnectionsCodec, compressedConnectionsKey } from './connectionsCodec.js'
|
||||
|
||||
// Default HNSW parameters
|
||||
|
|
@ -673,6 +673,9 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
// historical read here (its defer-gate checks `isGenerationVisible` first),
|
||||
// so an ignored `generation` cannot silently return now-vectors-as-at-gen.
|
||||
generation?: bigint
|
||||
// 8.0 #35 part-3: at-gen candidate vectors for the native exact-rerank. The JS
|
||||
// index serves "now" only and never receives this (gated off upstream); ignored.
|
||||
atGenerationVectors?: AtGenerationVectors
|
||||
}
|
||||
): Promise<Array<[string, number]>> {
|
||||
if (this.nouns.size === 0) {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue