feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank
Completes brainy's #35 side. When the at-gen vector gate serves a filtered semantic read, Brainy now ships the historically-correct candidate vectors so the native provider reranks against them with zero per-vector FFI crossing — the provider need not duplicate the per-generation retention Brainy already does. - New `AtGenerationVectors { ids: BigInt64Array; vectors: Float32Array; dim }` on the VectorIndexProvider.search options bag (row-major: vectors[i*dim..] == ids[i]; invariant vectors.length === ids.length*dim; ids = the candidate set; dim must match the index dim). The built-in JS index ignores it. - buildAtGenerationVectors resolves each universe id's vector AS OF the generation (the record before-image, or live getNoun when untouched since the pin — not readNounRaw, whose canonical path is empty under lazy-vector eviction), interns the id via the shared mapper, and packs row-major. Absent/vectorless/wrong-dim ids are dropped (not vector-rankable). Bounded by the filtered universe. Shape + the four clarifications (row-major, ids-are-the-candidate-set, dim-asserted, filtered-path-only) confirmed with cor in the handoff #35 thread. Inert until cor's native at-gen rerank advertises isGenerationVisible (honesty guard). Tested: the mock versioned provider now asserts it receives atGenerationVectors with the row-major invariant, correct dim, and ids resolving back to the at-gen universe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
6991bbe3d2
commit
c9e2169415
4 changed files with 132 additions and 9 deletions
|
|
@ -734,6 +734,34 @@ export function isGraphAccelerationProvider(
|
|||
return typeof c.traverse === 'function' && typeof c.graphCursorOpen === 'function'
|
||||
}
|
||||
|
||||
/**
|
||||
* Columnar at-`generation` candidate vectors for the 8.0 #35 FILTERED exact-rerank.
|
||||
* Brainy resolves the per-generation vector before-images for the at-gen filtered
|
||||
* universe (from its canonical generation records) and ships them flat, so a native
|
||||
* provider reranks against the historically-correct vectors with ZERO per-vector FFI
|
||||
* crossing — without the provider duplicating the per-gen retention Brainy already does.
|
||||
*
|
||||
* - **Row-major:** row `i` is `vectors[i*dim .. (i+1)*dim]` and belongs to `ids[i]`.
|
||||
* INVARIANT: `vectors.length === ids.length * dim` (the provider asserts + refuses on mismatch).
|
||||
* - **`ids` IS the candidate set:** entity ints (interned via the shared mapper) for the
|
||||
* at-gen metadata∩graph filtered universe. The provider reranks EXACTLY these
|
||||
* `(id, vector)` pairs and returns top-k by distance; any `allowedIds` passed alongside
|
||||
* is redundant on this path (the provider may AND it as belt-and-suspenders).
|
||||
* - **`dim`** must equal the provider's configured index dimension (asserted; mismatch → refuse).
|
||||
*
|
||||
* Supplied only when there is a BOUNDED filtered universe; the unfiltered-deep at-gen
|
||||
* case stays the provider's own retained-segment ANN ({@link VersionedIndexProvider.isGenerationVisible}
|
||||
* reports false there, so Brainy falls back to its materialization overlay).
|
||||
*/
|
||||
export interface AtGenerationVectors {
|
||||
/** Entity ints (shared-mapper interned), one per candidate; the candidate set. */
|
||||
ids: BigInt64Array
|
||||
/** Flat row-major vectors: `vectors[i*dim .. (i+1)*dim]` is `ids[i]`'s at-gen vector. */
|
||||
vectors: Float32Array
|
||||
/** Vector dimension; MUST equal the provider's configured index dim. */
|
||||
dim: number
|
||||
}
|
||||
|
||||
/**
|
||||
* The object returned by the `'vector'` provider factory — Brainy's vector
|
||||
* index contract. Implementations include Brainy's own JS HNSW index and any
|
||||
|
|
@ -782,6 +810,15 @@ export interface VectorIndexProvider {
|
|||
* `generation` is the same u64 counter handed to the graph index on writes.
|
||||
*/
|
||||
generation?: bigint
|
||||
/**
|
||||
* 8.0 #35 part-3: the at-`generation` candidate vectors for the FILTERED
|
||||
* exact-rerank — Brainy supplies the historically-correct vectors so the
|
||||
* provider need not retain them. Present only alongside `generation` on the
|
||||
* filtered at-gen path; the provider reranks `atGenerationVectors.ids` by
|
||||
* distance and returns top-k. See {@link AtGenerationVectors}. The built-in
|
||||
* JS index ignores it (it serves "now" only).
|
||||
*/
|
||||
atGenerationVectors?: AtGenerationVectors
|
||||
}
|
||||
): Promise<Array<[string, number]>>
|
||||
size(): number
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue