feat(8.0): asOf at-gen vector defer — provider-served historical semantic search (#35)

A filtered semantic read at a historical generation (db.asOf(g).find({ query, where }))
no longer unconditionally rebuilds an ephemeral JS-HNSW over every at-g vector
(O(n@G), the OOM ceiling the 2026-06-24 scaling audit flagged). When the vector
index is a VersionedIndexProvider that advertises isGenerationVisible(g), Brainy:
  1. resolves the at-g metadata∩graph universe from the record-overlay path (no
     materialization — it's the metadata-only historical find),
  2. routes the vector leg to the provider with { allowedIds, generation }, and
  3. composes the at-g entities ranked by the provider's at-g vector distance.
Without a versioned provider (the JS index, or a native one that refuses the
generation) it falls through to the existing materialization — unchanged.

- Seam (A): VectorIndexProvider.search gains an optional `generation?: bigint`
  ("omitted = now"), the vector mirror of the graph provider's trailing-gen + the
  #46 allowedIds field. JsHnswVectorIndex accepts and ignores it (no per-gen
  segments → refuse/fall-back posture; Brainy never routes a historical read there).
- Gate: Db.find tries the native at-gen vector path before materialize(); host
  exposes canServeVectorAtGeneration + vectorSearchAtGeneration.
- generation is Brainy's u64 commit counter — the same value handed to the graph
  index on writes (graphWriteGeneration), so it maps 1:1 to the native side.

Decided lockstep with the cor team (handoff #35 thread: (A) + vector-only). The
gen-g candidate-vector supply for cor's exact-rerank (part-3) is a separate,
non-blocking seam still being confirmed; the honesty guard keeps this path inactive
(falls through to materialize) until cor's native at-gen rerank is live.

Tested: provider routing + at-gen universe correctness (excludes future-born,
applies the filter) + page window via a mock versioned provider; seam-ignore on the
JS index; 38 db-mvcc/db-temporal historical-read tests green (materialize fallback).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-23 15:52:09 -07:00
parent 450084b6ce
commit 1c363e8c4b
7 changed files with 292 additions and 0 deletions

View file

@ -119,6 +119,14 @@ describe('#46 JS HNSW allowedIds pushdown (recall-preserving)', () => {
// Unrestricted behavior: the opaque buffer is not interpreted as a filter.
expect(ids.every((id) => disallowed.includes(id))).toBe(true)
})
it('ignores the #35 `generation` field — the JS index only ever holds "now"', async () => {
// The JS index has no retained per-generation segments, so it accepts and
// ignores `generation` (the refuse/fall-back posture) rather than throwing.
const withGen = await index.search(query, 3, undefined, { generation: 42n })
const withoutGen = await index.search(query, 3)
expect(withGen.map(([id]) => id)).toEqual(withoutGen.map(([id]) => id))
})
})
describe('#46 find() forwards the opaque universe to the vector beam walk', () => {