A filtered semantic read at a historical generation (db.asOf(g).find({ query, where }))
no longer unconditionally rebuilds an ephemeral JS-HNSW over every at-g vector
(O(n@G), the OOM ceiling the 2026-06-24 scaling audit flagged). When the vector
index is a VersionedIndexProvider that advertises isGenerationVisible(g), Brainy:
1. resolves the at-g metadata∩graph universe from the record-overlay path (no
materialization — it's the metadata-only historical find),
2. routes the vector leg to the provider with { allowedIds, generation }, and
3. composes the at-g entities ranked by the provider's at-g vector distance.
Without a versioned provider (the JS index, or a native one that refuses the
generation) it falls through to the existing materialization — unchanged.
- Seam (A): VectorIndexProvider.search gains an optional `generation?: bigint`
("omitted = now"), the vector mirror of the graph provider's trailing-gen + the
#46 allowedIds field. JsHnswVectorIndex accepts and ignores it (no per-gen
segments → refuse/fall-back posture; Brainy never routes a historical read there).
- Gate: Db.find tries the native at-gen vector path before materialize(); host
exposes canServeVectorAtGeneration + vectorSearchAtGeneration.
- generation is Brainy's u64 commit counter — the same value handed to the graph
index on writes (graphWriteGeneration), so it maps 1:1 to the native side.
Decided lockstep with the cor team (handoff #35 thread: (A) + vector-only). The
gen-g candidate-vector supply for cor's exact-rerank (part-3) is a separate,
non-blocking seam still being confirmed; the honesty guard keeps this path inactive
(falls through to materialize) until cor's native at-gen rerank is live.
Tested: provider routing + at-gen universe correctness (excludes future-born,
applies the filter) + page window via a mock versioned provider; seam-ignore on the
JS index; 38 db-mvcc/db-temporal historical-read tests green (materialize fallback).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Filtered semantic search now keeps its recall. A find({ query, where }) that
pairs vector search with a metadata filter restricts the HNSW beam walk to the
matching candidates INSIDE the traversal (walk-all, collect-allowed) rather than
filtering the top-k afterward — so a query whose nearest vectors are all filtered
out still returns the best matches that DO pass the filter, instead of empty.
- JS HNSW search() honors the 8.0 `allowedIds: OpaqueIdSet | ReadonlySet<string>`
contract param (ANDs with candidateIds; ignores the opaque Buffer form it can't
decode — only a native provider consumes that).
- MetadataIndexProvider gains an OPTIONAL `getIdSetForFilter(filter): OpaqueIdSet`
producer — the native (cor) metadata index returns its roaring filter result as
a serialized buffer; the JS index does not implement it.
- find() forwards the matched universe to the vector walk: the opaque buffer
(zero id materialization) when the native producer is present, alongside the
materialized visibility-precise candidateIds for the JS index. Visibility stays
correct via the existing post-search hard filter.
Tested: JS recall/restriction/AND/opaque-ignore on the index directly, plus
find() forwarding the opaque universe through to the beam walk via a stubbed
native producer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
