feat(8.0): asOf at-gen vector defer — provider-served historical semantic search (#35)

A filtered semantic read at a historical generation (db.asOf(g).find({ query, where }))
no longer unconditionally rebuilds an ephemeral JS-HNSW over every at-g vector
(O(n@G), the OOM ceiling the 2026-06-24 scaling audit flagged). When the vector
index is a VersionedIndexProvider that advertises isGenerationVisible(g), Brainy:
  1. resolves the at-g metadata∩graph universe from the record-overlay path (no
     materialization — it's the metadata-only historical find),
  2. routes the vector leg to the provider with { allowedIds, generation }, and
  3. composes the at-g entities ranked by the provider's at-g vector distance.
Without a versioned provider (the JS index, or a native one that refuses the
generation) it falls through to the existing materialization — unchanged.

- Seam (A): VectorIndexProvider.search gains an optional `generation?: bigint`
  ("omitted = now"), the vector mirror of the graph provider's trailing-gen + the
  #46 allowedIds field. JsHnswVectorIndex accepts and ignores it (no per-gen
  segments → refuse/fall-back posture; Brainy never routes a historical read there).
- Gate: Db.find tries the native at-gen vector path before materialize(); host
  exposes canServeVectorAtGeneration + vectorSearchAtGeneration.
- generation is Brainy's u64 commit counter — the same value handed to the graph
  index on writes (graphWriteGeneration), so it maps 1:1 to the native side.

Decided lockstep with the cor team (handoff #35 thread: (A) + vector-only). The
gen-g candidate-vector supply for cor's exact-rerank (part-3) is a separate,
non-blocking seam still being confirmed; the honesty guard keeps this path inactive
(falls through to materialize) until cor's native at-gen rerank is live.

Tested: provider routing + at-gen universe correctness (excludes future-born,
applies the filter) + page window via a mock versioned provider; seam-ignore on the
JS index; 38 db-mvcc/db-temporal historical-read tests green (materialize fallback).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

src/brainy.ts

@@ -6929,6 +6929,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         persistPinned: (targetPath, generation) =>
           this.persistPinnedGeneration(targetPath, generation),
         materializeAt: (generation) => this.materializeAtGeneration(generation),
+        canServeVectorAtGeneration: (generation) => this.canServeVectorAtGeneration(generation),
+        vectorSearchAtGeneration: (params, allowedIds, k, generation) =>
+          this.vectorSearchAtGeneration(params, allowedIds, k, generation),
         pinGeneration: (generation) => this.pinGeneration(generation),
         releaseGeneration: (generation) => this.releaseGeneration(generation),
         registerDbForFinalization: (db, generation, closeOnRelease) => {
@@ -7251,6 +7254,50 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     }
   }
 
+  /**
+   * @description 8.0 #35 — true when the vector index is a
+   * {@link VersionedIndexProvider} that advertises `isGenerationVisible(generation)`:
+   * it can serve the at-`generation` vector leg from retained segments, so a
+   * historical filtered semantic read needs no O(n@G) JS-HNSW rebuild. The built-in
+   * `JsHnswVectorIndex` is not versioned (only ever "now"), and a native provider
+   * that cannot honor `generation` MUST return `false` here (refuse, never fabricate
+   * now-vectors-as-at-gen), so the caller falls back to materialization.
+   */
+  private canServeVectorAtGeneration(generation: number): boolean {
+    return (
+      isVersionedIndexProvider(this.index) &&
+      this.index.isGenerationVisible(BigInt(generation))
+    )
+  }
+
+  /**
+   * @description 8.0 #35 — run the vector kNN AS OF `generation`, restricted to
+   * `allowedIds` (the at-gen metadata∩graph universe the `Db` resolved from the
+   * record layer). The versioned provider serves the at-gen walk; Brainy composes
+   * the metadata half. Only reached when {@link canServeVectorAtGeneration} is true.
+   * @param params - The semantic (`query`) or explicit-`vector` find params.
+   * @param allowedIds - The at-gen candidate universe (membership-correct at `generation`).
+   * @param k - Over-fetch (page + headroom).
+   * @param generation - The as-of generation (Brainy's u64 commit counter).
+   * @returns `[id, distance]` pairs, ascending distance (descending relevance).
+   */
+  private async vectorSearchAtGeneration(
+    params: FindParams<T>,
+    allowedIds: ReadonlySet<string>,
+    k: number,
+    generation: number
+  ): Promise<Array<[string, number]>> {
+    const vector = params.vector || (await this.embed(params.query!))
+    // The provider scores the at-gen vectors restricted to `allowedIds`. (#35 part-3:
+    // the gen-g candidate VECTORS — `atGenerationVectors` — are supplied here once that
+    // seam is confirmed with cor; a provider needing them refuses the generation until
+    // then, so `canServeVectorAtGeneration` gates this path off in the interim.)
+    return this.index.search(vector, k, undefined, {
+      allowedIds,
+      generation: BigInt(generation)
+    })
+  }
+
   // --- Transact planner ------------------------------------------------------
 
   /**

src/db/db.ts

@@ -144,6 +144,29 @@ export interface DbHost<T = any> {
    * caller) and freed via the returned handle's `close()`.
    */
   materializeAt(generation: number): Promise<HistoricalQueryHandle<T>>
+  /**
+   * 8.0 #35 at-gen vector defer: true when the vector index is a
+   * {@link ../plugin.js VersionedIndexProvider} that advertised
+   * `isGenerationVisible(generation)` — i.e. it can serve the at-`generation`
+   * vector leg natively from retained segments, so a filtered semantic read needs
+   * NO O(n@G) materialization. False on the JS index (and whenever a native
+   * provider refuses the generation), so the caller falls back to `materializeAt`.
+   */
+  canServeVectorAtGeneration(generation: number): boolean
+  /**
+   * Run the vector kNN for `params` (semantic or explicit-vector) AS OF
+   * `generation`, restricted to `allowedIds` (the at-gen metadata∩graph universe
+   * the caller resolved from the record layer). Returns `[id, distance]` pairs,
+   * descending relevance. Only reached when {@link DbHost.canServeVectorAtGeneration}
+   * is true — the provider serves the at-gen walk; Brainy composes the metadata
+   * half. `k` is the over-fetch (page + headroom).
+   */
+  vectorSearchAtGeneration(
+    params: FindParams<T>,
+    allowedIds: ReadonlySet<string>,
+    k: number,
+    generation: number
+  ): Promise<Array<[string, number]>>
   /** Add one refcounted pin (store + versioned providers) on `generation`. */
   pinGeneration(generation: number): void
   /** Release one refcounted pin (store + versioned providers) on `generation`. */
@@ -339,6 +362,12 @@ export class Db<T = any> {
     if (this.overlay) {
       this.assertOverlayCompatibleFind(params)
     } else if (findRequiresIndexes(params)) {
+      // 8.0 #35: a FILTERED semantic/vector read at a historical generation can be
+      // served by a native at-gen vector provider (no O(n@G) materialization) — the
+      // metadata∩graph universe comes free from the record-overlay path, the vector
+      // leg from the provider. Falls through to materialization when not available.
+      const native = await this.tryAtGenerationVectorFind(params)
+      if (native !== null) return native
       const materialized = await this.materialize()
       return materialized.find(params)
     }
@@ -416,6 +445,71 @@ export class Db<T = any> {
     }
   }
 
+  /**
+   * @description 8.0 #35 — try to serve a FILTERED semantic/vector read at this
+   * historical generation via the native at-gen vector provider, with no O(n@G)
+   * materialization. Eligible only when: the query needs the vector leg
+   * (`query`/`vector`) WITH a metadata filter (the `allowedIds` source) and NO
+   * other index dimension (`near`/`connected`/`cursor`/`aggregate`/
+   * `includeRelations`), AND the host's vector index can serve this generation
+   * ({@link DbHost.canServeVectorAtGeneration}). The at-gen metadata∩graph universe
+   * is resolved through this view's OWN record-overlay path (the metadata-only
+   * `find`, which costs no materialization); the provider then ranks the vector
+   * leg restricted to that universe, and rows are hydrated from the at-gen universe
+   * entities (so metadata reflects `generation`, not now). Returns `null` when
+   * ineligible — the caller falls back to materialization.
+   */
+  private async tryAtGenerationVectorFind(params: FindParams<T>): Promise<Result<T>[] | null> {
+    const wantsVector = params.query !== undefined || params.vector !== undefined
+    const hasOtherIndexDim =
+      params.near !== undefined ||
+      params.connected !== undefined ||
+      params.cursor !== undefined ||
+      params.aggregate !== undefined ||
+      params.includeRelations === true
+    const hasMetadataFilter = Boolean(
+      params.where || params.type || params.subtype || params.service || params.excludeVFS
+    )
+    if (!wantsVector || hasOtherIndexDim || !hasMetadataFilter) return null
+    if (!this.host.canServeVectorAtGeneration(this.gen)) return null
+
+    // At-gen metadata∩graph universe via the record-overlay path (no materialization):
+    // the same query with the vector legs stripped resolves through the metadata
+    // historical branch. Capped so a pathological filter can't materialize an
+    // unbounded universe; the provider walk is restricted to whatever the cap yields.
+    const universeParams: FindParams<T> = { ...params }
+    delete universeParams.query
+    delete universeParams.vector
+    delete universeParams.searchMode
+    delete universeParams.mode
+    delete universeParams.orderBy
+    delete universeParams.order
+    universeParams.limit = AT_GEN_VECTOR_UNIVERSE_CAP
+    universeParams.offset = 0
+    const universe = await this.find(universeParams)
+    if (universe.length === 0) return []
+
+    const limit = params.limit ?? 10
+    const offset = params.offset ?? 0
+    const allowedIds = new Set(universe.map((r) => r.id))
+    const scored = await this.host.vectorSearchAtGeneration(
+      params,
+      allowedIds,
+      (offset + limit) * 2,
+      this.gen
+    )
+
+    // Compose: each at-gen metadata entity (from the universe) ranked by the
+    // provider's at-gen vector distance.
+    const byId = new Map(universe.map((r) => [r.id, r]))
+    const ranked: Result<T>[] = []
+    for (const [id, distance] of scored) {
+      const row = byId.get(id)
+      if (row) ranked.push({ ...row, score: Math.max(0, Math.min(1, 1 / (1 + distance))) })
+    }
+    return ranked.slice(offset, offset + limit)
+  }
+
   /**
    * @description Serialize part or all of this database value into a portable
    * `PortableGraph` document, read **as of this view's generation**. Because `export`
@@ -986,6 +1080,14 @@ function findRequiresIndexes(params: FindParams): boolean {
   return indexOnlyFindDimensions(params).some(([, present]) => present)
 }
 
+/**
+ * Cap on the at-gen metadata universe materialized for a native at-gen vector find
+ * (#35) — bounds a pathological filter; the provider's vector walk is restricted to
+ * whatever the cap yields. Only the metadata (no vectors) is held, so this is far
+ * cheaper than the full-corpus JS-HNSW rebuild it replaces.
+ */
+const AT_GEN_VECTOR_UNIVERSE_CAP = 100_000
+
 /**
  * @description Build a `Result` row from a record/overlay-resolved entity,
  * mirroring the live metadata-only find path (flattened fields, score `1.0`).

src/hnsw/hnswIndex.ts

@@ -667,6 +667,12 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
       rerank?: { multiplier: number }
       candidateIds?: string[]
       allowedIds?: OpaqueIdSet | ReadonlySet<string>
+      // 8.0 #35 at-gen seam: the JS index has no retained per-generation segments
+      // (it only ever holds "now"), so it ignores `generation` — exactly the
+      // refuse/fall-back posture the contract requires. Brainy never routes a
+      // historical read here (its defer-gate checks `isGenerationVisible` first),
+      // so an ignored `generation` cannot silently return now-vectors-as-at-gen.
+      generation?: bigint
     }
   ): Promise<Array<[string, number]>> {
     if (this.nouns.size === 0) {

src/plugin.ts

@@ -769,6 +769,19 @@ export interface VectorIndexProvider {
        * falls back to `filter`.
        */
       allowedIds?: OpaqueIdSet | ReadonlySet<string>
+      /**
+       * As-of generation for historical (time-travel) reads — the vector-side
+       * mirror of the {@link GraphAccelerationProvider} trailing `generation?`.
+       * Omitted = "now" (the live index). When set, a {@link VersionedIndexProvider}
+       * serves the kNN / exact-rerank over its retained at-`generation` segments
+       * instead of the current vectors, so `db.asOf(g)` semantic queries need no
+       * O(n@G) JS-HNSW rebuild. Brainy only passes it when the provider advertised
+       * `isGenerationVisible(generation)` at pin time; a provider that does not
+       * honor it MUST refuse/fall back (never score current vectors as if at-gen) —
+       * Brainy then serves the historical leg from its own materialization. Brainy's
+       * `generation` is the same u64 counter handed to the graph index on writes.
+       */
+      generation?: bigint
     }
   ): Promise<Array<[string, number]>>
   size(): number