soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix(8.0): re-validate find() results against the predicate (index-integrity guard)

Browse source 
find() trusted the metadata index's returned ids: it loaded each id's entity
and returned it, checking only that the entity existed — never that it actually
matched the query. The indexes are acceleration structures; the loaded entity
is ground truth. A stale or cross-bucket index posting (e.g. an id left in a
field-value bucket by a delete or an `update({ field: undefined })`) therefore
surfaced an entity matching NEITHER the requested type NOR the where filter — a
production report saw a timeslot (NounType.Event) returned for
`find({ type: Person, where: { entityType: 'staff' } })`.

Add a single egress chokepoint after the result IIFE that re-validates every
result with `entityMatchesFind` (type/subtype/where/service/excludeVFS) — the
live-path mirror of the historical path's per-candidate check in db.ts. It
covers every find() branch (metadata, vector, text, proximity, graph) and
`similar()` (which delegates to find) in one place. A no-op on a healthy index;
on a corrupted one it drops the bad row instead of returning a phantom. Full
unit + integration gate green confirms the where re-validation is consistent
with the index (no valid rows dropped).

This is the brainy-side safety net. The durable corruption itself lives in the
metadata index that owns the query (the native provider when present) and is
addressed separately.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-24 16:25:12 -07:00
parent 8d5032bee5
commit 3d116190f3
2 changed files with 121 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

21
src/brainy.ts
View file

@ -146,6 +146,7 @@ import { resolveJsHnswConfig, DEFAULT_RECALL } from './utils/recallPreset.js'
import * as fs from 'node:fs'
import * as os from 'node:os'
import { Db, type DbHost, type HistoricalQueryHandle } from './db/db.js'
import { entityMatchesFind } from './db/whereMatcher.js'
import {
importGraph,
isPortableGraph,
@ -4706,7 +4707,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const isHidden = (id: string): boolean => hiddenIds.size > 0 && hiddenIds.has(id)
const startTime = Date.now()
const result = await (async () => {
let result = await (async () => {
let results: Result<T>[] = []
// Distinguish between search criteria (need vector search) and filter criteria (metadata only)
@ -5140,6 +5141,24 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return results.slice(finalOffset, finalOffset + limit)
})()
// Index-integrity guard — applied ONCE here so every find() path (metadata,
// vector, text, proximity, graph) is covered uniformly. The indexes are
// acceleration structures; the loaded entity is ground truth. Re-validate
// each result against the query predicate so a stale or cross-bucket index
// entry — e.g. an id left in a field-value posting by a delete or an
// `update({ field: undefined })` — can never surface an entity that does not
// actually match `type`/`subtype`/`where`/`service`/`excludeVFS`. This is
// the live-path mirror of the historical path's per-candidate
// `entityMatchesFind` (db.ts). On a healthy index it is a no-op; on a
// corrupted one it drops the bad row instead of returning a phantom.
if (result.length > 0) {
result = result.filter(
(r) =>
r.entity != null &&
entityMatchesFind(r.entity as unknown as Entity, params as unknown as FindParams)
)
}
// includeVectors — opt-in vector hydration. Default (false) keeps the perf
// contract: every result path above builds entities via the metadata-only
// fast path, so `entity.vector` is the empty stub. When requested, fetch the