soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: never serve a silent [] from find({connected}) on a cold-loaded graph

Browse source 
On a cold process start of a large brain (above the eager index-rebuild
threshold), a native graph adjacency can report size()>0 — its membership set
reloaded — while the source->target edges did NOT load, so find({connected}),
neighbors() and related() returned [] for edges that are persisted on disk. A
database returning empty for data that exists, based purely on warm/cold state,
is a correctness bug; it hit a production deployment after every deploy.

Refactor the cold-load guard into verifyGraphAdjacencyLive(), which gates its
heal/throw decision on a GLOBAL known-edge sample (a persisted verb's source,
which by definition has an outgoing edge) rather than the queried anchor: a
stale adjacency is rebuilt from storage, an unrecoverable one throws
GraphIndexNotReadyError instead of serving [], and a genuinely edgeless anchor
still returns [] with no spurious rebuild. executeGraphSearch re-verifies before
trusting an empty connected result and re-collects after a heal. Adds a 5-case
integration test (self-heal / loud-throw / edgeless-no-false-positive / healthy
/ re-collect) plus updated unit coverage.
This commit is contained in:
David Snelling 2026-06-29 16:40:02 -07:00
parent d1665bb1a9
commit fd699d0e07
3 changed files with 251 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

98
src/brainy.ts
View file

@ -192,7 +192,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
private _versions?: VersioningAPI
private _vfs?: VirtualFileSystem
private _vfsInitialized = false // Track VFS init completion separately
private _graphAdjacencyVerified = false // 7.33.2: one-time graph-adjacency cold-load consistency check
private _graphAdjacencyVerified = false // graph-adjacency cold-load consistency: verified-live this session
private _graphAdjacencyVerifying = false // re-entrancy guard: a verify (rebuild→reads) is in flight
private _hub?: IntegrationHub // Integration Hub for external tools
private _pendingMigrationRunner?: MigrationRunner // Deferred migration runner for large datasets
private _aggregationIndex?: AggregationIndex // Incremental aggregation engine
@ -2488,7 +2489,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
paramsOrId?: string | GetRelationsParams
): Promise<Relation<T>[]> {
await this.ensureInitialized()
await this.ensureGraphAdjacencyConsistent()
await this.verifyGraphAdjacencyLive()
// Handle string ID shorthand: getRelations(id) -> getRelations({ from: id })
const params = typeof paramsOrId === 'string'
@ -8178,32 +8179,52 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* })
*/
/**
* @description 7.33.2 — guard against a graph index that loaded its relationship
* COUNT (manifest) on a cold open but NOT its source→target adjacency, so graph
* traversals (`find({ connected })`, `neighbors`, `related`) would silently return
* `[]` despite persisted edges. Observed with a native graph provider whose
* cold-open SSTable load is swallowed on some storage adapters. Runs ONCE per brain
* (cached): if the index claims edges exist but a known persisted edge's source
* resolves to no neighbors, force a rebuild from storage; if even that can't read
* the edge, throw {@link GraphIndexNotReadyError} rather than serving `[]` as truth.
* O(1) (one verb read + one neighbor probe) and a no-op once the adjacency is live.
* @description Verify that the graph adjacency is actually LIVE before a graph read trusts
* its result. A native graph index can load its relationship COUNT (manifest) on a cold open
* of a LARGE brain (≥10k nouns, which skips the eager index rebuild) but NOT its
* source→target adjacency, so `getNeighbors()` returns `[]` for EVERY source even though
* edges are persisted — and `find({ connected })` / `neighbors()` / `related()` would serve
* that `[]` as if it were truth.
*
* The check is gated on a GLOBAL known-edge sample (a real persisted verb's `sourceId`, which
* by definition HAS an outgoing edge) — NOT on any queried anchor, because brainy cannot
* cheaply tell "adjacency unloaded" from "this node is genuinely edgeless" per-anchor
* (`getVerbsBySource`/`getVerbsByTarget` route through the same `isInitialized`-gated fast path
* that lies on cold open). If that known-edge source resolves to no neighbors, the adjacency
* did not load: rebuild from storage and re-probe; if even that fails, throw
* {@link GraphIndexNotReadyError} rather than returning `[]`.
*
* @returns `'live'` when the adjacency is already trustworthy (or there is genuinely nothing
* to verify), or `'rebuilt'` when a cold-unloaded adjacency was just healed from storage —
* in which case callers that observed an empty result must RE-RUN their collection.
* @throws {GraphIndexNotReadyError} when the index claims edges but cannot serve a known
* persisted edge even after a rebuild.
*/
private async ensureGraphAdjacencyConsistent(): Promise<void> {
if (this._graphAdjacencyVerified) return
// Set up-front so a re-entrant call (rebuild → reads) cannot loop.
this._graphAdjacencyVerified = true
private async verifyGraphAdjacencyLive(): Promise<'live' | 'rebuilt'> {
if (this._graphAdjacencyVerified) return 'live'
// Re-entrancy: rebuild() triggers reads (neighbors/getRelations) that call back into this
// guard. While a verify is in flight, short-circuit so we cannot recurse into rebuild().
if (this._graphAdjacencyVerifying) return 'live'
this._graphAdjacencyVerifying = true
try {
const claimed = await this.graphIndex.size()
if (!claimed || claimed <= 0) return // no edges claimed — nothing to verify
if (!claimed || claimed <= 0) return 'live' // no edges claimed — nothing to verify
const sample = await this.storage.getVerbs({ pagination: { limit: 1 } })
const verb = sample.items?.[0]
if (!verb || !verb.sourceId) return // no edges in storage — stale count, harmless
if (!verb || !verb.sourceId) {
// no edges in storage — stale count, harmless; don't re-probe on every read.
this._graphAdjacencyVerified = true
return 'live'
}
const probe = await this.graphIndex.getNeighbors(verb.sourceId, { limit: 1 })
if (probe.length > 0) return // adjacency is live — the common case
if (probe.length > 0) {
this._graphAdjacencyVerified = true
return 'live' // adjacency is live — the common case
}
// INCONSISTENT: the index reports edges but a persisted edge's source has none →
// INCONSISTENT: the index reports edges but a KNOWN persisted edge's source has none →
// the adjacency did not load on open. Rebuild it from storage.
if (!this.config.silent) {
console.warn(
@ -8221,6 +8242,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
`neighbors() and related() cannot be served reliably for this brain.`
)
}
this._graphAdjacencyVerified = true
return 'rebuilt'
} catch (err) {
if (err instanceof GraphIndexNotReadyError) throw err
// A transient probe/rebuild failure must not break the actual query NOR be
@ -8229,6 +8252,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
if (!this.config.silent) {
console.warn(`[Brainy] Graph adjacency consistency check skipped (transient): ${err}`)
}
return 'live'
} finally {
this._graphAdjacencyVerifying = false
}
}
@ -8242,7 +8268,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
}
): Promise<string[]> {
await this.ensureInitialized()
await this.ensureGraphAdjacencyConsistent()
await this.verifyGraphAdjacencyLive()
const direction = options?.direction || 'both'
const limit = options?.limit
@ -8798,13 +8824,35 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const connectedIds = new Set<string>()
if (from) {
for (const id of await collect(from, toNeighborDir(direction))) connectedIds.add(id)
// Populate connectedIds from the from/to anchors. Extracted into a closure so it can be
// re-run VERBATIM after a cold-load heal (verdict 'rebuilt') without changing collection
// semantics — only re-executing the same traversal once the adjacency is actually loaded.
const populate = async (): Promise<void> => {
if (from) {
for (const id of await collect(from, toNeighborDir(direction))) connectedIds.add(id)
}
if (to) {
const reverse: 'in' | 'out' | 'both' = direction === 'in' ? 'out' : direction === 'out' ? 'in' : 'both'
for (const id of await collect(to, toNeighborDir(reverse))) connectedIds.add(id)
}
}
if (to) {
const reverse: 'in' | 'out' | 'both' = direction === 'in' ? 'out' : direction === 'out' ? 'in' : 'both'
for (const id of await collect(to, toNeighborDir(reverse))) connectedIds.add(id)
await populate()
// Cold-load guard: an empty connected set is suspicious. The native adjacency can report
// size()>0 on a cold open yet have loaded NO source→target edges — so traversal silently
// returns []. Re-verify against a GLOBAL known-edge sample (NOT the queried anchor, which
// may be genuinely edgeless). If the adjacency was dead and a rebuild healed it, re-collect;
// if it stays dead, verifyGraphAdjacencyLive() throws GraphIndexNotReadyError. A genuinely
// edgeless anchor verifies 'live' and the empty result stands — no spurious rebuild/throw.
let rechecked = false
if (connectedIds.size === 0 && !rechecked) {
const verdict = await this.verifyGraphAdjacencyLive()
if (verdict === 'rebuilt') {
rechecked = true
connectedIds.clear()
await populate()
}
}
// Subtype filter (depth-1 only — see guard above). Intersect connectedIds with the set of