fix: graph adjacency cold-load consistency guard — no more silent [] on connected

A native graph provider can load its relationship COUNT (manifest) on a cold open
but fail to load the source→target adjacency itself (observed on mmap-filesystem:
the SSTable-segment load is swallowed). The result is find({ connected }),
neighbors(), and related() silently returning [] despite persisted edges — and
staying empty after warm-up. Because it is [] not an error, callers cannot tell
real data is missing.

Brainy now runs a one-time consistency check on the first graph read: if the index
reports relationships exist but a known persisted edge's source resolves to no
neighbors, force a rebuild from storage (which re-scans + repopulates the adjacency);
if even that cannot read the edge, throw the new GraphIndexNotReadyError instead of
serving [] as truth. O(1) probe (one verb + one neighbor lookup), cached per brain,
and a no-op once the adjacency is live — so it self-heals the cold-open case and is
loud when it genuinely can't.

Wired into neighbors() + getRelations() (the graph-read primitives behind
find({ connected })). The underlying cold-open load is a native-provider concern
(addressed upstream); this makes the failure self-healing + loud rather than silent.

Tests: tests/unit/brainy/graph-adjacency-cold-load.test.ts (live → no-op; broken →
rebuild; unfixable → throws; size 0 / no edges → no-op; transient failure → re-checks
without breaking the query). Full unit gate green (1531/1531).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-24 10:55:12 -07:00
parent 811c7da89e
commit 1694f68419
4 changed files with 240 additions and 0 deletions

View file

@ -10,6 +10,32 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so
---
## v7.33.2 — 2026-06-24
**Affected products:** any consumer using graph traversals — `find({ connected: { from, via } })`,
`neighbors()`, `related()` — with a **native graph index** on **`mmap-filesystem`** (or any adapter
whose cold-open adjacency load can fail), where a process restart could leave graph queries returning
`[]` despite persisted edges. Drop-in; no API or data changes.
### Fix — graph adjacency no longer silently returns `[]` on a cold open
On a fresh brain open, a native graph provider can load its relationship **count** (the manifest) but
fail to load the source→target **adjacency** itself — so `find({ connected })` / `neighbors()` /
`related()` returned an empty array **indistinguishable from "no edges"**, and stayed empty after
warm-up. Because it was `[]` and not an error, callers could not tell real data was missing.
Brainy now runs a one-time consistency check on the first graph read: if the index reports that
relationships exist but a known persisted edge's source resolves to **no** neighbors, it **rebuilds
the adjacency from storage**. If even a rebuild cannot read the edge, it throws a loud, catchable
`GraphIndexNotReadyError` instead of serving `[]` as truth. The probe is O(1) (one edge + one neighbor
lookup), cached per brain, and a **no-op once the adjacency is live** — so consumers can return to
clean graph queries instead of in-memory-filter workarounds.
> The underlying cold-open load is a native-provider concern (being addressed upstream); this Brainy
> change makes the failure **self-healing and loud** rather than silent.
---
## v7.33.1 — 2026-06-22
**Affected products:** any consumer on `filesystem` / `mmap-filesystem` storage with **more than one

View file

@ -48,6 +48,7 @@ import { MmapVectorBackend } from './hnsw/mmapVectorBackend.js'
import { ConnectionsCodec } from './hnsw/connectionsCodec.js'
import { TransactionManager } from './transaction/TransactionManager.js'
import { RevisionConflictError } from './transaction/RevisionConflictError.js'
import { GraphIndexNotReadyError } from './errors/brainyError.js'
import {
ValidationConfig,
validateAddParams,
@ -191,6 +192,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
private _versions?: VersioningAPI
private _vfs?: VirtualFileSystem
private _vfsInitialized = false // Track VFS init completion separately
private _graphAdjacencyVerified = false // 7.33.2: one-time graph-adjacency cold-load consistency check
private _hub?: IntegrationHub // Integration Hub for external tools
private _pendingMigrationRunner?: MigrationRunner // Deferred migration runner for large datasets
private _aggregationIndex?: AggregationIndex // Incremental aggregation engine
@ -2449,6 +2451,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
paramsOrId?: string | GetRelationsParams
): Promise<Relation<T>[]> {
await this.ensureInitialized()
await this.ensureGraphAdjacencyConsistent()
// Handle string ID shorthand: getRelations(id) -> getRelations({ from: id })
const params = typeof paramsOrId === 'string'
@ -8122,6 +8125,61 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* verbType: VerbType.RELATES_TO
* })
*/
/**
* @description 7.33.2 guard against a graph index that loaded its relationship
* COUNT (manifest) on a cold open but NOT its sourcetarget adjacency, so graph
* traversals (`find({ connected })`, `neighbors`, `related`) would silently return
* `[]` despite persisted edges. Observed with a native graph provider whose
* cold-open SSTable load is swallowed on some storage adapters. Runs ONCE per brain
* (cached): if the index claims edges exist but a known persisted edge's source
* resolves to no neighbors, force a rebuild from storage; if even that can't read
* the edge, throw {@link GraphIndexNotReadyError} rather than serving `[]` as truth.
* O(1) (one verb read + one neighbor probe) and a no-op once the adjacency is live.
*/
private async ensureGraphAdjacencyConsistent(): Promise<void> {
if (this._graphAdjacencyVerified) return
// Set up-front so a re-entrant call (rebuild → reads) cannot loop.
this._graphAdjacencyVerified = true
try {
const claimed = await this.graphIndex.size()
if (!claimed || claimed <= 0) return // no edges claimed — nothing to verify
const sample = await this.storage.getVerbs({ pagination: { limit: 1 } })
const verb = sample.items?.[0]
if (!verb || !verb.sourceId) return // no edges in storage — stale count, harmless
const probe = await this.graphIndex.getNeighbors(verb.sourceId, { limit: 1 })
if (probe.length > 0) return // adjacency is live — the common case
// INCONSISTENT: the index reports edges but a persisted edge's source has none →
// the adjacency did not load on open. Rebuild it from storage.
if (!this.config.silent) {
console.warn(
`[Brainy] Graph adjacency reports ${claimed} relationship(s) but a persisted edge ` +
`resolves to none — the persisted adjacency did not load on open. Rebuilding from storage…`
)
}
await this.graphIndex.rebuild()
const reprobe = await this.graphIndex.getNeighbors(verb.sourceId, { limit: 1 })
if (reprobe.length === 0) {
throw new GraphIndexNotReadyError(
`Graph adjacency index reports ${claimed} relationship(s) but returns no edges even ` +
`after a rebuild — the persisted adjacency could not be loaded. find({ connected }), ` +
`neighbors() and related() cannot be served reliably for this brain.`
)
}
} catch (err) {
if (err instanceof GraphIndexNotReadyError) throw err
// A transient probe/rebuild failure must not break the actual query NOR be
// masked as "no data". Allow a re-check on the next graph read and fall through.
this._graphAdjacencyVerified = false
if (!this.config.silent) {
console.warn(`[Brainy] Graph adjacency consistency check skipped (transient): ${err}`)
}
}
}
async neighbors(
entityId: string,
options?: {
@ -8132,6 +8190,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
}
): Promise<string[]> {
await this.ensureInitialized()
await this.ensureGraphAdjacencyConsistent()
const direction = options?.direction || 'both'
const limit = options?.limit

View file

@ -11,6 +11,7 @@ export type BrainyErrorType =
| 'RETRY_EXHAUSTED'
| 'VALIDATION'
| 'FIELD_NOT_INDEXED'
| 'GRAPH_INDEX_NOT_READY'
/**
* Custom error class for Brainy operations
@ -223,3 +224,27 @@ export class BrainyError extends Error {
return BrainyError.storage(error.message, error)
}
}
/**
* Thrown when the graph adjacency index reports that relationships exist (its
* persisted manifest/count loaded) but the sourcetarget adjacency itself did
* NOT load so graph traversals (`find({ connected })`, `neighbors()`,
* `related()`) would otherwise return an EMPTY array indistinguishable from
* "no edges". Brainy detects this on the first graph read (one verb + one
* neighbor probe), attempts a rebuild from storage, and raises this LOUD,
* catchable error only if even that cannot read a known persisted edge
* replacing silent data-invisibility with a clear failure.
*
* Observed with a native graph provider whose cold-open adjacency load is
* swallowed on certain storage adapters; the fix is upstream in the provider,
* but Brainy refuses to serve `[]` as if it were truth.
*/
export class GraphIndexNotReadyError extends BrainyError {
constructor(message: string, originalError?: Error) {
super(message, 'GRAPH_INDEX_NOT_READY', false, originalError)
this.name = 'GraphIndexNotReadyError'
if (Error.captureStackTrace) {
Error.captureStackTrace(this, GraphIndexNotReadyError)
}
}
}

View file

@ -0,0 +1,130 @@
/**
* @module tests/unit/brainy/graph-adjacency-cold-load
* @description 7.33.2 the graph-adjacency cold-load consistency guard. A native
* graph provider can load its relationship COUNT (manifest) on a cold open but fail
* to load the sourcetarget adjacency (observed with cor 2.7.5's NativeGraphAdjacencyIndex
* on mmap-filesystem: the SSTable-segment load is swallowed). The result is
* `find({ connected })` / `neighbors()` / `related()` silently returning `[]` despite
* persisted edges. Brainy now probes one known persisted edge on the first graph read;
* if the index claims edges but the source resolves to none, it forces a rebuild from
* storage, and only if even that fails does it throw a loud, catchable error instead of
* serving `[]` as truth. These exercise the guard against a stubbed graph index.
*/
import { describe, it, expect } from 'vitest'
import { Brainy } from '../../../src/index.js'
import { GraphIndexNotReadyError } from '../../../src/errors/brainyError.js'
import { createTestConfig } from '../../helpers/test-factory.js'
/** Replace the brain's graph index with a stub that models a (possibly broken) cold load. */
function stubGraphIndex(
brain: any,
opts: { size: number; neighborsBeforeRebuild: string[]; neighborsAfterRebuild?: string[] }
) {
let rebuilt = false
const calls = { rebuild: 0, getNeighbors: 0 }
brain.graphIndex = {
size: () => opts.size,
getNeighbors: async () => {
calls.getNeighbors++
return rebuilt ? opts.neighborsAfterRebuild ?? [] : opts.neighborsBeforeRebuild
},
rebuild: async () => {
calls.rebuild++
rebuilt = true
},
flush: async () => {},
close: async () => {}
}
return calls
}
async function freshBrain(): Promise<any> {
const brain: any = new Brainy(createTestConfig())
await brain.init()
brain._graphAdjacencyVerified = false
return brain
}
describe('7.33.2 graph-adjacency cold-load consistency guard', () => {
it('is a no-op when the adjacency is live (a persisted edge resolves to neighbors)', async () => {
const brain = await freshBrain()
const calls = stubGraphIndex(brain, { size: 12, neighborsBeforeRebuild: ['n1'] })
brain.storage.getVerbs = async () => ({ items: [{ id: 'v1', sourceId: 's1', targetId: 't1' }], hasMore: false })
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(0)
await brain.close()
})
it('force-rebuilds when the index reports edges but the source probes empty', async () => {
const brain = await freshBrain()
const calls = stubGraphIndex(brain, { size: 12, neighborsBeforeRebuild: [], neighborsAfterRebuild: ['n1'] })
brain.storage.getVerbs = async () => ({ items: [{ id: 'v1', sourceId: 's1', targetId: 't1' }], hasMore: false })
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(1) // detected the empty adjacency + rebuilt it
// Cached: a second call does not re-probe or re-rebuild.
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(1)
await brain.close()
})
it('throws GraphIndexNotReadyError when even a rebuild cannot load the adjacency', async () => {
const brain = await freshBrain()
stubGraphIndex(brain, { size: 12, neighborsBeforeRebuild: [], neighborsAfterRebuild: [] }) // never loads
brain.storage.getVerbs = async () => ({ items: [{ id: 'v1', sourceId: 's1', targetId: 't1' }], hasMore: false })
await expect(brain.ensureGraphAdjacencyConsistent()).rejects.toBeInstanceOf(GraphIndexNotReadyError)
await brain.close()
})
it('no-op when no edges are claimed (size 0) — never even probes storage', async () => {
const brain = await freshBrain()
const calls = stubGraphIndex(brain, { size: 0, neighborsBeforeRebuild: [] })
let probedStorage = false
brain.storage.getVerbs = async () => {
probedStorage = true
return { items: [], hasMore: false }
}
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(0)
expect(probedStorage).toBe(false)
await brain.close()
})
it('no-op when storage has no edges (stale count) — does not rebuild or throw', async () => {
const brain = await freshBrain()
const calls = stubGraphIndex(brain, { size: 12, neighborsBeforeRebuild: [] })
brain.storage.getVerbs = async () => ({ items: [], hasMore: false })
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(0)
await brain.close()
})
it('does not loop or break the query on a transient probe failure (re-checks next time)', async () => {
const brain = await freshBrain()
const calls = stubGraphIndex(brain, { size: 12, neighborsBeforeRebuild: [], neighborsAfterRebuild: ['n1'] })
let firstCall = true
brain.storage.getVerbs = async () => {
if (firstCall) {
firstCall = false
throw new Error('transient storage hiccup')
}
return { items: [{ id: 'v1', sourceId: 's1', targetId: 't1' }], hasMore: false }
}
// First call: probe throws transiently → swallowed (no GraphIndexNotReadyError), flag reset.
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(0)
expect(brain._graphAdjacencyVerified).toBe(false) // re-check allowed
// Second call: storage works → detects the empty adjacency + rebuilds.
await brain.ensureGraphAdjacencyConsistent()
expect(calls.rebuild).toBe(1)
await brain.close()
})
})