feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door
The read gate stops consulting the unnamed isReady() boolean: every provider
may expose healthReport() (sync, O(1), composed from exact ledgers —
HealthReport with a monotonic generation, per-invariant source
ledger|deep|unledgered, missing {count, sample}), and one readiness
authority (assessProviderHealth) derives the verdict. Unledgered families
are UNKNOWN — never healthy, never broken; a report that throws is a loud
not-ready, never a shrug. Reads at the four index choke points refuse with
the typed NotReady errors, narrated once per (provider, generation) — a
read NEVER starts a store walk:
- the first-read lazy build retires (open builds instead, regardless of
size — the ≥10k deferral and the "lazy loading on first query" branch go;
disableAutoRebuild is re-meant honestly in its docs);
- the verify*Live read-path rebuild triggers retire (refuse-or-serve);
- the read-time consistency probe that could launch a dark rebuild from an
ordinary find() retires;
- repairIndex({ rebuild: ['metadata'|'graph'|'vector'] | 'all' }) is the
one explicit door: rebuilds the named leg unconditionally and reports
rebuilt per family; bare repairIndex() stays report-driven.
test(lifecycle): the biography lane — a store's whole life, refereed
tests/lifecycle/: an independent shadow model referees every read after
every chapter (founding, a working day, clean restart, crash, repair,
second life). Chapters 1-3 green. Chapters 4-6 assert the true contract and
are marked .fails as a release-blocking finding (the kill-matrix
convention): after a crash + adopt reopen the metadata index computes its
'catchup' watermark verdict and nothing consumes it — find() serves the
pre-crash index while canonical and counts recover. The catchup wiring is
the cure; a passing .fails will force the marker's removal. The lane runs
in the integration gate (config + coverage guard).
This commit is contained in:
parent
a8b5ca0c8f
commit
f8f64780b1
19 changed files with 2160 additions and 652 deletions
|
|
@ -1,29 +1,29 @@
|
|||
/**
|
||||
* @module tests/integration/cold-graph-connected-8.0
|
||||
* @description BRAINY-COLD-GRAPH-CONNECTED (8.0) — regression coverage for the silent-empty
|
||||
* graph-traversal bug, gated on the converged 8.0 contract: a sync `graphIndex.isReady()` that
|
||||
* is true ONLY when the source→target EDGES are loaded (NOT the membership/manifest count).
|
||||
* graph-traversal bug, gated on the honest readiness signal: a sync `graphIndex.isReady()`
|
||||
* that is true ONLY when the source→target EDGES are loaded (NOT the membership/manifest count).
|
||||
*
|
||||
* On the FIRST `find({ connected })` after a cold process start of a LARGE brain (≥10k nouns,
|
||||
* which skips the eager index rebuild), a native graph adjacency can reload its relationship
|
||||
* COUNT (so `size() > 0`) but NOT its edges — so `getNeighbors()` returns `[]` for EVERY source
|
||||
* and brainy would serve that `[]` as if the anchor were genuinely edgeless.
|
||||
* On the FIRST `find({ connected })` after a cold process start, a native graph adjacency can
|
||||
* reload its relationship COUNT (so `size() > 0`) but NOT its edges — so `getNeighbors()` returns
|
||||
* `[]` for EVERY source and brainy would serve that `[]` as if the anchor were genuinely edgeless.
|
||||
*
|
||||
* The 8.0 guard (`verifyGraphAdjacencyLive`) prefers the honest `isReady()` signal:
|
||||
* - `isReady() === false` → hydrate the id-mapper, rebuild from storage, re-check; a still-false
|
||||
* `isReady()` throws {@link GraphIndexNotReadyError} instead of returning `[]` ('rebuilt' when
|
||||
* the rebuild heals it);
|
||||
* RE-POINTED to the health-gate law: `verifyGraphAdjacencyLive` NEVER rebuilds and NEVER walks the
|
||||
* store from a read — a read-path rebuild is exactly the dark-rebuild failure mode the law retires
|
||||
* (open() alone owns building). The guard now:
|
||||
* - `isReady() === false` → THROWS {@link GraphIndexNotReadyError} immediately — no rebuild attempt;
|
||||
* - a genuinely edgeless anchor with `isReady() === true` verifies 'live' and the empty result
|
||||
* stands — no spurious rebuild, no throw;
|
||||
* - a provider WITHOUT `isReady()` falls back to the shipped 7.x known-edge-sample probe.
|
||||
* stands — no spurious throw;
|
||||
* - a provider WITHOUT `isReady()` falls back to the shipped known-edge-sample probe, which is
|
||||
* now READ-ONLY: it refuses loudly (throws) rather than self-healing via rebuild.
|
||||
*
|
||||
* These exercise REAL `find({ connected })` against an in-memory brain whose graph index is
|
||||
* instrumented with a test-double `isReady()` (and, for the fallback case, an empty-then-healed
|
||||
* instrumented with a test-double `isReady()` (and, for the fallback case, an always-empty
|
||||
* `getNeighbors`). Only the readiness/edge surface is wrapped; the underlying real adjacency
|
||||
* (built by `relate()`) is unmasked once a rebuild "heals" it.
|
||||
* (built by `relate()`) is what a healthy provider actually serves.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, afterEach } from 'vitest'
|
||||
import { describe, it, expect, afterEach, vi } from 'vitest'
|
||||
import { Brainy } from '../../src/index.js'
|
||||
import { NounType, VerbType } from '../../src/types/graphTypes.js'
|
||||
import { GraphIndexNotReadyError } from '../../src/errors/brainyError.js'
|
||||
|
|
@ -63,17 +63,17 @@ async function buildBrain(
|
|||
}
|
||||
|
||||
/**
|
||||
* Instrument the brain's real graph index with a test-double `isReady()` (the 8.0 contract) plus
|
||||
* an edge surface that goes empty while NOT ready. `getNeighbors` returns `[]` while `!ready`
|
||||
* (modelling the cold-unloaded adjacency) and delegates to the REAL index once a rebuild flips
|
||||
* `ready` on. `rebuild` is counted; it heals (`ready = true`) only when `healsOnRebuild` is set.
|
||||
* Pass `failFirstRebuild` to make the FIRST rebuild throw a transient error (without healing) so
|
||||
* the empty-result re-collect path in executeGraphSearch is exercised.
|
||||
* Instrument the brain's real graph index with a test-double `isReady()` (the honest-readiness
|
||||
* contract) plus an edge surface that goes empty while NOT ready. `getNeighbors` returns `[]`
|
||||
* while `!ready` (modelling the cold-unloaded adjacency) and delegates to the REAL index once
|
||||
* `ready` flips true (used only by the "healthy" control cases — the guard itself never flips
|
||||
* this anymore, since it never rebuilds). `rebuild` is counted so tests can assert it is NEVER
|
||||
* called by a read.
|
||||
*/
|
||||
function instrumentIsReady(
|
||||
brain: any,
|
||||
opts: { ready: boolean; healsOnRebuild: boolean; failFirstRebuild?: boolean }
|
||||
): { rebuildCalls: number } {
|
||||
opts: { ready: boolean }
|
||||
): { rebuildCalls: number; ready: boolean } {
|
||||
const gi = brain.graphIndex
|
||||
const origGetNeighbors = gi.getNeighbors.bind(gi)
|
||||
const state = { ready: opts.ready, rebuildCalls: 0 }
|
||||
|
|
@ -85,10 +85,6 @@ function instrumentIsReady(
|
|||
|
||||
gi.rebuild = async (): Promise<void> => {
|
||||
state.rebuildCalls++
|
||||
if (opts.failFirstRebuild && state.rebuildCalls === 1) {
|
||||
throw new Error('transient rebuild hiccup')
|
||||
}
|
||||
if (opts.healsOnRebuild) state.ready = true // unmask the real (already-populated) adjacency
|
||||
}
|
||||
|
||||
return state
|
||||
|
|
@ -96,12 +92,12 @@ function instrumentIsReady(
|
|||
|
||||
/**
|
||||
* Fallback instrumentation — a provider WITHOUT `isReady()` (older cortex / JS baseline). Wraps
|
||||
* `getNeighbors` to return `[]` while `broken` and delegates to the REAL index once a rebuild
|
||||
* heals it. This is the shipped 7.x known-edge-sample probe path on 8.0.
|
||||
* `getNeighbors` to always return `[]` while `broken`. This is the shipped known-edge-sample
|
||||
* probe path — now READ-ONLY: it refuses loudly rather than self-healing.
|
||||
*/
|
||||
function instrumentNoIsReady(
|
||||
brain: any,
|
||||
opts: { broken: boolean; healsOnRebuild: boolean }
|
||||
opts: { broken: boolean }
|
||||
): { rebuildCalls: number } {
|
||||
const gi = brain.graphIndex
|
||||
// Ensure the provider does NOT expose isReady() — the default JS provider doesn't.
|
||||
|
|
@ -114,13 +110,12 @@ function instrumentNoIsReady(
|
|||
|
||||
gi.rebuild = async (): Promise<void> => {
|
||||
state.rebuildCalls++
|
||||
if (opts.healsOnRebuild) state.broken = false
|
||||
}
|
||||
|
||||
return state
|
||||
}
|
||||
|
||||
describe('BRAINY-COLD-GRAPH-CONNECTED 8.0 — isReady()-gated, never serves a silent []', () => {
|
||||
describe('BRAINY-COLD-GRAPH-CONNECTED 8.0 — isReady()-gated, never serves a silent [], never rebuilds from a read', () => {
|
||||
let brains: any[] = []
|
||||
afterEach(async () => {
|
||||
for (const b of brains) {
|
||||
|
|
@ -131,35 +126,37 @@ describe('BRAINY-COLD-GRAPH-CONNECTED 8.0 — isReady()-gated, never serves a si
|
|||
}
|
||||
}
|
||||
brains = []
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
it('(a) isReady() false → rebuild heals it true → find({ connected }) returns correct N (rebuilt)', async () => {
|
||||
const { brain, anchorId, targetIds } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
const state = instrumentIsReady(brain, { ready: false, healsOnRebuild: true })
|
||||
|
||||
const results = await brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
|
||||
expect(state.rebuildCalls).toBeGreaterThanOrEqual(1) // detected not-ready + healed it
|
||||
const ids = results.map((r: any) => r.id).sort()
|
||||
expect(ids).toEqual(targetIds.sort()) // B, C, D — the real edges, served after the heal
|
||||
})
|
||||
|
||||
it('(b) isReady() stays false after rebuild → throws GraphIndexNotReadyError (NOT a silent [])', async () => {
|
||||
it('(a) isReady() false → THROWS GraphIndexNotReadyError immediately, no rebuild attempt', async () => {
|
||||
const { brain, anchorId } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
instrumentIsReady(brain, { ready: false, healsOnRebuild: false }) // rebuild never makes it ready
|
||||
const state = instrumentIsReady(brain, { ready: false })
|
||||
|
||||
await expect(
|
||||
brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
).rejects.toBeInstanceOf(GraphIndexNotReadyError)
|
||||
|
||||
expect(state.rebuildCalls).toBe(0) // a read never rebuilds — it refuses loudly instead
|
||||
})
|
||||
|
||||
it('(b) isReady() stays false → throws GraphIndexNotReadyError (NOT a silent [])', async () => {
|
||||
const { brain, anchorId } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
const state = instrumentIsReady(brain, { ready: false })
|
||||
|
||||
await expect(
|
||||
brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
).rejects.toBeInstanceOf(GraphIndexNotReadyError)
|
||||
expect(state.rebuildCalls).toBe(0)
|
||||
})
|
||||
|
||||
it('(c) edgeless anchor + isReady() true → returns [] with NO rebuild and NO throw', async () => {
|
||||
// The anchor has no edges, but E -> F does — the adjacency is genuinely loaded (ready).
|
||||
const { brain, anchorId } = await buildBrain({ anchorEdges: false })
|
||||
brains.push(brain)
|
||||
const state = instrumentIsReady(brain, { ready: true, healsOnRebuild: false })
|
||||
const state = instrumentIsReady(brain, { ready: true })
|
||||
|
||||
const results = await brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
|
||||
|
|
@ -170,7 +167,7 @@ describe('BRAINY-COLD-GRAPH-CONNECTED 8.0 — isReady()-gated, never serves a si
|
|||
it('(d) healthy isReady() true → correct results, NO rebuild', async () => {
|
||||
const { brain, anchorId, targetIds } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
const state = instrumentIsReady(brain, { ready: true, healsOnRebuild: false })
|
||||
const state = instrumentIsReady(brain, { ready: true })
|
||||
|
||||
const results = await brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
|
||||
|
|
@ -179,30 +176,30 @@ describe('BRAINY-COLD-GRAPH-CONNECTED 8.0 — isReady()-gated, never serves a si
|
|||
expect(ids).toEqual(targetIds.sort())
|
||||
})
|
||||
|
||||
it('(e) provider WITHOUT isReady() → falls back to the known-edge-sample probe (self-heals)', async () => {
|
||||
const { brain, anchorId, targetIds } = await buildBrain({ anchorEdges: true })
|
||||
it('(e) provider WITHOUT isReady() → the known-edge-sample probe REFUSES LOUDLY (never self-heals)', async () => {
|
||||
const { brain, anchorId } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
const state = instrumentNoIsReady(brain, { broken: true, healsOnRebuild: true })
|
||||
const state = instrumentNoIsReady(brain, { broken: true })
|
||||
|
||||
const results = await brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
await expect(
|
||||
brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
).rejects.toBeInstanceOf(GraphIndexNotReadyError)
|
||||
|
||||
expect(state.rebuildCalls).toBeGreaterThanOrEqual(1) // detected the empty adjacency + healed it
|
||||
const ids = results.map((r: any) => r.id).sort()
|
||||
expect(ids).toEqual(targetIds.sort()) // B, C, D — served after the heal
|
||||
expect(state.rebuildCalls).toBe(0) // the fallback probe is READ-ONLY — it never calls rebuild()
|
||||
})
|
||||
|
||||
it('(f) executeGraphSearch re-collect: a transient first rebuild leaves connectedIds empty; the empty-result guard then heals + re-collects', async () => {
|
||||
// First verify (inside neighbors()) hits a transient rebuild failure → returns 'live' without
|
||||
// healing, so getNeighbors stays empty and connectedIds is empty. The empty connectedIds set
|
||||
// then drives executeGraphSearch's own verify, whose rebuild now heals → 'rebuilt' → re-collect.
|
||||
const { brain, anchorId, targetIds } = await buildBrain({ anchorEdges: true })
|
||||
it('(f) an empty connectedIds set re-verifies against a not-serving adjacency and throws, rather than serving [] as truth', async () => {
|
||||
// executeGraphSearch's cold-load guard (connectedIds.size === 0 → re-verify) used to
|
||||
// interpret a healed rebuild as "re-collect and serve." That rebuild-and-heal path is
|
||||
// retired: the re-verify now either confirms a genuinely edgeless anchor ('live', case (c))
|
||||
// or — as here — discovers the adjacency itself is not serving, and throws.
|
||||
const { brain, anchorId } = await buildBrain({ anchorEdges: true })
|
||||
brains.push(brain)
|
||||
const state = instrumentIsReady(brain, { ready: false, healsOnRebuild: true, failFirstRebuild: true })
|
||||
const state = instrumentIsReady(brain, { ready: false })
|
||||
|
||||
const results = await brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
|
||||
expect(state.rebuildCalls).toBeGreaterThanOrEqual(2) // first transient, second heals
|
||||
const ids = results.map((r: any) => r.id).sort()
|
||||
expect(ids).toEqual(targetIds.sort()) // re-collected after the heal
|
||||
await expect(
|
||||
brain.find({ connected: { from: anchorId, direction: 'out' }, limit: 10 })
|
||||
).rejects.toBeInstanceOf(GraphIndexNotReadyError)
|
||||
expect(state.rebuildCalls).toBe(0)
|
||||
})
|
||||
})
|
||||
|
|
|
|||
352
tests/integration/health-gate.test.ts
Normal file
352
tests/integration/health-gate.test.ts
Normal file
|
|
@ -0,0 +1,352 @@
|
|||
/**
|
||||
* @module tests/integration/health-gate
|
||||
* @description Pins for the health-by-accounting read gate: the read gate stops
|
||||
* consulting an unnamed `isReady()` boolean and reads a NAMED, sync, O(1)
|
||||
* {@link HealthReport}; no read path may ever start a store walk; the open path
|
||||
* brings every provider to serving before it returns; an explicit operator door
|
||||
* (`repairIndex({ rebuild: [...] })`) rebuilds a named leg unconditionally.
|
||||
*
|
||||
* Providers here are white-box test doubles: a `healthReport()` (or, for the
|
||||
* interim-path pins, an `isReady()`) function assigned directly onto the LIVE
|
||||
* JS provider object, the same pattern `tests/unit/validate-invariants-delegation.test.ts`
|
||||
* uses for `validateInvariants`. This exercises brainy's real gate/verify code
|
||||
* against a controlled provider self-report — no engine mocks.
|
||||
*/
|
||||
import { describe, it, expect, afterEach, vi } from 'vitest'
|
||||
import { mkdtempSync, rmSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import {
|
||||
Brainy,
|
||||
NounType,
|
||||
VerbType,
|
||||
GraphIndexNotReadyError,
|
||||
MetadataIndexNotReadyError,
|
||||
VectorIndexNotReadyError
|
||||
} from '../../src/index.js'
|
||||
import type { HealthReport, LedgerInvariantResult } from '../../src/plugin.js'
|
||||
import { prodLog } from '../../src/utils/logger.js'
|
||||
import { createTestConfig } from '../helpers/test-factory.js'
|
||||
|
||||
/** The white-box surface these pins drive on a live brain instance. */
|
||||
interface BrainInternals {
|
||||
storage: {
|
||||
getNoun(id: string): Promise<unknown>
|
||||
getNounMetadata(id: string): Promise<unknown>
|
||||
getNouns(options?: unknown): Promise<unknown>
|
||||
getVerbs(options?: unknown): Promise<unknown>
|
||||
}
|
||||
index: { healthReport?: () => HealthReport; isReady?: () => boolean; rebuild(): Promise<void> }
|
||||
metadataIndex: {
|
||||
healthReport?: () => HealthReport
|
||||
isReady?: () => boolean
|
||||
rebuild(): Promise<void>
|
||||
validateInvariants?: () => Promise<unknown>
|
||||
}
|
||||
graphIndex: {
|
||||
healthReport?: () => HealthReport
|
||||
isReady?: () => boolean
|
||||
rebuild(): Promise<void>
|
||||
validateInvariants?: () => Promise<unknown>
|
||||
}
|
||||
rebuildIndexesIfNeeded(force?: boolean): Promise<void>
|
||||
}
|
||||
|
||||
function internalsOf(brain: Brainy): BrainInternals {
|
||||
return brain as unknown as BrainInternals
|
||||
}
|
||||
|
||||
function invariant(overrides: Partial<LedgerInvariantResult> = {}): LedgerInvariantResult {
|
||||
return {
|
||||
name: 'manifest-residency',
|
||||
holds: true,
|
||||
detail: 'ok',
|
||||
heal: 'none',
|
||||
source: 'ledger',
|
||||
...overrides
|
||||
}
|
||||
}
|
||||
|
||||
function healthReport(overrides: Partial<HealthReport> = {}): HealthReport {
|
||||
return {
|
||||
provider: 'vector',
|
||||
healthy: true,
|
||||
serving: true,
|
||||
invariants: [],
|
||||
checkedAt: Date.now(),
|
||||
durationMs: 1,
|
||||
generation: 1,
|
||||
unledgered: [],
|
||||
...overrides
|
||||
}
|
||||
}
|
||||
|
||||
const brains: Brainy[] = []
|
||||
const dirs: string[] = []
|
||||
afterEach(async () => {
|
||||
for (const b of brains.splice(0)) await b.close().catch(() => {})
|
||||
for (const d of dirs.splice(0)) rmSync(d, { recursive: true, force: true })
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('health gate (a) — not-serving refuses loudly, ZERO canonical reads during the refusal', () => {
|
||||
it('metadata not-serving: find() throws MetadataIndexNotReadyError naming the failing invariant', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'row', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.metadataIndex.healthReport = () =>
|
||||
healthReport({
|
||||
provider: 'metadata',
|
||||
serving: false,
|
||||
healthy: false,
|
||||
invariants: [invariant({ name: 'posted-count-floor', holds: false, heal: 'rebuild', detail: 'posted 2 < canonical 5' })]
|
||||
})
|
||||
|
||||
const getNounSpy = vi.spyOn(internals.storage, 'getNoun')
|
||||
const getNounMetadataSpy = vi.spyOn(internals.storage, 'getNounMetadata')
|
||||
const getNounsSpy = vi.spyOn(internals.storage, 'getNouns')
|
||||
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).rejects.toBeInstanceOf(MetadataIndexNotReadyError)
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).rejects.toThrow(/posted-count-floor/)
|
||||
|
||||
expect(getNounSpy).not.toHaveBeenCalled()
|
||||
expect(getNounMetadataSpy).not.toHaveBeenCalled()
|
||||
expect(getNounsSpy).not.toHaveBeenCalled()
|
||||
|
||||
delete internals.metadataIndex.healthReport
|
||||
})
|
||||
|
||||
it('graph not-serving: related() throws GraphIndexNotReadyError naming the failing invariant, no canonical reads', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
const a = await brain.add({ data: 'a', type: NounType.Person })
|
||||
const b = await brain.add({ data: 'b', type: NounType.Person })
|
||||
await brain.relate({ from: a, to: b, type: VerbType.Knows })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.graphIndex.healthReport = () =>
|
||||
healthReport({
|
||||
provider: 'graph',
|
||||
serving: false,
|
||||
healthy: false,
|
||||
invariants: [invariant({ name: 'adjacency-residency', holds: false, heal: 'rebuild', detail: 'edges not loaded' })]
|
||||
})
|
||||
|
||||
const getNounSpy = vi.spyOn(internals.storage, 'getNoun')
|
||||
const getVerbsSpy = vi.spyOn(internals.storage, 'getVerbs')
|
||||
|
||||
await expect(brain.related({ from: a })).rejects.toBeInstanceOf(GraphIndexNotReadyError)
|
||||
await expect(brain.related({ from: a })).rejects.toThrow(/adjacency-residency/)
|
||||
|
||||
expect(getNounSpy).not.toHaveBeenCalled()
|
||||
expect(getVerbsSpy).not.toHaveBeenCalled()
|
||||
|
||||
delete internals.graphIndex.healthReport
|
||||
})
|
||||
})
|
||||
|
||||
describe('health gate (b) — unledgered is unknown: never blocks a serving provider', () => {
|
||||
it('serving:true with an unledgered family and no failing invariant serves normally; at most one narration', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'row', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.metadataIndex.healthReport = () =>
|
||||
healthReport({
|
||||
provider: 'metadata',
|
||||
serving: true,
|
||||
healthy: true,
|
||||
invariants: [],
|
||||
unledgered: ['canonical-verb-coverage']
|
||||
})
|
||||
|
||||
const warnSpy = vi.spyOn(prodLog, 'warn')
|
||||
|
||||
const r1 = await brain.find({ where: { team: 'atlas' } })
|
||||
const r2 = await brain.find({ where: { team: 'atlas' } })
|
||||
expect(r1.length).toBe(1)
|
||||
expect(r2.length).toBe(1)
|
||||
|
||||
const narrations = warnSpy.mock.calls.filter(
|
||||
([msg]) => typeof msg === 'string' && msg.includes('canonical-verb-coverage')
|
||||
)
|
||||
expect(narrations.length).toBe(1) // one narration at most across both reads (same generation)
|
||||
|
||||
delete internals.metadataIndex.healthReport
|
||||
})
|
||||
})
|
||||
|
||||
describe('health gate (c) — degraded-but-serving narrates once per generation', () => {
|
||||
it('a heal:"repair" failure serves; narrates once per generation, twice across a generation bump', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'row', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
let generation = 1
|
||||
internals.index.healthReport = () =>
|
||||
healthReport({
|
||||
provider: 'vector',
|
||||
serving: true,
|
||||
healthy: false,
|
||||
invariants: [invariant({ name: 'stale-vector-counter', holds: false, heal: 'repair', detail: 'counter drift' })],
|
||||
generation
|
||||
})
|
||||
|
||||
const warnSpy = vi.spyOn(prodLog, 'warn')
|
||||
const countNarrations = () =>
|
||||
warnSpy.mock.calls.filter(([msg]) => typeof msg === 'string' && msg.includes('stale-vector-counter')).length
|
||||
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).resolves.toHaveLength(1)
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).resolves.toHaveLength(1)
|
||||
expect(countNarrations()).toBe(1) // same generation both times — one narration
|
||||
|
||||
generation = 2
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).resolves.toHaveLength(1)
|
||||
expect(countNarrations()).toBe(2) // generation bumped — a second narration
|
||||
|
||||
delete internals.index.healthReport
|
||||
})
|
||||
})
|
||||
|
||||
describe('health gate (d) — interim isReady()-only path (no healthReport) is unchanged', () => {
|
||||
it('isReady() === true serves; isReady() === false refuses via the typed NotReady error', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'row', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.metadataIndex.isReady = () => true
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).resolves.toHaveLength(1)
|
||||
|
||||
internals.metadataIndex.isReady = () => false
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).rejects.toBeInstanceOf(MetadataIndexNotReadyError)
|
||||
|
||||
delete internals.metadataIndex.isReady
|
||||
})
|
||||
})
|
||||
|
||||
describe('health gate (e) — open builds; the first read never does', () => {
|
||||
it('disableAutoRebuild:true on a populated store: open narrates + builds; the first find() triggers zero rebuilds', async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'brainy-healthgate-open-'))
|
||||
dirs.push(dir)
|
||||
|
||||
const writer = new Brainy({
|
||||
storage: { type: 'filesystem', path: dir },
|
||||
requireSubtype: false,
|
||||
silent: true,
|
||||
disableAutoRebuild: true
|
||||
})
|
||||
await writer.init()
|
||||
brains.push(writer)
|
||||
await writer.add({ data: 'row one', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await writer.flush()
|
||||
await brains.pop()!.close()
|
||||
|
||||
const warnSpy = vi.spyOn(prodLog, 'warn')
|
||||
const reader = new Brainy({
|
||||
storage: { type: 'filesystem', path: dir },
|
||||
requireSubtype: false,
|
||||
silent: true,
|
||||
disableAutoRebuild: true
|
||||
})
|
||||
const internals = internalsOf(reader)
|
||||
const rebuildSpy = vi.spyOn(internals, 'rebuildIndexesIfNeeded')
|
||||
|
||||
await reader.init()
|
||||
brains.push(reader)
|
||||
|
||||
expect(rebuildSpy).toHaveBeenCalledTimes(1) // open() built it, exactly once
|
||||
expect(
|
||||
warnSpy.mock.calls.some(
|
||||
([msg]) => typeof msg === 'string' && msg.includes('open() is building')
|
||||
)
|
||||
).toBe(true)
|
||||
|
||||
rebuildSpy.mockClear()
|
||||
const rows = await reader.find({ where: { team: 'atlas' } })
|
||||
expect(rebuildSpy).toHaveBeenCalledTimes(0) // the read never builds
|
||||
expect(rows.length).toBe(1)
|
||||
}, 30000)
|
||||
})
|
||||
|
||||
describe('health gate (f) — the ceremony door: explicit rebuild bypasses invariant consultation', () => {
|
||||
it("repairIndex({ rebuild: ['graph'] }) rebuilds unconditionally without consulting validateInvariants", async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'x', type: NounType.Concept })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
let validateCalls = 0
|
||||
internals.graphIndex.validateInvariants = async () => {
|
||||
validateCalls++
|
||||
return healthReport({ provider: 'graph' })
|
||||
}
|
||||
const rebuildSpy = vi.spyOn(internals.graphIndex, 'rebuild')
|
||||
|
||||
const report = await brain.repairIndex({ rebuild: ['graph'] })
|
||||
|
||||
expect(rebuildSpy).toHaveBeenCalledTimes(1)
|
||||
expect(validateCalls).toBe(0) // the door never consults validateInvariants to decide
|
||||
|
||||
const graphFamily = report.families.find((f) => f.family === 'provider:graph')
|
||||
expect(graphFamily?.rebuilt).toBe(true)
|
||||
expect(graphFamily?.checked).toBe(true)
|
||||
expect(graphFamily?.reason).toBe('explicit rebuild requested')
|
||||
|
||||
delete internals.graphIndex.validateInvariants
|
||||
})
|
||||
|
||||
it('bare repairIndex() on a healthy provider calls no rebuild()', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'x', type: NounType.Concept })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.graphIndex.validateInvariants = async () => healthReport({ provider: 'graph', healthy: true, serving: true })
|
||||
const rebuildSpy = vi.spyOn(internals.graphIndex, 'rebuild')
|
||||
|
||||
await brain.repairIndex()
|
||||
|
||||
expect(rebuildSpy).not.toHaveBeenCalled()
|
||||
|
||||
delete internals.graphIndex.validateInvariants
|
||||
})
|
||||
})
|
||||
|
||||
describe('health gate (g) — a throwing healthReport() is a contract violation, never read as healthy', () => {
|
||||
it('healthReport() that throws refuses loudly with the typed NotReady error naming the throw', async () => {
|
||||
const brain = new Brainy(createTestConfig({ silent: true }))
|
||||
await brain.init()
|
||||
brains.push(brain)
|
||||
await brain.add({ data: 'row', type: NounType.Document, metadata: { team: 'atlas' } })
|
||||
await brain.flush()
|
||||
|
||||
const internals = internalsOf(brain)
|
||||
internals.index.healthReport = () => {
|
||||
throw new Error('accelerator: mmap window busy')
|
||||
}
|
||||
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).rejects.toBeInstanceOf(VectorIndexNotReadyError)
|
||||
await expect(brain.find({ where: { team: 'atlas' } })).rejects.toThrow(/mmap window busy/)
|
||||
|
||||
delete internals.index.healthReport
|
||||
})
|
||||
})
|
||||
Loading…
Add table
Add a link
Reference in a new issue