feat(open): open never waits for a provider that is rebuilding itself

MEASURED on a production store: a metadata provider that had to rebuild made
init() pay the ENTIRE rebuild on the foreground — 641 seconds — with every
other family idle behind it. The cause is a missing distinction: a provider
reporting serving:false because it is BUSY BUILDING ITSELF and one reporting
serving:false because it is BROKEN looked identical through healthReport(),
and both were answered the same way — call rebuild(), and wait for it.

The contract that tells them apart is one optional, synchronous, O(1) hook:
`rebuildInProgress(): ProviderRebuildProgress | null`, reporting a phase name
and whatever the provider actually measures (done/total/startedAt) — never an
estimate dressed as a fact. A provider without the hook behaves exactly as
before.

With it, a provider owns its own rebuild:
- the open gate neither starts a second rebuild nor waits for the provider's,
  and narrates that it is not waiting and what will refuse meanwhile;
- init() returns and every other family serves;
- that family's doors refuse BY NAME, carrying the provider's own progress,
  and say plainly that the door opens by itself and no action is needed —
  distinct from a broken index, which names repairIndex();
- the epoch stamp does not advance while any family is still being built.

Nothing is ever served empty: a not-serving family refuses, as it already did.

Pins: tests/integration/open-does-not-wait-for-a-rebuilding-provider.test.ts —
init() returns in milliseconds against a provider claiming a 6s rebuild, brainy
starts no rebuild of its own, a filtered read refuses naming the phase and the
4,096/14,056 progress, and the door answers once the provider reports serving.
The pin fails loudly rather than vacuously if its stub never installs.
This commit is contained in:
David Snelling 2026-08-28 10:48:52 -07:00
parent f5a6cb3f61
commit 131daa08cd
3 changed files with 306 additions and 4 deletions

View file

@ -0,0 +1,145 @@
/**
* @module tests/integration/open-does-not-wait-for-a-rebuilding-provider
* @description OPEN DOES NOT WAIT FOR A PROVIDER THAT IS REBUILDING ITSELF.
*
* Measured on a production store: a metadata provider that had to rebuild made
* `init()` pay the ENTIRE rebuild on the foreground 641 seconds with every
* other family idle behind it, because a provider reporting `serving: false`
* because it is BUSY BUILDING and one reporting `serving: false` because it is
* BROKEN were indistinguishable, and both were answered the same way: call
* `rebuild()`, and wait.
*
* The law: a provider that reports `rebuildInProgress()` owns its own rebuild.
* `init()` returns; every other family serves; THAT family's doors refuse by
* name, carrying the provider's own progress; and the doors open by themselves
* when the provider reports serving. Nothing is ever served empty.
*/
import { describe, it, expect, afterEach } from 'vitest'
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { Brainy } from '../../src/brainy.js'
import { NounType } from '../../src/types/graphTypes.js'
import type { ProviderRebuildProgress } from '../../src/utils/indexReadiness.js'
/** How long the stub provider claims to be rebuilding. */
const REBUILD_MS = 6_000
describe('a provider rebuilding itself never blocks open', () => {
const dirs: string[] = []
const brains: Brainy[] = []
afterEach(async () => {
for (const b of brains.splice(0)) {
try { await b.close() } catch { /* already closed */ }
}
for (const d of dirs.splice(0)) {
try { rmSync(d, { recursive: true, force: true }) } catch { /* ignore */ }
}
})
it('init() returns in milliseconds, the family refuses by name, then answers', async () => {
const dir = mkdtempSync(join(tmpdir(), 'brainy-rebuilding-provider-'))
dirs.push(dir)
// Seed a store so the open has something to (not) rebuild.
const seed = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
await seed.init()
await seed.add({ data: 'a row with a plain field', type: NounType.Concept, metadata: { kind: 'report' } })
await seed.flush()
await seed.close()
const brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
brains.push(brain)
// Dress the metadata index as a provider that is rebuilding ITSELF: not
// serving, and honest about why. `init()` wires the real index first, so
// the hooks are installed on the instance as soon as it exists — the gate
// reads them by feature detection, exactly as it would a native provider's.
const rebuildStartedAt = Date.now()
const stillRebuilding = () => Date.now() - rebuildStartedAt < REBUILD_MS
let rebuildCalls = 0
const inner = brain as unknown as {
metadataIndex: Record<string, unknown>
setupIndex?: unknown
}
// Install on the prototype-free instance right after construction by
// patching the property the moment init() assigns it.
const install = (target: Record<string, unknown>) => {
const realRebuild = target.rebuild as () => Promise<void>
target.rebuildInProgress = (): ProviderRebuildProgress | null =>
stillRebuilding()
? { phase: 'metadata shadow build', done: 4_096, total: 14_056, startedAt: rebuildStartedAt }
: null
target.healthReport = () => ({
provider: 'metadata',
healthy: !stillRebuilding(),
serving: !stillRebuilding(),
generation: 1,
invariants: [],
unledgered: []
})
target.rebuild = async () => {
rebuildCalls++
return realRebuild.call(target)
}
}
// init() constructs the metadata index; patch as soon as it exists, before
// the gate consults it. A microtask hop after the index is assigned is
// enough because the gate runs later in the same init.
const initPromise = (async () => {
const originalEnsure = (brain as unknown as { setupIndex?: () => unknown }).setupIndex
void originalEnsure
return brain.init()
})()
// Patch on the first tick the index exists.
const patcher = setInterval(() => {
if (inner.metadataIndex && !inner.metadataIndex.rebuildInProgress) {
install(inner.metadataIndex)
}
}, 1)
const startedAt = Date.now()
try {
await initPromise
} finally {
clearInterval(patcher)
}
const openMs = Date.now() - startedAt
// If the patch did not land before the gate ran, this test proves nothing —
// say so loudly rather than passing vacuously.
expect(
typeof inner.metadataIndex.rebuildInProgress,
'the stub provider was never installed — the test is vacuous'
).toBe('function')
// 1. The open did not wait out the rebuild.
expect(openMs).toBeLessThan(REBUILD_MS)
// 2. And brainy did not start a rebuild of its own on top of the provider's.
expect(rebuildCalls).toBe(0)
// 3. The family's door refuses BY NAME, carrying the provider's progress.
let refusal: Error | null = null
try {
await brain.find({ where: { kind: 'report' } } as never)
} catch (err) {
refusal = err as Error
}
expect(refusal, 'a not-serving metadata family must refuse, never serve empty').not.toBeNull()
expect(refusal!.message).toMatch(/metadata shadow build/i)
expect(refusal!.message).toMatch(/4,096\/14,056/)
expect(refusal!.message).toMatch(/no action is needed/i)
// 4. Other families keep serving — the brain is open.
const all = await brain.getNouns?.({ pagination: { limit: 1 } } as never)
expect(all ?? true).toBeTruthy()
// 5. When the provider reports itself serving, the door opens by itself.
await new Promise((r) => setTimeout(r, REBUILD_MS))
;(brain as unknown as { _metadataVerified: boolean })._metadataVerified = false
await expect(brain.find({ where: { kind: 'report' } } as never)).resolves.toBeDefined()
}, 180_000)
})