soulcraftlabs/open-brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 342 Packages Wiki Activity Actions 6

feat(open): open never waits for a provider that is rebuilding itself

Browse source 
MEASURED on a production store: a metadata provider that had to rebuild made
init() pay the ENTIRE rebuild on the foreground — 641 seconds — with every
other family idle behind it. The cause is a missing distinction: a provider
reporting serving:false because it is BUSY BUILDING ITSELF and one reporting
serving:false because it is BROKEN looked identical through healthReport(),
and both were answered the same way — call rebuild(), and wait for it.

The contract that tells them apart is one optional, synchronous, O(1) hook:
`rebuildInProgress(): ProviderRebuildProgress | null`, reporting a phase name
and whatever the provider actually measures (done/total/startedAt) — never an
estimate dressed as a fact. A provider without the hook behaves exactly as
before.

With it, a provider owns its own rebuild:
- the open gate neither starts a second rebuild nor waits for the provider's,
  and narrates that it is not waiting and what will refuse meanwhile;
- init() returns and every other family serves;
- that family's doors refuse BY NAME, carrying the provider's own progress,
  and say plainly that the door opens by itself and no action is needed —
  distinct from a broken index, which names repairIndex();
- the epoch stamp does not advance while any family is still being built.

Nothing is ever served empty: a not-serving family refuses, as it already did.

Pins: tests/integration/open-does-not-wait-for-a-rebuilding-provider.test.ts —
init() returns in milliseconds against a provider claiming a 6s rebuild, brainy
starts no rebuild of its own, a filtered read refuses naming the phase and the
4,096/14,056 progress, and the door answers once the provider reports serving.
The pin fails loudly rather than vacuously if its stub never installs.
This commit is contained in:
David Snelling 2026-08-28 10:48:52 -07:00
parent f5a6cb3f61
commit 131daa08cd
3 changed files with 306 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

85
src/brainy.ts
View file

@ -198,7 +198,12 @@ import {
import { isDeterministicEmbedMode } from './embeddings/deterministicEmbedMode.js'
import { GenerationConflictError, StoreInconsistentError } from './db/errors.js'
import { BrainyError, GraphIndexNotReadyError, MetadataIndexNotReadyError, MigrationInProgressError, VectorIndexNotReadyError } from './errors/brainyError.js'
import { assessIndexReadiness, assessProviderHealth } from './utils/indexReadiness.js'
import {
assessIndexReadiness,
assessProviderHealth,
assessProviderRebuild,
describeRebuildProgress
} from './utils/indexReadiness.js'
import { reconstructNounWrapper } from './db/factLog.js'
import { asBrainyFieldRefusal } from './db/fieldAddressing.js'
import {
@ -4540,6 +4545,19 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this._graphAdjacencyVerified = true
return 'live'
}
// A provider that is REBUILDING ITSELF gets a refusal that says so,
// with its own progress: open deliberately did not wait for it (see
// rebuildIndexesIfNeeded), so this door is temporarily closed and will
// open on its own. Anything else is a broken index needing a repair.
const rebuilding = assessProviderRebuild(this.graphIndex)
if (rebuilding) {
throw new GraphIndexNotReadyError(
`Graph adjacency index is ${describeRebuildProgress(rebuilding)} and is not serving ` +
`yet. find({ connected }), neighbors() and related() refuse rather than serve an ` +
`empty result. The brain is open and every other family is serving; this door opens ` +
`by itself when the provider reports serving — no action is needed.`
)
}
throw new GraphIndexNotReadyError(
`Graph adjacency index is not serving (via ${assessment.via}): ` +
`${assessment.reasons.join('; ') || 'not ready'}. find({ connected }), neighbors() and ` +
@ -4643,6 +4661,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this._metadataVerified = true
return 'live'
}
const rebuilding = assessProviderRebuild(this.metadataIndex)
if (rebuilding) {
throw new MetadataIndexNotReadyError(
`Metadata field index is ${describeRebuildProgress(rebuilding)} and is not serving ` +
`yet. find({ where }) and other filtered reads refuse rather than serve an empty ` +
`result. The brain is open and every other family is serving; this door opens by ` +
`itself when the provider reports serving — no action is needed.`
)
}
throw new MetadataIndexNotReadyError(
`Metadata field index is not serving (via ${assessment.via}): ` +
`${assessment.reasons.join('; ') || 'not ready'}. find({ where }) and other filtered ` +
@ -4772,6 +4799,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this._vectorVerified = true
return 'live'
}
const rebuilding = assessProviderRebuild(this.index)
if (rebuilding) {
throw new VectorIndexNotReadyError(
`Vector index is ${describeRebuildProgress(rebuilding)} and is not serving yet. ` +
`Semantic find({ query }) and proximity search refuse rather than serve an empty ` +
`result. The brain is open and every other family is serving; this door opens by ` +
`itself when the provider reports serving — no action is needed.`
)
}
throw new VectorIndexNotReadyError(
`Vector index is not serving (via ${assessment.via}): ` +
`${assessment.reasons.join('; ') || 'not ready'}. Semantic find({ query }) and ` +
@ -17144,6 +17180,19 @@ export class Brainy<T = any> implements BrainyInterface<T> {
}
if (assessment.readiness === 'not-ready') {
// A provider REBUILDING ITSELF gets a refusal that says so, with its
// own progress: open deliberately did not wait for it, this door is
// temporarily closed, and it opens by itself. Distinct from a broken
// index, which needs an operator.
const rebuilding = assessProviderRebuild(provider)
if (rebuilding) {
throw new ErrorClass(
`${name} index is ${describeRebuildProgress(rebuilding)} and is not serving yet. ` +
`Reads of this family refuse rather than serve an empty result. The brain is open ` +
`and every other family is serving; this door opens by itself when the provider ` +
`reports serving — no action is needed.`
)
}
throw new ErrorClass(
`${name} index is not serving (via ${assessment.via}): ` +
`${assessment.reasons.join('; ') || 'not ready'}. Reads refuse rather than serve an ` +
@ -17466,9 +17515,37 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// by awaitMigrationLock meanwhile (nothing serves from a half-built index).
// Gated per-index, so a non-migrating sibling still rebuilds when it needs
// to; a migrating provider is skipped even under epoch-drift or size()===0.
const metadataMigrating = this.providerIsMigrating(this.metadataIndex)
const vectorMigrating = this.providerIsMigrating(this.index)
const graphMigrating = this.providerIsMigrating(this.graphIndex)
// SELF-REBUILD DEFERENCE (the sibling of the migration lock, and the
// reason a production open took 641 seconds): a provider that reports
// `rebuildInProgress()` is ALREADY rebuilding its own index. Brainy must
// neither start a second rebuild nor WAIT for the provider's — init()
// returns, every other family serves, and that family's own doors refuse
// by name (carrying this progress) until the provider reports serving.
// A provider without the hook behaves exactly as before.
const metadataRebuilding = assessProviderRebuild(this.metadataIndex)
const vectorRebuilding = assessProviderRebuild(this.index)
const graphRebuilding = assessProviderRebuild(this.graphIndex)
for (const [leg, progress] of [
['metadata', metadataRebuilding],
['vector', vectorRebuilding],
['graph', graphRebuilding]
] as const) {
if (progress) {
prodLog.narrate(
`[Brainy] open(): the ${leg} provider is ${describeRebuildProgress(progress)} — ` +
`open does NOT wait for it. The brain opens now, every other family serves, and ` +
`${leg} reads refuse by name until the provider reports itself serving.`
)
}
}
const metadataMigrating =
this.providerIsMigrating(this.metadataIndex) || metadataRebuilding !== null
const vectorMigrating = this.providerIsMigrating(this.index) || vectorRebuilding !== null
const graphMigrating = this.providerIsMigrating(this.graphIndex) || graphRebuilding !== null
// The epoch stamp certifies EVERY derived index, so it must not advance
// while any family is still being built — by a migration lock or by the
// provider itself.
const anyMigrating = metadataMigrating || vectorMigrating || graphMigrating
// Per-leg decision, in precedence order: a migrating provider owns its