2026-07-13 13:08:52 -07:00
/ * *
* @module indexReadiness
* @description The single honest - readiness classifier shared by the vector ,
* graph and metadata index sites . It exists to kill "Pattern A" — the dishonest
* readiness proxy where ` size() > 0 ` / ` isInitialized ` is treated as " this index
* actually serves queries . " A cold native index that loaded its COUNT but not its
* SERVING structure passes those proxies and silently returns ` [] ` .
*
* This classifier reads ONLY the provider ' s OPTIONAL , honest ` isReady() ` signal
* ( see { @link import ( '../plugin.js' ) . VectorIndexProvider . isReady } ,
* { @link import ( '../plugin.js' ) . GraphIndexProvider . isReady } ,
* { @link import ( '../plugin.js' ) . MetadataIndexProvider . isReady } ) . It NEVER inspects
* ` size() ` or ` isInitialized ` . When ` isReady() ` is absent , callers must fall back
* to a KNOWN - ITEM PROBE ( a real search / lookup that must return a known - present
* datum ) before trusting an empty result — never a ` size() ` proxy .
feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door
The read gate stops consulting the unnamed isReady() boolean: every provider
may expose healthReport() (sync, O(1), composed from exact ledgers —
HealthReport with a monotonic generation, per-invariant source
ledger|deep|unledgered, missing {count, sample}), and one readiness
authority (assessProviderHealth) derives the verdict. Unledgered families
are UNKNOWN — never healthy, never broken; a report that throws is a loud
not-ready, never a shrug. Reads at the four index choke points refuse with
the typed NotReady errors, narrated once per (provider, generation) — a
read NEVER starts a store walk:
- the first-read lazy build retires (open builds instead, regardless of
size — the ≥10k deferral and the "lazy loading on first query" branch go;
disableAutoRebuild is re-meant honestly in its docs);
- the verify*Live read-path rebuild triggers retire (refuse-or-serve);
- the read-time consistency probe that could launch a dark rebuild from an
ordinary find() retires;
- repairIndex({ rebuild: ['metadata'|'graph'|'vector'] | 'all' }) is the
one explicit door: rebuilds the named leg unconditionally and reports
rebuilt per family; bare repairIndex() stays report-driven.
test(lifecycle): the biography lane — a store's whole life, refereed
tests/lifecycle/: an independent shadow model referees every read after
every chapter (founding, a working day, clean restart, crash, repair,
second life). Chapters 1-3 green. Chapters 4-6 assert the true contract and
are marked .fails as a release-blocking finding (the kill-matrix
convention): after a crash + adopt reopen the metadata index computes its
'catchup' watermark verdict and nothing consumes it — find() serves the
pre-crash index while canonical and counts recover. The catchup wiring is
the cure; a passing .fails will force the marker's removal. The lane runs
in the integration gate (config + coverage guard).
2026-08-24 12:45:51 -07:00
*
* { @link assessProviderHealth } is the NEWER , PREFERRED authority : it reads a
* provider 's NAMED, synchronous, O(1) {@link import(' . . / plugin . js ' ) . HealthReport }
* when one is exposed , and falls back to this file ' s ` isReady() ` classifier only
* when the provider does not ( yet ) expose a health report . Read paths in
* ` brainy.ts ` call ` assessProviderHealth ` exclusively — ` assessIndexReadiness `
* stays exported for the other call sites ( ` storage/baseStorage.ts ` ) and for the
* fallback branch inside ` assessProviderHealth ` itself .
2026-07-13 13:08:52 -07:00
* /
feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door
The read gate stops consulting the unnamed isReady() boolean: every provider
may expose healthReport() (sync, O(1), composed from exact ledgers —
HealthReport with a monotonic generation, per-invariant source
ledger|deep|unledgered, missing {count, sample}), and one readiness
authority (assessProviderHealth) derives the verdict. Unledgered families
are UNKNOWN — never healthy, never broken; a report that throws is a loud
not-ready, never a shrug. Reads at the four index choke points refuse with
the typed NotReady errors, narrated once per (provider, generation) — a
read NEVER starts a store walk:
- the first-read lazy build retires (open builds instead, regardless of
size — the ≥10k deferral and the "lazy loading on first query" branch go;
disableAutoRebuild is re-meant honestly in its docs);
- the verify*Live read-path rebuild triggers retire (refuse-or-serve);
- the read-time consistency probe that could launch a dark rebuild from an
ordinary find() retires;
- repairIndex({ rebuild: ['metadata'|'graph'|'vector'] | 'all' }) is the
one explicit door: rebuilds the named leg unconditionally and reports
rebuilt per family; bare repairIndex() stays report-driven.
test(lifecycle): the biography lane — a store's whole life, refereed
tests/lifecycle/: an independent shadow model referees every read after
every chapter (founding, a working day, clean restart, crash, repair,
second life). Chapters 1-3 green. Chapters 4-6 assert the true contract and
are marked .fails as a release-blocking finding (the kill-matrix
convention): after a crash + adopt reopen the metadata index computes its
'catchup' watermark verdict and nothing consumes it — find() serves the
pre-crash index while canonical and counts recover. The catchup wiring is
the cure; a passing .fails will force the marker's removal. The lane runs
in the integration gate (config + coverage guard).
2026-08-24 12:45:51 -07:00
import type { HealthReport } from '../plugin.js'
2026-07-13 13:08:52 -07:00
/** A provider that MAY expose the honest cold-load readiness signal. */
export interface MaybeReadyProvider {
isReady ? : ( ) = > boolean
}
feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door
The read gate stops consulting the unnamed isReady() boolean: every provider
may expose healthReport() (sync, O(1), composed from exact ledgers —
HealthReport with a monotonic generation, per-invariant source
ledger|deep|unledgered, missing {count, sample}), and one readiness
authority (assessProviderHealth) derives the verdict. Unledgered families
are UNKNOWN — never healthy, never broken; a report that throws is a loud
not-ready, never a shrug. Reads at the four index choke points refuse with
the typed NotReady errors, narrated once per (provider, generation) — a
read NEVER starts a store walk:
- the first-read lazy build retires (open builds instead, regardless of
size — the ≥10k deferral and the "lazy loading on first query" branch go;
disableAutoRebuild is re-meant honestly in its docs);
- the verify*Live read-path rebuild triggers retire (refuse-or-serve);
- the read-time consistency probe that could launch a dark rebuild from an
ordinary find() retires;
- repairIndex({ rebuild: ['metadata'|'graph'|'vector'] | 'all' }) is the
one explicit door: rebuilds the named leg unconditionally and reports
rebuilt per family; bare repairIndex() stays report-driven.
test(lifecycle): the biography lane — a store's whole life, refereed
tests/lifecycle/: an independent shadow model referees every read after
every chapter (founding, a working day, clean restart, crash, repair,
second life). Chapters 1-3 green. Chapters 4-6 assert the true contract and
are marked .fails as a release-blocking finding (the kill-matrix
convention): after a crash + adopt reopen the metadata index computes its
'catchup' watermark verdict and nothing consumes it — find() serves the
pre-crash index while canonical and counts recover. The catchup wiring is
the cure; a passing .fails will force the marker's removal. The lane runs
in the integration gate (config + coverage guard).
2026-08-24 12:45:51 -07:00
/** A provider that MAY expose the named, synchronous, O(1) health report. */
export interface MaybeHealthReportingProvider {
healthReport ? : ( ) = > HealthReport
}
2026-07-13 13:08:52 -07:00
/** Three-valued honest-readiness verdict. */
export type IndexReadiness = 'ready' | 'not-ready' | 'unknown'
/ * *
* @description Classify an index provider ' s honest readiness .
* @param provider - Any index provider ( vector / graph / metadata ) or ` null ` .
* @returns
* - ` 'ready' ` when ` isReady() === true ` ( serving structure loaded — trust it ) ;
* - ` 'not-ready' ` when ` isReady() === false ` ( count / manifest loaded , NOT serving — rebuild ) ;
* - ` 'unknown' ` when the provider exposes no ` isReady() ` ( caller must probe / keep the JS heuristic ) .
* /
export function assessIndexReadiness ( provider : unknown ) : IndexReadiness {
const p = provider as MaybeReadyProvider | null | undefined
if ( p == null || typeof p . isReady !== 'function' ) return 'unknown'
return p . isReady ( ) ? 'ready' : 'not-ready'
}
feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door
The read gate stops consulting the unnamed isReady() boolean: every provider
may expose healthReport() (sync, O(1), composed from exact ledgers —
HealthReport with a monotonic generation, per-invariant source
ledger|deep|unledgered, missing {count, sample}), and one readiness
authority (assessProviderHealth) derives the verdict. Unledgered families
are UNKNOWN — never healthy, never broken; a report that throws is a loud
not-ready, never a shrug. Reads at the four index choke points refuse with
the typed NotReady errors, narrated once per (provider, generation) — a
read NEVER starts a store walk:
- the first-read lazy build retires (open builds instead, regardless of
size — the ≥10k deferral and the "lazy loading on first query" branch go;
disableAutoRebuild is re-meant honestly in its docs);
- the verify*Live read-path rebuild triggers retire (refuse-or-serve);
- the read-time consistency probe that could launch a dark rebuild from an
ordinary find() retires;
- repairIndex({ rebuild: ['metadata'|'graph'|'vector'] | 'all' }) is the
one explicit door: rebuilds the named leg unconditionally and reports
rebuilt per family; bare repairIndex() stays report-driven.
test(lifecycle): the biography lane — a store's whole life, refereed
tests/lifecycle/: an independent shadow model referees every read after
every chapter (founding, a working day, clean restart, crash, repair,
second life). Chapters 1-3 green. Chapters 4-6 assert the true contract and
are marked .fails as a release-blocking finding (the kill-matrix
convention): after a crash + adopt reopen the metadata index computes its
'catchup' watermark verdict and nothing consumes it — find() serves the
pre-crash index while canonical and counts recover. The catchup wiring is
the cure; a passing .fails will force the marker's removal. The lane runs
in the integration gate (config + coverage guard).
2026-08-24 12:45:51 -07:00
/ * *
* @description Which signal { @link assessProviderHealth } actually consulted to
* produce its verdict — surfaced so callers can narrate ( and tests can pin ) how
* a provider was judged , not just what the judgment was .
* - ` 'health-report' ` — the provider ' s ` healthReport() ` was called ( the authority ) .
* - ` 'is-ready' ` — no ` healthReport() ` ; fell back to the provider ' s ` isReady() ` .
* - ` 'size-heuristic' ` — no ` healthReport() ` and no ` isReady() ` ; caller must keep its own size - based heuristic .
* - ` 'none' ` — there was no provider to assess ( ` null ` / ` undefined ` ) .
* /
export type ProviderHealthVia = 'health-report' | 'is-ready' | 'size-heuristic' | 'none'
/** The result of {@link assessProviderHealth}. */
export interface ProviderHealthAssessment {
/** The honest readiness verdict — see {@link IndexReadiness}. */
readiness : IndexReadiness
/** The provider's raw {@link HealthReport}, when one was obtained; `null` otherwise. */
report : HealthReport | null
/** Which signal produced the verdict — see {@link ProviderHealthVia}. */
via : ProviderHealthVia
/** Human-readable reasons: named failing invariants (with `heal`), unledgered families, or the fallback-path explanation. Empty when the provider is healthy and ready. */
reasons : string [ ]
}
/ * *
* @description THE read - gate authority . Prefers a provider ' s NAMED ,
* synchronous , O ( 1 ) { @link HealthReport } over the older ` isReady() ` / size
* heuristics ; falls back to { @link assessIndexReadiness } ' s semantics only when
* a provider does not ( yet ) expose ` healthReport() ` .
*
* Derivation :
* - ` healthReport() ` present → call it ( wrapped in try / c a t c h ) . A T H R O W i s a
* CONTRACT VIOLATION , not "unknown" : returns ` readiness: 'not-ready' ` ,
* ` via: 'health-report' ` , and a reason naming the throw — never swallowed
* into ` 'unknown' ` .
* - Otherwise → ` readiness = report.serving ? 'ready' : 'not-ready' ` ; ` reasons `
* names every invariant with ` holds: false ` ( with its ` heal ` ) , plus an
* ` unledgered: [...] ` line when { @link HealthReport . unledgered } is non - empty .
* UNLEDGERED IS UNKNOWN : an unledgered family never flips a serving provider
* to not - ready , and never flips a not - serving provider to ready — ` serving `
* is always the provider ' s own verdict , verbatim .
* - No ` healthReport() ` → fall back to { @link assessIndexReadiness } ' s semantics :
* ` via: 'is-ready' ` when ` isReady() ` exists , ` via: 'size-heuristic' ` when
* neither hook exists ( caller must keep its own size - based heuristic ) ,
* ` via: 'none' ` when there is no provider at all .
* @param provider - Any index provider ( vector / graph / metadata ) or ` null ` / ` undefined ` .
* /
export function assessProviderHealth ( provider : unknown ) : ProviderHealthAssessment {
const p = provider as ( MaybeHealthReportingProvider & MaybeReadyProvider ) | null | undefined
if ( p == null ) {
return { readiness : 'unknown' , report : null , via : 'none' , reasons : [ 'no provider to assess' ] }
}
if ( typeof p . healthReport === 'function' ) {
let report : HealthReport
try {
report = p . healthReport ( )
} catch ( err ) {
const message = err instanceof Error ? err.message : String ( err )
return {
readiness : 'not-ready' ,
report : null ,
via : 'health-report' ,
reasons : [ ` healthReport() threw: ${ message } — a health-report throw is a contract violation, never read as healthy ` ]
}
}
const reasons : string [ ] = [ ]
for ( const invariant of report . invariants ) {
if ( ! invariant . holds ) {
reasons . push ( ` ${ invariant . name } (heal: ${ invariant . heal } ): ${ invariant . detail } ` )
}
}
if ( report . unledgered . length > 0 ) {
reasons . push ( ` unledgered: ${ report . unledgered . join ( ', ' ) } ` )
}
return {
readiness : report.serving ? 'ready' : 'not-ready' ,
report ,
via : 'health-report' ,
reasons
}
}
const readiness = assessIndexReadiness ( p )
if ( readiness === 'unknown' ) {
return {
readiness ,
report : null ,
via : 'size-heuristic' ,
reasons : [ 'provider exposes neither healthReport() nor isReady() — falling back to the size heuristic' ]
}
}
return {
readiness ,
report : null ,
via : 'is-ready' ,
reasons : readiness === 'not-ready' ? [ 'isReady() returned false' ] : [ ]
}
}
feat(open): open never waits for a provider that is rebuilding itself
MEASURED on a production store: a metadata provider that had to rebuild made
init() pay the ENTIRE rebuild on the foreground — 641 seconds — with every
other family idle behind it. The cause is a missing distinction: a provider
reporting serving:false because it is BUSY BUILDING ITSELF and one reporting
serving:false because it is BROKEN looked identical through healthReport(),
and both were answered the same way — call rebuild(), and wait for it.
The contract that tells them apart is one optional, synchronous, O(1) hook:
`rebuildInProgress(): ProviderRebuildProgress | null`, reporting a phase name
and whatever the provider actually measures (done/total/startedAt) — never an
estimate dressed as a fact. A provider without the hook behaves exactly as
before.
With it, a provider owns its own rebuild:
- the open gate neither starts a second rebuild nor waits for the provider's,
and narrates that it is not waiting and what will refuse meanwhile;
- init() returns and every other family serves;
- that family's doors refuse BY NAME, carrying the provider's own progress,
and say plainly that the door opens by itself and no action is needed —
distinct from a broken index, which names repairIndex();
- the epoch stamp does not advance while any family is still being built.
Nothing is ever served empty: a not-serving family refuses, as it already did.
Pins: tests/integration/open-does-not-wait-for-a-rebuilding-provider.test.ts —
init() returns in milliseconds against a provider claiming a 6s rebuild, brainy
starts no rebuild of its own, a filtered read refuses naming the phase and the
4,096/14,056 progress, and the door answers once the provider reports serving.
The pin fails loudly rather than vacuously if its stub never installs.
2026-08-28 10:48:52 -07:00
/ * *
* @description A provider ' s self - report that it is REBUILDING ITS OWN index
* right now . Returned by the optional ` rebuildInProgress() ` hook .
*
* The distinction this exists to make : a provider reporting ` serving: false `
* because it is BROKEN and a provider reporting ` serving: false ` because it is
* BUSY BUILDING ITSELF look identical through ` healthReport() ` alone , and
* brainy treated both the same way — it called ` rebuild() ` and waited for it ,
* on the foreground of ` init() ` . A production store whose metadata provider
* had to rebuild paid 641 SECONDS of that wait before ` init() ` returned , with
* every other family idle behind it .
*
* A provider that reports progress here owns its own rebuild : brainy neither
* starts one nor waits for it , ` init() ` returns , the other families serve , and
* THAT family ' s doors refuse by name — carrying this progress — until the
* provider reports itself serving .
*
* Every field but ` phase ` is optional and every field is a MEASUREMENT : a
* provider reports only what it actually tracks , never an estimate dressed as
* a fact .
* /
export interface ProviderRebuildProgress {
/** The provider's own name for what it is doing. Quoted verbatim in refusals. */
phase : string
/** Units completed so far, if the provider counts them. */
done? : number
/** Units expected in total, if the provider knows it. */
total? : number
/** Epoch millis when this rebuild started, if the provider tracks it. */
startedAt? : number
}
/** A provider that can report a rebuild it is running itself. */
interface MaybeRebuildingProvider {
rebuildInProgress ? : ( ) = > ProviderRebuildProgress | null
}
/ * *
* @description Ask a provider whether it is rebuilding itself right now .
* Synchronous , O ( 1 ) , feature - detected : a provider without the hook reports
* nothing and is treated exactly as before .
* @param provider - Any index provider , or ` null ` / ` undefined ` .
* @returns The provider ' s progress , or ` null ` when it is not rebuilding ( or
* does not implement the hook ) .
* /
export function assessProviderRebuild ( provider : unknown ) : ProviderRebuildProgress | null {
const p = provider as MaybeRebuildingProvider | null | undefined
if ( p == null || typeof p . rebuildInProgress !== 'function' ) return null
try {
const progress = p . rebuildInProgress ( )
if ( ! progress || typeof progress . phase !== 'string' || progress . phase . length === 0 ) {
return null
}
return progress
} catch {
// A throwing hook says nothing trustworthy about a rebuild; fall through to
// the ordinary health verdict rather than inventing one.
return null
}
}
/ * *
* @description Render a rebuild progress report as one operator - facing clause ,
* for a refusal message . Includes only what the provider actually measured .
* @param progress - The provider ' s report .
* @returns A clause such as ` rebuilding ("metadata shadow build", 4,096/14,056, 12s elapsed) ` .
* /
export function describeRebuildProgress ( progress : ProviderRebuildProgress ) : string {
const parts : string [ ] = [ ` " ${ progress . phase } " ` ]
if ( typeof progress . done === 'number' && typeof progress . total === 'number' ) {
parts . push ( ` ${ progress . done . toLocaleString ( ) } / ${ progress . total . toLocaleString ( ) } ` )
} else if ( typeof progress . done === 'number' ) {
parts . push ( ` ${ progress . done . toLocaleString ( ) } done ` )
}
if ( typeof progress . startedAt === 'number' ) {
parts . push ( ` ${ Math . round ( ( Date . now ( ) - progress . startedAt ) / 1000 ) } s elapsed ` )
}
return ` rebuilding ( ${ parts . join ( ', ' ) } ) `
}