MEASURED on a production store: a metadata provider that had to rebuild made init() pay the ENTIRE rebuild on the foreground — 641 seconds — with every other family idle behind it. The cause is a missing distinction: a provider reporting serving:false because it is BUSY BUILDING ITSELF and one reporting serving:false because it is BROKEN looked identical through healthReport(), and both were answered the same way — call rebuild(), and wait for it. The contract that tells them apart is one optional, synchronous, O(1) hook: `rebuildInProgress(): ProviderRebuildProgress | null`, reporting a phase name and whatever the provider actually measures (done/total/startedAt) — never an estimate dressed as a fact. A provider without the hook behaves exactly as before. With it, a provider owns its own rebuild: - the open gate neither starts a second rebuild nor waits for the provider's, and narrates that it is not waiting and what will refuse meanwhile; - init() returns and every other family serves; - that family's doors refuse BY NAME, carrying the provider's own progress, and say plainly that the door opens by itself and no action is needed — distinct from a broken index, which names repairIndex(); - the epoch stamp does not advance while any family is still being built. Nothing is ever served empty: a not-serving family refuses, as it already did. Pins: tests/integration/open-does-not-wait-for-a-rebuilding-provider.test.ts — init() returns in milliseconds against a provider claiming a 6s rebuild, brainy starts no rebuild of its own, a filtered read refuses naming the phase and the 4,096/14,056 progress, and the door answers once the provider reports serving. The pin fails loudly rather than vacuously if its stub never installs.
235 lines
10 KiB
TypeScript
235 lines
10 KiB
TypeScript
/**
|
|
* @module indexReadiness
|
|
* @description The single honest-readiness classifier shared by the vector,
|
|
* graph and metadata index sites. It exists to kill "Pattern A" — the dishonest
|
|
* readiness proxy where `size() > 0` / `isInitialized` is treated as "this index
|
|
* actually serves queries." A cold native index that loaded its COUNT but not its
|
|
* SERVING structure passes those proxies and silently returns `[]`.
|
|
*
|
|
* This classifier reads ONLY the provider's OPTIONAL, honest `isReady()` signal
|
|
* (see {@link import('../plugin.js').VectorIndexProvider.isReady},
|
|
* {@link import('../plugin.js').GraphIndexProvider.isReady},
|
|
* {@link import('../plugin.js').MetadataIndexProvider.isReady}). It NEVER inspects
|
|
* `size()` or `isInitialized`. When `isReady()` is absent, callers must fall back
|
|
* to a KNOWN-ITEM PROBE (a real search/lookup that must return a known-present
|
|
* datum) before trusting an empty result — never a `size()` proxy.
|
|
*
|
|
* {@link assessProviderHealth} is the NEWER, PREFERRED authority: it reads a
|
|
* provider's NAMED, synchronous, O(1) {@link import('../plugin.js').HealthReport}
|
|
* when one is exposed, and falls back to this file's `isReady()` classifier only
|
|
* when the provider does not (yet) expose a health report. Read paths in
|
|
* `brainy.ts` call `assessProviderHealth` exclusively — `assessIndexReadiness`
|
|
* stays exported for the other call sites (`storage/baseStorage.ts`) and for the
|
|
* fallback branch inside `assessProviderHealth` itself.
|
|
*/
|
|
|
|
import type { HealthReport } from '../plugin.js'
|
|
|
|
/** A provider that MAY expose the honest cold-load readiness signal. */
|
|
export interface MaybeReadyProvider {
|
|
isReady?: () => boolean
|
|
}
|
|
|
|
/** A provider that MAY expose the named, synchronous, O(1) health report. */
|
|
export interface MaybeHealthReportingProvider {
|
|
healthReport?: () => HealthReport
|
|
}
|
|
|
|
/** Three-valued honest-readiness verdict. */
|
|
export type IndexReadiness = 'ready' | 'not-ready' | 'unknown'
|
|
|
|
/**
|
|
* @description Classify an index provider's honest readiness.
|
|
* @param provider - Any index provider (vector / graph / metadata) or `null`.
|
|
* @returns
|
|
* - `'ready'` when `isReady() === true` (serving structure loaded — trust it);
|
|
* - `'not-ready'` when `isReady() === false` (count/manifest loaded, NOT serving — rebuild);
|
|
* - `'unknown'` when the provider exposes no `isReady()` (caller must probe / keep the JS heuristic).
|
|
*/
|
|
export function assessIndexReadiness(provider: unknown): IndexReadiness {
|
|
const p = provider as MaybeReadyProvider | null | undefined
|
|
if (p == null || typeof p.isReady !== 'function') return 'unknown'
|
|
return p.isReady() ? 'ready' : 'not-ready'
|
|
}
|
|
|
|
/**
|
|
* @description Which signal {@link assessProviderHealth} actually consulted to
|
|
* produce its verdict — surfaced so callers can narrate (and tests can pin) how
|
|
* a provider was judged, not just what the judgment was.
|
|
* - `'health-report'` — the provider's `healthReport()` was called (the authority).
|
|
* - `'is-ready'` — no `healthReport()`; fell back to the provider's `isReady()`.
|
|
* - `'size-heuristic'` — no `healthReport()` and no `isReady()`; caller must keep its own size-based heuristic.
|
|
* - `'none'` — there was no provider to assess (`null`/`undefined`).
|
|
*/
|
|
export type ProviderHealthVia = 'health-report' | 'is-ready' | 'size-heuristic' | 'none'
|
|
|
|
/** The result of {@link assessProviderHealth}. */
|
|
export interface ProviderHealthAssessment {
|
|
/** The honest readiness verdict — see {@link IndexReadiness}. */
|
|
readiness: IndexReadiness
|
|
/** The provider's raw {@link HealthReport}, when one was obtained; `null` otherwise. */
|
|
report: HealthReport | null
|
|
/** Which signal produced the verdict — see {@link ProviderHealthVia}. */
|
|
via: ProviderHealthVia
|
|
/** Human-readable reasons: named failing invariants (with `heal`), unledgered families, or the fallback-path explanation. Empty when the provider is healthy and ready. */
|
|
reasons: string[]
|
|
}
|
|
|
|
/**
|
|
* @description THE read-gate authority. Prefers a provider's NAMED,
|
|
* synchronous, O(1) {@link HealthReport} over the older `isReady()` / size
|
|
* heuristics; falls back to {@link assessIndexReadiness}'s semantics only when
|
|
* a provider does not (yet) expose `healthReport()`.
|
|
*
|
|
* Derivation:
|
|
* - `healthReport()` present → call it (wrapped in try/catch). A THROW is a
|
|
* CONTRACT VIOLATION, not "unknown": returns `readiness: 'not-ready'`,
|
|
* `via: 'health-report'`, and a reason naming the throw — never swallowed
|
|
* into `'unknown'`.
|
|
* - Otherwise → `readiness = report.serving ? 'ready' : 'not-ready'`; `reasons`
|
|
* names every invariant with `holds: false` (with its `heal`), plus an
|
|
* `unledgered: [...]` line when {@link HealthReport.unledgered} is non-empty.
|
|
* UNLEDGERED IS UNKNOWN: an unledgered family never flips a serving provider
|
|
* to not-ready, and never flips a not-serving provider to ready — `serving`
|
|
* is always the provider's own verdict, verbatim.
|
|
* - No `healthReport()` → fall back to {@link assessIndexReadiness}'s semantics:
|
|
* `via: 'is-ready'` when `isReady()` exists, `via: 'size-heuristic'` when
|
|
* neither hook exists (caller must keep its own size-based heuristic),
|
|
* `via: 'none'` when there is no provider at all.
|
|
* @param provider - Any index provider (vector / graph / metadata) or `null`/`undefined`.
|
|
*/
|
|
export function assessProviderHealth(provider: unknown): ProviderHealthAssessment {
|
|
const p = provider as (MaybeHealthReportingProvider & MaybeReadyProvider) | null | undefined
|
|
|
|
if (p == null) {
|
|
return { readiness: 'unknown', report: null, via: 'none', reasons: ['no provider to assess'] }
|
|
}
|
|
|
|
if (typeof p.healthReport === 'function') {
|
|
let report: HealthReport
|
|
try {
|
|
report = p.healthReport()
|
|
} catch (err) {
|
|
const message = err instanceof Error ? err.message : String(err)
|
|
return {
|
|
readiness: 'not-ready',
|
|
report: null,
|
|
via: 'health-report',
|
|
reasons: [`healthReport() threw: ${message} — a health-report throw is a contract violation, never read as healthy`]
|
|
}
|
|
}
|
|
|
|
const reasons: string[] = []
|
|
for (const invariant of report.invariants) {
|
|
if (!invariant.holds) {
|
|
reasons.push(`${invariant.name} (heal:${invariant.heal}): ${invariant.detail}`)
|
|
}
|
|
}
|
|
if (report.unledgered.length > 0) {
|
|
reasons.push(`unledgered: ${report.unledgered.join(', ')}`)
|
|
}
|
|
|
|
return {
|
|
readiness: report.serving ? 'ready' : 'not-ready',
|
|
report,
|
|
via: 'health-report',
|
|
reasons
|
|
}
|
|
}
|
|
|
|
const readiness = assessIndexReadiness(p)
|
|
if (readiness === 'unknown') {
|
|
return {
|
|
readiness,
|
|
report: null,
|
|
via: 'size-heuristic',
|
|
reasons: ['provider exposes neither healthReport() nor isReady() — falling back to the size heuristic']
|
|
}
|
|
}
|
|
return {
|
|
readiness,
|
|
report: null,
|
|
via: 'is-ready',
|
|
reasons: readiness === 'not-ready' ? ['isReady() returned false'] : []
|
|
}
|
|
}
|
|
|
|
/**
|
|
* @description A provider's self-report that it is REBUILDING ITS OWN index
|
|
* right now. Returned by the optional `rebuildInProgress()` hook.
|
|
*
|
|
* The distinction this exists to make: a provider reporting `serving: false`
|
|
* because it is BROKEN and a provider reporting `serving: false` because it is
|
|
* BUSY BUILDING ITSELF look identical through `healthReport()` alone, and
|
|
* brainy treated both the same way — it called `rebuild()` and waited for it,
|
|
* on the foreground of `init()`. A production store whose metadata provider
|
|
* had to rebuild paid 641 SECONDS of that wait before `init()` returned, with
|
|
* every other family idle behind it.
|
|
*
|
|
* A provider that reports progress here owns its own rebuild: brainy neither
|
|
* starts one nor waits for it, `init()` returns, the other families serve, and
|
|
* THAT family's doors refuse by name — carrying this progress — until the
|
|
* provider reports itself serving.
|
|
*
|
|
* Every field but `phase` is optional and every field is a MEASUREMENT: a
|
|
* provider reports only what it actually tracks, never an estimate dressed as
|
|
* a fact.
|
|
*/
|
|
export interface ProviderRebuildProgress {
|
|
/** The provider's own name for what it is doing. Quoted verbatim in refusals. */
|
|
phase: string
|
|
/** Units completed so far, if the provider counts them. */
|
|
done?: number
|
|
/** Units expected in total, if the provider knows it. */
|
|
total?: number
|
|
/** Epoch millis when this rebuild started, if the provider tracks it. */
|
|
startedAt?: number
|
|
}
|
|
|
|
/** A provider that can report a rebuild it is running itself. */
|
|
interface MaybeRebuildingProvider {
|
|
rebuildInProgress?: () => ProviderRebuildProgress | null
|
|
}
|
|
|
|
/**
|
|
* @description Ask a provider whether it is rebuilding itself right now.
|
|
* Synchronous, O(1), feature-detected: a provider without the hook reports
|
|
* nothing and is treated exactly as before.
|
|
* @param provider - Any index provider, or `null`/`undefined`.
|
|
* @returns The provider's progress, or `null` when it is not rebuilding (or
|
|
* does not implement the hook).
|
|
*/
|
|
export function assessProviderRebuild(provider: unknown): ProviderRebuildProgress | null {
|
|
const p = provider as MaybeRebuildingProvider | null | undefined
|
|
if (p == null || typeof p.rebuildInProgress !== 'function') return null
|
|
try {
|
|
const progress = p.rebuildInProgress()
|
|
if (!progress || typeof progress.phase !== 'string' || progress.phase.length === 0) {
|
|
return null
|
|
}
|
|
return progress
|
|
} catch {
|
|
// A throwing hook says nothing trustworthy about a rebuild; fall through to
|
|
// the ordinary health verdict rather than inventing one.
|
|
return null
|
|
}
|
|
}
|
|
|
|
/**
|
|
* @description Render a rebuild progress report as one operator-facing clause,
|
|
* for a refusal message. Includes only what the provider actually measured.
|
|
* @param progress - The provider's report.
|
|
* @returns A clause such as `rebuilding ("metadata shadow build", 4,096/14,056, 12s elapsed)`.
|
|
*/
|
|
export function describeRebuildProgress(progress: ProviderRebuildProgress): string {
|
|
const parts: string[] = [`"${progress.phase}"`]
|
|
if (typeof progress.done === 'number' && typeof progress.total === 'number') {
|
|
parts.push(`${progress.done.toLocaleString()}/${progress.total.toLocaleString()}`)
|
|
} else if (typeof progress.done === 'number') {
|
|
parts.push(`${progress.done.toLocaleString()} done`)
|
|
}
|
|
if (typeof progress.startedAt === 'number') {
|
|
parts.push(`${Math.round((Date.now() - progress.startedAt) / 1000)}s elapsed`)
|
|
}
|
|
return `rebuilding (${parts.join(', ')})`
|
|
}
|