fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface

A production deployment's warm report showed metadata: 'unavailable' under a
native metadata provider. brain.warm()'s metadata leg only duck-typed the
built-in JS manager's hydrateAll() method, which a native provider has no
reason to implement.

- MetadataIndexProvider (src/plugin.ts) gains an optional warm?(): Promise<void>
  hook, mirroring the existing vector and graph provider hooks. brain.warm()
  now checks the active provider's own warm() FIRST, falls back to the JS
  manager's hydrateAll() when absent, and reports 'unavailable' only when
  neither exists -- never init() as a stand-in, since a native provider's
  init() may be a cheap verify rather than a real warm.
- Tests (tests/unit/brainy/warm.test.ts): a live provider instance shaped to
  have warm() reports 'warmed' and the hook called with no hydrateAll
  fallback; shaped to have neither hook reports 'unavailable' (pins the
  honest branch); the unmodified built-in JS manager still reports 'warmed'
  via hydrateAll(), unchanged.

Additive scope agreed mid-flight with the native-provider team: a
maintenance-debt observability seam so an operator sees a grind coming
instead of discovering it as a CPU storm.

- New optional maintenanceDebt?(): Promise<ProviderMaintenanceDebt> hook on
  all three provider contracts (vector, metadata, graph -- the same three
  warm?() lives on). ProviderMaintenanceDebt is fields-all-optional: a
  provider reports only what it truly measures (pendingBytes, pendingItems,
  lastPassCompletedAt, lastPassOutcome, converging), never an estimate
  dressed as fact.
- New public brain.maintenanceDebt(): a pure passthrough -- for each surface
  it calls only the active provider's own hook and reports the payload
  verbatim, or 'unavailable' when absent. No thresholds, no polling, no
  JS-side estimation; the provider owns the numbers, the operator owns the
  policy.
- ProviderMaintenanceDebt, MaintenanceDebtReport, and MaintenanceDebtOutcome
  are exported from the package root.
- Tests (tests/unit/brainy/maintenance-debt.test.ts): hook present reports
  'reported' with the exact payload passed through; hook absent reports
  'unavailable' on every surface; mixed surfaces resolve independently of
  each other.

RELEASES.md gains the 8.10.1 entry covering both fixes above and this
feature, including the no-hot-retry contract from the prior commit.
This commit is contained in:
David Snelling 2026-07-24 16:02:01 -07:00
parent 003e2a74ea
commit 5b2cbf74e5
6 changed files with 471 additions and 6 deletions

View file

@ -171,6 +171,37 @@ export interface ProviderInvariantReport {
durationMs: number
}
/**
* @description A provider's self-report of its own outstanding background
* maintenance work (compaction, deferred writes, a build-newverifyswap in
* flight, etc.) the observability seam so an operator sees a grind coming
* (rising pending bytes/items, a stalled pass) instead of discovering it as a
* CPU storm or a timeout under transaction budget pressure. Every field is
* OPTIONAL and every field is a MEASUREMENT: a provider reports ONLY what it
* actually tracks, never an estimate dressed up as a fact. Absence of the
* {@link VectorIndexProvider.maintenanceDebt} /
* {@link GraphIndexProvider.maintenanceDebt} /
* {@link MetadataIndexProvider.maintenanceDebt} hook itself means the
* provider does not track debt at all brainy reports that surface
* `'unavailable'` rather than inventing zeros. Brainy performs NO threshold
* checks, NO polling, and NO JS-side estimation over this payload it is a
* pure passthrough via {@link Brainy.maintenanceDebt}; the provider owns the
* numbers and the operator owns the policy (what threshold matters, what to
* do about it).
*/
export interface ProviderMaintenanceDebt {
/** Bytes of outstanding/unmerged work, if the provider measures it (e.g. unflushed writes, unmerged segments). */
pendingBytes?: number
/** Count of outstanding items (records, segments, nodes) awaiting the provider's background pass. */
pendingItems?: number
/** Epoch millis when the provider's last maintenance pass finished, if it tracks one. */
lastPassCompletedAt?: number
/** How the last pass ended, if the provider tracks pass outcomes. */
lastPassOutcome?: 'completed' | 'partial' | 'failed'
/** `true` if the provider's own measurements show debt trending down (making progress); `false` if flat or growing; omitted if the provider can't tell. */
converging?: boolean
}
/**
* The `'metadataIndex'` provider a drop-in for `MetadataIndexManager`.
* Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
@ -181,6 +212,34 @@ export interface MetadataIndexProvider {
flush(): Promise<void>
rebuild(): Promise<void>
/**
* @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap
* pretouch, full sparse-index hydration) so first queries run at
* steady-state cost. Optional; absence means the provider demand-loads.
* Mirrors {@link GraphIndexProvider.warm} / the vector provider's `warm?()`
* (`src/plugin.ts` VectorIndexProvider). Distinct from `init()`: `init` is
* required and runs once automatically during brain startup; `warm` is a
* separate, explicit step a caller opts into via `brain.warm()` (or
* `warmOnOpen`) to pre-pay demand-load cost `init` left lazy. Idempotent
* calling it more than once must be safe and cheap on a brain that is
* already warm. A provider that already loads everything eagerly in
* `init()` may implement `warm` as a no-op or omit it `brain.warm()`
* falls back to the built-in JS manager's `hydrateAll()` duck-type when
* absent, and to an honest `'unavailable'` when neither exists.
*/
warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. unmerged postings) BEFORE it grinds a transaction
* into a budget-busting op. Absence means this provider does not track
* debt; `brain.maintenanceDebt()` reports this surface `'unavailable'`
* rather than guessing. See {@link ProviderMaintenanceDebt} for the
* measure-only-what-you-track contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/**
* @description OPTIONAL honest durability signal (readiness contract,
* mirrors `isReady?()` on the graph and vector providers). `true` the
@ -395,6 +454,18 @@ export interface GraphIndexProvider {
*/
warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. a build-newverifyswap in flight, unmerged
* adjacency segments) BEFORE it grinds a transaction into a
* budget-busting op. Absence means this provider does not track debt;
* `brain.maintenanceDebt()` reports this surface `'unavailable'` rather
* than guessing. See {@link ProviderMaintenanceDebt} for the
* measure-only-what-you-track contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/**
* @description OPTIONAL. A native provider returns true from the moment its
* `init()` detects a large epoch-drift until its background
@ -1057,6 +1128,18 @@ export interface VectorIndexProvider {
*/
warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. unflushed writes, a pending rebuild) BEFORE it
* grinds a transaction into a budget-busting op. Absence means this
* provider does not track debt; `brain.maintenanceDebt()` reports this
* surface `'unavailable'` rather than guessing. See
* {@link ProviderMaintenanceDebt} for the measure-only-what-you-track
* contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/**
* @description OPTIONAL honest durability signal (readiness contract,
* mirrors {@link GraphIndexProvider.isReady}). `true` the persisted