fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface

A production deployment's warm report showed metadata: 'unavailable' under a
native metadata provider. brain.warm()'s metadata leg only duck-typed the
built-in JS manager's hydrateAll() method, which a native provider has no
reason to implement.

- MetadataIndexProvider (src/plugin.ts) gains an optional warm?(): Promise<void>
  hook, mirroring the existing vector and graph provider hooks. brain.warm()
  now checks the active provider's own warm() FIRST, falls back to the JS
  manager's hydrateAll() when absent, and reports 'unavailable' only when
  neither exists -- never init() as a stand-in, since a native provider's
  init() may be a cheap verify rather than a real warm.
- Tests (tests/unit/brainy/warm.test.ts): a live provider instance shaped to
  have warm() reports 'warmed' and the hook called with no hydrateAll
  fallback; shaped to have neither hook reports 'unavailable' (pins the
  honest branch); the unmodified built-in JS manager still reports 'warmed'
  via hydrateAll(), unchanged.

Additive scope agreed mid-flight with the native-provider team: a
maintenance-debt observability seam so an operator sees a grind coming
instead of discovering it as a CPU storm.

- New optional maintenanceDebt?(): Promise<ProviderMaintenanceDebt> hook on
  all three provider contracts (vector, metadata, graph -- the same three
  warm?() lives on). ProviderMaintenanceDebt is fields-all-optional: a
  provider reports only what it truly measures (pendingBytes, pendingItems,
  lastPassCompletedAt, lastPassOutcome, converging), never an estimate
  dressed as fact.
- New public brain.maintenanceDebt(): a pure passthrough -- for each surface
  it calls only the active provider's own hook and reports the payload
  verbatim, or 'unavailable' when absent. No thresholds, no polling, no
  JS-side estimation; the provider owns the numbers, the operator owns the
  policy.
- ProviderMaintenanceDebt, MaintenanceDebtReport, and MaintenanceDebtOutcome
  are exported from the package root.
- Tests (tests/unit/brainy/maintenance-debt.test.ts): hook present reports
  'reported' with the exact payload passed through; hook absent reports
  'unavailable' on every surface; mixed surfaces resolve independently of
  each other.

RELEASES.md gains the 8.10.1 entry covering both fixes above and this
feature, including the no-hot-retry contract from the prior commit.
This commit is contained in:
David Snelling 2026-07-24 16:02:01 -07:00
parent 003e2a74ea
commit 5b2cbf74e5
6 changed files with 471 additions and 6 deletions

View file

@ -31,6 +31,68 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
---
## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers)
From a production incident: a native-provider op ground 38-40s inside a transaction,
blew the ~32s apply-phase budget, was rolled back (zero loss, by design), and a
downstream pipeline hot-retried the identical operation into a 6-minute, 100%-CPU
storm. Investigation confirmed Brainy itself never auto-retries a timed-out
transaction — the storm was entirely the consumer's own retry loop, driven by a
"retryable" doc-prose claim with no machine-readable contract to branch on. This
release closes that contract gap and, separately, fixes a real `warm()` reporting gap
surfaced by the same investigation.
- **`TransactionTimeoutError` is now a machine-readable no-hot-retry contract.** Two
new typed, always-`true` fields replace prose-only guidance:
- `retryable: true` — the operation MAY succeed on a later attempt, once the
underlying slowness resolves or the budget is deliberately raised
(`transactionBudgetFloorMs`, or a batch's own `timeoutMs` override).
- `hotRetryUnsafe: true` — an immediate, identical retry re-pays the FULL cost of
the work that just timed out (it does not resume partway) and can cascade into
exactly the CPU storm above. **Never loop on this error.** The documented pattern
is a latch, not a retry loop:
```
on TransactionTimeoutError:
record { at: Date.now(), error }
rethrow loudly to your own caller
hold a cooldown window before any re-attempt
clear the latch only on a subsequent success
```
- `context` (unchanged, now fully documented) carries the backoff inputs:
`timeoutMs`, `operationIndex`, `elapsedMs`, `totalOperations`, `operationName`.
- Every "retryable" doc-prose site referencing this error (`transact()`'s
`timeoutMs` option, `transactionBudgetFloorMs`, `Transaction.execute()`) now
points at these fields instead of bare prose.
- Regression-pinned: the engine never internally re-drives a timed-out operation
(verified via an execution counter through both the single-op write path and
`add()`'s upsert-race retry loop), so this has always been true — it is now
provable and typed.
- **Dead code removed**: `TransactionManager.executeTransactionWithResult()` had zero
callers in this codebase and is deleted.
- **`brain.warm()`'s metadata surface now routes through the ACTIVE provider.** A
production deployment's warm report showed `metadata: 'unavailable'` under a native
metadata provider — the previous logic only duck-typed the built-in JS manager's
`hydrateAll()` method, which a native provider has no reason to implement. The
metadata provider contract (`MetadataIndexProvider`, `src/plugin.ts`) gains an
optional `warm?(): Promise<void>` hook, mirroring the existing vector and graph
provider hooks. `brain.warm()` now checks the active provider's own `warm()` FIRST,
falls back to the JS manager's `hydrateAll()` when absent, and only reports
`'unavailable'` when neither exists — never `init()` as a stand-in, since a native
provider's `init()` may be a cheap verify rather than a real warm. A native
provider lights this surface up the same way `@soulcraft/cor` already lights the
vector and graph surfaces: implement `warm()` on its metadata provider.
- **New: `brain.maintenanceDebt()`** — the observability seam so an operator sees a
provider's outstanding background maintenance work (pending bytes/items, last pass
outcome, whether it's converging) BEFORE it grinds into the kind of budget-busting
op this release's timeout contract exists for, instead of discovering it as a CPU
storm. It is a pure passthrough: brainy applies no thresholds, no polling, and no
estimation — it calls each active provider's own optional `maintenanceDebt?()` hook
(vector, metadata, graph — the same three contracts `warm?()` lives on) and reports
the payload verbatim, or `'unavailable'` when a surface's provider doesn't track
debt. Useful as a pre-warm/post-warm check or a boot gate. `@soulcraft/cor` does not
yet implement the hook as of this release — expect it on cor's next release; until
then all three surfaces honestly report `'unavailable'`.
## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
From a production deployment's cold-restart incident: the FIRST writes after every