feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names

Three pieces addressing the cold-restart-write incident where a production
deployment's first writes after every restart (33-35s each on a cold page
cache) blew the op-count-scaled transact budget mid-batch: every write is
itself a multi-op transaction, so one cold operation consumed the whole
budget, the gate before the next operation tripped, and the write rolled
back atomically - refused, retried, and refused again until the page cache
warmed passively.

- The budget's start-gating contract is now explicit and pinned: it gates
  STARTING the next operation, never rolling back completed work for
  elapsed time (the shipped schedule since 8.7.0, now stated in contract
  JSDoc, guarded by code for operation 0, and enforced by regression
  tests). The 30s floor is configurable via transactionBudgetFloorMs for
  stores whose cold operations legitimately run long.
- New brain.warm() eagerly loads the vector index, metadata index, and
  graph adjacency so first operations after a cold restart run at
  steady-state cost. Returns a WarmReport with an honest per-surface
  outcome (warmed / probed / unavailable) - never reports a probe as a
  warm. warmOnOpen: true runs it during init(). New optional provider hook
  warm() on the vector and graph plugin contracts.
- Vector-index transaction op classes renamed from the backend-specific
  AddToHNSWOperation / RemoveFromHNSWOperation to backend-neutral
  AddToVectorIndexOperation / RemoveFromVectorIndexOperation, stamping the
  active backend into the emitted op-name string (AddToVectorIndex(js-hnsw)
  vs a native provider's own identity) so journals never misdirect an
  operator toward an index that isn't running.
This commit is contained in:
David Snelling 2026-07-22 16:42:26 -07:00
parent d08679fc84
commit 55b867c998
13 changed files with 1099 additions and 57 deletions

View file

@ -31,6 +31,62 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
---
## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
From a production deployment's cold-restart incident: the FIRST writes after every
restart on a large brain measured 3335s each (page cache cold) against the transact
apply budget — Brainy's own op-count-scaled budget, `max(30s, opCount × 2s)`, e.g.
32,000ms for a 16-op batch. There is no external deadline in this story, and no
post-completion veto either: every write is itself a multi-operation transaction (a
single `add()` applies several operations — canonical writes, the vector-index insert,
the metadata-index update), so ONE cold operation that legitimately runs ~33s consumes
the whole budget, the gate before the NEXT operation trips, and the write rolls back
atomically (zero loss, by design) — refused, retried, refused again, until the page
cache warms passively (~30 minutes). The cure is not weaker atomicity; it is the two
new knobs below — a budget floor sized for cold stores, and a warm contract that pays
demand-load cost OFF the transaction path.
- **The budget's start-gating contract is now explicit, documented, and pinned by
regression tests.** The budget gates STARTING the next operation — completed work is
never rolled back for elapsed time — and a transaction's first operation now
unconditionally starts by code, not merely because elapsed time happens to be ~0 when
it is checked. This has been the shipped schedule since 8.7.0 (no behavior change for
existing integrations); it is now stated in `Transaction.execute()`'s contract JSDoc
and enforced by tests so it cannot silently regress. Mid-batch atomicity is unchanged:
a trip before operation `i+1` still rolls back `0..i` and throws a retryable
`TransactionTimeoutError`.
- **The budget's 30s floor is now configurable**: `new Brainy({ transactionBudgetFloorMs })`
raises (or lowers) the floor of `max(transactionBudgetFloorMs, opCount × 2000)` for every
internal transact batch. Useful for a store whose cold writes legitimately run past 30s
per operation, so a bulk batch gets a proportionally larger runway instead of tripping
mid-batch on cold-cache latency.
- **New: `brain.warm()`** — eagerly loads/faults-in the vector index, metadata index, and
graph adjacency so the first real operation after a cold restart runs at steady-state
cost instead of paying demand-load latency on the critical path. Returns a `WarmReport`
with one honest outcome per surface — never conflate the first two:
- `'warmed'` — the surface's own provider `warm()` hook ran, or a full-hydration seam
loaded every shard/field/segment from storage. Steady-state cost is paid.
- `'probed'` — no `warm()` hook was available, so a best-effort read (one `search()` call
for the vector index) faulted in *some* backing storage as a side effect — real work,
but never reported as `'warmed'`.
- `'unavailable'` — nothing ran (no hook, no hydration seam, or nothing to probe).
- **New config: `warmOnOpen: true`** makes `init()` await `brain.warm()` before it resolves
— a deliberate blocking trade-off: startup takes longer, the first request doesn't.
Default `false` (unchanged lazy behavior).
- **New optional provider hook: `warm?(): Promise<void>`** on the vector and graph
acceleration provider contracts (`src/plugin.ts`) — a native provider can implement it to
eagerly pretouch its own backing storage (e.g. mmap pretouch); absence means brainy falls
back to the probe/hydration behavior above.
- **Transaction op-name strings changed in journals/timings**: the vector-index
transaction operations were renamed from `AddToHNSW`/`RemoveFromHNSW` to backend-neutral
`AddToVectorIndex(...)`/`RemoveFromVectorIndex(...)` — the old names hard-coded an
algorithm that may not be the one actually running (a non-HNSW native vector provider
emitting `"RemoveFromHNSW"` has sent an operator hunting an index that doesn't exist).
The parenthesized suffix names the ACTIVE backend: `js-hnsw` for the built-in engine, or
the native provider's own identity when it self-identifies. **If you parse these op-name
strings** (log processors, journal tooling), update your matcher from
`AddToHNSW`/`RemoveFromHNSW` to `AddToVectorIndex(`/`RemoveFromVectorIndex(`.
## v8.9.0 — 2026-07-19 (flush is durability-only: history maintenance moves to close())
The write path stops paying maintenance costs — the last structural piece of the