feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names
Three pieces addressing the cold-restart-write incident where a production deployment's first writes after every restart (33-35s each on a cold page cache) blew the op-count-scaled transact budget mid-batch: every write is itself a multi-op transaction, so one cold operation consumed the whole budget, the gate before the next operation tripped, and the write rolled back atomically - refused, retried, and refused again until the page cache warmed passively. - The budget's start-gating contract is now explicit and pinned: it gates STARTING the next operation, never rolling back completed work for elapsed time (the shipped schedule since 8.7.0, now stated in contract JSDoc, guarded by code for operation 0, and enforced by regression tests). The 30s floor is configurable via transactionBudgetFloorMs for stores whose cold operations legitimately run long. - New brain.warm() eagerly loads the vector index, metadata index, and graph adjacency so first operations after a cold restart run at steady-state cost. Returns a WarmReport with an honest per-surface outcome (warmed / probed / unavailable) - never reports a probe as a warm. warmOnOpen: true runs it during init(). New optional provider hook warm() on the vector and graph plugin contracts. - Vector-index transaction op classes renamed from the backend-specific AddToHNSWOperation / RemoveFromHNSWOperation to backend-neutral AddToVectorIndexOperation / RemoveFromVectorIndexOperation, stamping the active backend into the emitted op-name string (AddToVectorIndex(js-hnsw) vs a native provider's own identity) so journals never misdirect an operator toward an index that isn't running.
This commit is contained in:
parent
d08679fc84
commit
55b867c998
13 changed files with 1099 additions and 57 deletions
|
|
@ -37,22 +37,47 @@ const DEFAULT_OPTIONS: Required<TransactionOptions> = {
|
|||
maxRollbackRetries: 3
|
||||
}
|
||||
|
||||
/**
|
||||
* The floor term of {@link transactTimeoutBudget}'s scaling formula when the
|
||||
* caller supplies neither an explicit override nor `config.transactionBudgetFloorMs`.
|
||||
*/
|
||||
const DEFAULT_BUDGET_FLOOR_MS = 30_000
|
||||
|
||||
/**
|
||||
* The apply-phase budget for a batch of `opCount` operations.
|
||||
*
|
||||
* An explicit override wins untouched. Otherwise the budget SCALES with the
|
||||
* batch: `max(30 000 ms, opCount × 2 000 ms)`. The per-op term is calibrated
|
||||
* from field data — bulk imports on network-attached disks measure ~2 s per
|
||||
* operation (each op pays canonical writes + fsync + index maintenance) — so
|
||||
* a flat 30 s budget silently capped honest work at ~15 operations while
|
||||
* looking generous for small batches. Scaling keeps small transacts
|
||||
* fast-failing and gives bulk ones a budget proportional to the work they
|
||||
* actually asked for; a trip still rolls back atomically and throws a
|
||||
* retryable, fully-labeled TransactionTimeoutError.
|
||||
* An explicit `override` wins untouched (a caller-specified deadline for this
|
||||
* one batch). Otherwise the budget SCALES with the batch:
|
||||
* `max(floorMs, opCount × 2 000)`, where `floorMs` defaults to `30 000` and
|
||||
* is overridable via `BrainyConfig.transactionBudgetFloorMs` for the whole
|
||||
* brain. The per-op term is calibrated from field data — bulk imports on
|
||||
* network-attached disks measure ~2 s per operation (each op pays canonical
|
||||
* writes + fsync + index maintenance) — so a flat 30 s budget silently capped
|
||||
* honest work at ~15 operations while looking generous for small batches. A
|
||||
* 16-op batch, for example, scales to `max(30 000, 16 × 2 000) = 32 000`.
|
||||
* Scaling keeps small transacts fast-failing and gives bulk ones a budget
|
||||
* proportional to the work they actually asked for.
|
||||
*
|
||||
* This is a **mid-batch** budget only: it governs whether the transaction's
|
||||
* NEXT operation may start (see {@link Transaction.execute}), never whether
|
||||
* already-completed work is rolled back after the fact. A trip mid-batch
|
||||
* still rolls back every operation applied so far, atomically, and throws a
|
||||
* retryable, fully-labeled TransactionTimeoutError — that zero-loss guarantee
|
||||
* doesn't change; only the point at which the clock stops mattering does (at
|
||||
* the last operation, not one check later).
|
||||
*
|
||||
* @param opCount - Number of operations in the batch.
|
||||
* @param override - A full override for this call; wins over everything else.
|
||||
* @param floorMs - The scaling floor for this call (e.g. `config.transactionBudgetFloorMs`).
|
||||
* Ignored when `override` is set. Defaults to `30 000` when omitted.
|
||||
*/
|
||||
export function transactTimeoutBudget(opCount: number, override?: number): number {
|
||||
export function transactTimeoutBudget(
|
||||
opCount: number,
|
||||
override?: number,
|
||||
floorMs?: number
|
||||
): number {
|
||||
if (override !== undefined) return override
|
||||
return Math.max(30_000, opCount * 2_000)
|
||||
return Math.max(floorMs ?? DEFAULT_BUDGET_FLOOR_MS, opCount * 2_000)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -98,7 +123,20 @@ export class Transaction implements TransactionContext {
|
|||
}
|
||||
|
||||
/**
|
||||
* Execute all operations atomically
|
||||
* Execute all operations atomically.
|
||||
*
|
||||
* Budget semantics: the configured budget gates starting the next
|
||||
* operation; completed work is never rolled back for elapsed time. Elapsed
|
||||
* time is checked ONLY before starting operation `i` for `i ≥ 1` — never
|
||||
* before operation 0 (an operation always gets to start) and never again
|
||||
* after the final operation completes. Concretely: a single-op transaction
|
||||
* can never time out post-hoc — it either runs (and its result stands, no
|
||||
* matter how long it took) or a `TransactionTimeoutError` is thrown before
|
||||
* it starts, which cannot happen since there is no operation before it. A
|
||||
* multi-op transaction whose op `i` overruns the budget stops op `i+1` from
|
||||
* starting, rolls back operations `0..i` (reverse order), and throws
|
||||
* `TransactionTimeoutError` — that zero-loss rollback guarantee for
|
||||
* mid-batch trips is unchanged; only the post-completion check is gone.
|
||||
*/
|
||||
async execute(): Promise<void> {
|
||||
if (this.state !== 'pending') {
|
||||
|
|
@ -128,10 +166,14 @@ export class Transaction implements TransactionContext {
|
|||
// already-applied writes as torn, generation-less state in canonical
|
||||
// storage.
|
||||
for (let i = 0; i < this.operations.length; i++) {
|
||||
// Budget check BEFORE starting the next operation. A trip here throws
|
||||
// into the catch below and rolls back like any other failure — it must
|
||||
// never bypass rollback.
|
||||
if (Date.now() - this.startTime > this.options.timeout) {
|
||||
// Budget gates STARTING the next operation; completed work is never
|
||||
// rolled back for elapsed time. Skipped for i === 0 (operation 0
|
||||
// always gets to start — nothing has run yet to have overrun) and
|
||||
// never re-checked after the final operation completes (there is no
|
||||
// "next operation" left to gate). A trip here throws into the catch
|
||||
// below and rolls back like any other failure — it must never bypass
|
||||
// rollback.
|
||||
if (i > 0 && Date.now() - this.startTime > this.options.timeout) {
|
||||
throw new TransactionTimeoutError(this.options.timeout, i, {
|
||||
elapsedMs: Date.now() - this.startTime,
|
||||
totalOperations: this.operations.length,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue