feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names
Three pieces addressing the cold-restart-write incident where a production deployment's first writes after every restart (33-35s each on a cold page cache) blew the op-count-scaled transact budget mid-batch: every write is itself a multi-op transaction, so one cold operation consumed the whole budget, the gate before the next operation tripped, and the write rolled back atomically - refused, retried, and refused again until the page cache warmed passively. - The budget's start-gating contract is now explicit and pinned: it gates STARTING the next operation, never rolling back completed work for elapsed time (the shipped schedule since 8.7.0, now stated in contract JSDoc, guarded by code for operation 0, and enforced by regression tests). The 30s floor is configurable via transactionBudgetFloorMs for stores whose cold operations legitimately run long. - New brain.warm() eagerly loads the vector index, metadata index, and graph adjacency so first operations after a cold restart run at steady-state cost. Returns a WarmReport with an honest per-surface outcome (warmed / probed / unavailable) - never reports a probe as a warm. warmOnOpen: true runs it during init(). New optional provider hook warm() on the vector and graph plugin contracts. - Vector-index transaction op classes renamed from the backend-specific AddToHNSWOperation / RemoveFromHNSWOperation to backend-neutral AddToVectorIndexOperation / RemoveFromVectorIndexOperation, stamping the active backend into the emitted op-name string (AddToVectorIndex(js-hnsw) vs a native provider's own identity) so journals never misdirect an operator toward an index that isn't running.
This commit is contained in:
parent
d08679fc84
commit
55b867c998
13 changed files with 1099 additions and 57 deletions
|
|
@ -423,6 +423,46 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
prodLog.debug('✅ Type-aware cache warming completed')
|
||||
}
|
||||
|
||||
/**
|
||||
* Full hydration — the {@link Brainy.warm} readiness seam for the metadata
|
||||
* index. Unlike {@link warmCache} / {@link warmCacheForTopTypes} (which
|
||||
* warm only a heuristic subset: common fields plus the top-N types' top
|
||||
* fields), this loads EVERY field's sparse index the field registry knows
|
||||
* about — a real read through {@link loadSparseIndex} into the unified
|
||||
* cache for each field, not a stat/existence check. Idempotent: an
|
||||
* already-cached field's `loadSparseIndex` call is a cheap cache hit.
|
||||
*
|
||||
* Re-reads the field registry first when `fieldIndexes` is empty (a warm()
|
||||
* call issued before `init()` populated it would otherwise hydrate
|
||||
* nothing), then loads every discovered field in parallel.
|
||||
*/
|
||||
async hydrateAll(): Promise<void> {
|
||||
if (this.fieldIndexes.size === 0) {
|
||||
await this.loadFieldRegistry()
|
||||
}
|
||||
|
||||
const fields = Array.from(this.fieldIndexes.keys())
|
||||
if (fields.length === 0) {
|
||||
prodLog.debug('[MetadataIndex] hydrateAll: no persisted fields to hydrate')
|
||||
return
|
||||
}
|
||||
|
||||
prodLog.debug(`[MetadataIndex] hydrateAll: loading ${fields.length} field(s) — ${fields.join(', ')}`)
|
||||
|
||||
await Promise.all(
|
||||
fields.map(async field => {
|
||||
try {
|
||||
await this.loadSparseIndex(field)
|
||||
} catch (error) {
|
||||
// A single field's load failure doesn't abort the rest of the
|
||||
// hydration — warm() is a best-effort readiness step, never a
|
||||
// correctness gate (queries still demand-load on miss).
|
||||
prodLog.debug(`[MetadataIndex] hydrateAll: field '${field}' failed to load:`, error)
|
||||
}
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire an in-memory lock for coordinating concurrent metadata index writes
|
||||
* Uses in-memory locks since MetadataIndexManager doesn't have direct file system access
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue