fix: cold-open no longer re-derives durable indexes — complete the readiness contract for all three providers
A production deployment measured ~48 seconds on EVERY reopen of an 11k-entity brain. Root cause: brainy's rebuild gate decided from in-memory size()/count, which read 0 for a durable-but-not-resident index, so it re-read every entity file to rebuild from scratch. At GA we gave only the GRAPH provider a readiness contract (init() eager cold-load + isReady() honest signal) so it would never eat that spurious rebuild; the vector and metadata providers never got it, and brainy never even eager-inited the vector provider. Complete the contract symmetrically: - plugin.ts: VectorIndexProvider gains optional init()+isReady(); MetadataIndexProvider gains isReady() — mirroring GraphIndexProvider. Additive and optional; a provider that exposes nothing keeps today's behavior. - brainy.ts: eager-init every provider that exposes init() (after metadata init() so the id-mapper is hydrated first), then decide per leg in precedence order — migrating (skip) -> epoch drift (rebuild) -> isReady() -> a per-leg empty fallback. The old instant fast-path keyed off this.index.size()>0, a dishonest proxy that skipped the metadata/graph checks whenever the vector was warm and never fired on a real cold process anyway; removed. The per-leg fallbacks differ because "empty" means different things: the JS vector's rebuild() IS its load, so size()===0 correctly triggers it; the id-mapper backs metadata, so totalEntries===0 (past the empty-store return) is a real load failure; but entities do not imply edges, so a graph size()===0 is a valid empty state, not a load failure. - The JS graph now COLD-LOADS its durable LSM instead of re-deriving from a full canonical verb scan on every boot (baseStorage._initializeGraphIndex loads the persisted SSTables via a new GraphAdjacencyIndex.init(); it self-heals from canonical only when the durable state is genuinely missing). This removes an O(E)-per-open cost every filesystem consumer paid. - LSMTree.loadManifest loads its SSTables BEFORE publishing the relationship count, and resets to an honest-empty state on load failure — a tree can no longer claim persisted relationships while holding none (the silent-empty cold-load class the query-time guards exist to prevent). Verified end-to-end against a built brain: a warm reopen (with edges and edgeless) reloads only the JS vector; the graph and metadata cold-load with no rebuild, and queries return correct results. New tests in cold-open-rebuild-gate.test.ts pin the contract (isReady() defers, self-heal still fires); migration-deference updated to drive size-based deference through the vector, the leg where empty->rebuild remains correct. Pairs with the native provider's isReady()/init() implementation — brainy's gate defers only to a signal the provider exposes.
This commit is contained in:
parent
4fde94bc2f
commit
61c247c923
8 changed files with 425 additions and 57 deletions
|
|
@ -125,6 +125,20 @@ export interface MetadataIndexProvider {
|
|||
flush(): Promise<void>
|
||||
rebuild(): Promise<void>
|
||||
|
||||
/**
|
||||
* @description OPTIONAL honest durability signal (readiness contract,
|
||||
* mirrors `isReady?()` on the graph and vector providers). `true` ⇔ the
|
||||
* persisted field postings are loaded (or cheaply demand-loadable) and
|
||||
* consistent with what the provider last persisted — a rebuild from the
|
||||
* canonical records would be redundant. When exposed, the rebuild gate
|
||||
* defers to this signal INSTEAD of the `getStats().totalEntries === 0`
|
||||
* heuristic (a durable provider may legitimately report 0 resident entries
|
||||
* on a cold open while its postings sit loadable on disk). Absent → the
|
||||
* gate keeps the count heuristic. Never return `true` when the durable
|
||||
* state failed to load.
|
||||
*/
|
||||
isReady?(): boolean
|
||||
|
||||
/**
|
||||
* @description OPTIONAL. A native provider returns true from the moment its
|
||||
* `init()` detects a large epoch-drift until its background
|
||||
|
|
@ -902,6 +916,33 @@ export interface VectorIndexProvider {
|
|||
flush(): Promise<number>
|
||||
getPersistMode(): 'immediate' | 'deferred'
|
||||
|
||||
/**
|
||||
* @description OPTIONAL eager cold-load (readiness contract, mirrors
|
||||
* {@link GraphIndexProvider.init}). Called once during brain init — AFTER the
|
||||
* metadata provider's `init()` (the id-mapper is hydrated first, so a
|
||||
* provider whose vector slots resolve through interned ints reads a complete
|
||||
* mapping) and BEFORE the rebuild gate — so a durable provider loads (or
|
||||
* verifies it can demand-load) its persisted index and reports
|
||||
* `isReady() === true` at the gate instead of eating a spurious
|
||||
* rebuild-from-canonical on every open. The built-in JS index omits it:
|
||||
* `rebuild()` IS its load path.
|
||||
*/
|
||||
init?(): Promise<void>
|
||||
|
||||
/**
|
||||
* @description OPTIONAL honest durability signal (readiness contract,
|
||||
* mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted
|
||||
* derived index is loaded (or cheaply demand-loadable) and consistent with
|
||||
* what the provider last persisted — a rebuild from the canonical records
|
||||
* would be redundant work. When exposed, the rebuild gate defers to this
|
||||
* signal INSTEAD of the `size() === 0` heuristic (an mmap/disk-native index
|
||||
* may legitimately report 0 resident entries while fully durable). Absent →
|
||||
* the gate keeps the size heuristic. Never return `true` when the durable
|
||||
* state failed to load — that converts a recoverable rebuild into silent
|
||||
* empty results.
|
||||
*/
|
||||
isReady?(): boolean
|
||||
|
||||
/**
|
||||
* @description OPTIONAL. A native provider returns true from the moment its
|
||||
* `init()` detects a large epoch-drift until its background
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue