feat(persistence): the engine owns its flush cadence — callers never call flush() in hot paths again
A4 of the service-class pair (SELF-ENGINE-LIFECYCLE-SPRINT, David-directed: 'why do we need manual flushes at all?'). The production disease: 829 caller-scheduled per-write flushes convoying into 45-66s write walls — cadence hand-rolled a layer above the only layer that can see dirty-node counts and IO pressure. - BrainyConfig.persistence: policy 'auto' (DEFAULT) | 'manual', with flushEveryWrites (512) / flushIntervalMs (30s) / flushOnIdleMs (2s) triggers. Auto = the engine kicks ONE single-flight BACKGROUND flush at a threshold or when the store goes quiet; write acks NEVER await it (a hung flush cannot block a write — pinned); a failed background flush is LOUD and re-arms the trigger. 'manual' restores caller-owned cadence. - Triggers wired at both write chokepoints (single-op post-commit + transact post-commit); idle timer unref'd; close() tears the timer down and drains the flight before its own final flush. - RECOVERY SEMANTICS documented on the config: canonical records are durable per-write regardless of policy — a crash between background flushes loses derived state only, which converges at next open (epoch machinery + the new incremental aggregation catch-up), bounded by the un-flushed window. Never data loss. Pins: write-count trigger fires one background flush with zero caller calls · idle trigger · manual never self-flushes · THE ACK LAW (writes acknowledge under a never-resolving flush). Gates: unit 1917/1917 · integration 760 · conformance 27/27 — green WITH auto as the default.
This commit is contained in:
parent
1dc861d299
commit
3236a01bef
3 changed files with 214 additions and 1 deletions
|
|
@ -2028,6 +2028,39 @@ export interface BrainyConfig {
|
|||
*/
|
||||
force?: boolean
|
||||
|
||||
/**
|
||||
* THE ENGINE OWNS ITS FLUSH CADENCE (the persistence policy —
|
||||
* SELF-ENGINE-LIFECYCLE-SPRINT, David-directed: "why do we need manual
|
||||
* flushes at all?"). Under `'auto'` (the DEFAULT) the engine schedules
|
||||
* single-flight background flushes itself — triggered by write count,
|
||||
* elapsed time, and idle — so callers NEVER call `flush()` in a hot path
|
||||
* (a production consumer's 829 per-write flushes convoyed into 45–66s
|
||||
* write walls; the cadence belongs to the layer that can see dirty-node
|
||||
* counts and IO pressure). `flush()` remains public as an awaitable
|
||||
* durability BARRIER for the rare "must be on disk before I proceed"
|
||||
* moment — calling it is never wrong, just no longer necessary.
|
||||
*
|
||||
* RECOVERY SEMANTICS (the documented promise): canonical records are
|
||||
* durable per-write, independent of this policy — a crash between
|
||||
* background flushes loses NO data. What a flush persists is DERIVED
|
||||
* state (index postings, deferred HNSW nodes, counters, aggregation
|
||||
* stamps); after a crash, derived state converges at the next open from
|
||||
* canonical records (epoch machinery + incremental aggregation catch-up),
|
||||
* paying a bounded catch-up cost proportional to the un-flushed window —
|
||||
* never data loss.
|
||||
*
|
||||
* `'manual'` restores the pre-9.1 behavior: the engine never flushes on
|
||||
* its own (except at `close()`); the caller owns the cadence.
|
||||
*/
|
||||
persistence?: {
|
||||
policy?: 'auto' | 'manual'
|
||||
/** Background flush after this many committed writes (default 512). */
|
||||
flushEveryWrites?: number
|
||||
/** Background flush when this much time has passed since the last flush, checked at write time (default 30_000). */
|
||||
flushIntervalMs?: number
|
||||
/** Background flush after the store goes quiet for this long with dirty state (default 2_000). */
|
||||
flushOnIdleMs?: number
|
||||
}
|
||||
}
|
||||
|
||||
// ============= Neural API Types =============
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue