feat(persistence): the engine owns its flush cadence — callers never call flush() in hot paths again
Some checks failed
CI / Node 22 (push) Has been cancelled
CI / Node 24 (push) Has been cancelled
CI / Bun (latest) (push) Has been cancelled

A4 of the service-class pair (SELF-ENGINE-LIFECYCLE-SPRINT, David-directed:
'why do we need manual flushes at all?'). The production disease: 829
caller-scheduled per-write flushes convoying into 45-66s write walls —
cadence hand-rolled a layer above the only layer that can see dirty-node
counts and IO pressure.

- BrainyConfig.persistence: policy 'auto' (DEFAULT) | 'manual', with
  flushEveryWrites (512) / flushIntervalMs (30s) / flushOnIdleMs (2s)
  triggers. Auto = the engine kicks ONE single-flight BACKGROUND flush at
  a threshold or when the store goes quiet; write acks NEVER await it (a
  hung flush cannot block a write — pinned); a failed background flush is
  LOUD and re-arms the trigger. 'manual' restores caller-owned cadence.
- Triggers wired at both write chokepoints (single-op post-commit +
  transact post-commit); idle timer unref'd; close() tears the timer down
  and drains the flight before its own final flush.
- RECOVERY SEMANTICS documented on the config: canonical records are
  durable per-write regardless of policy — a crash between background
  flushes loses derived state only, which converges at next open (epoch
  machinery + the new incremental aggregation catch-up), bounded by the
  un-flushed window. Never data loss.

Pins: write-count trigger fires one background flush with zero caller
calls · idle trigger · manual never self-flushes · THE ACK LAW (writes
acknowledge under a never-resolving flush). Gates: unit 1917/1917 ·
integration 760 · conformance 27/27 — green WITH auto as the default.
This commit is contained in:
David Snelling 2026-08-05 16:00:39 -07:00
parent 1dc861d299
commit 3236a01bef
3 changed files with 214 additions and 1 deletions

View file

@ -0,0 +1,97 @@
/**
* @module tests/unit/brainy/persistence-policy
* @description THE ENGINE-OWNED FLUSH CADENCE pins (A4,
* SELF-ENGINE-LIFECYCLE-SPRINT, David-directed: callers NEVER call flush()
* in hot paths). The production disease: 829 caller-scheduled per-write
* flushes convoying into 4566 second write walls cadence hand-rolled a
* layer above the only layer that can see dirty state and IO pressure.
*
* Pinned here: (1) the write-count trigger fires a BACKGROUND flush without
* any caller flush(); (2) the idle trigger; (3) `'manual'` restores
* caller-owned cadence exactly; (4) THE ACK LAW a write acknowledges
* without awaiting any background flush, even one that never resolves.
*/
import { describe, it, expect, afterEach, vi } from 'vitest'
import { Brainy } from '../../../src/index.js'
import { NounType } from '../../../src/types/graphTypes.js'
const brains: Brainy[] = []
async function mk(persistence?: {
policy?: 'auto' | 'manual'
flushEveryWrites?: number
flushIntervalMs?: number
flushOnIdleMs?: number
}): Promise<Brainy> {
const b = new Brainy({
storage: { type: 'memory' },
requireSubtype: false,
...(persistence && { persistence })
})
await b.init()
brains.push(b)
return b
}
afterEach(async () => {
for (const b of brains.splice(0)) await b.close().catch(() => {})
vi.restoreAllMocks()
})
describe('persistence policy — the engine owns its flush cadence', () => {
it('write-count trigger: N committed writes fire ONE background flush, no caller flush()', async () => {
const brain = await mk({ flushEveryWrites: 5, flushOnIdleMs: 60_000, flushIntervalMs: 600_000 })
const flushSpy = vi.spyOn(brain, 'flush')
for (let i = 0; i < 5; i++) {
await brain.add({ data: `w${i}`, type: NounType.Document, metadata: { i } })
}
await vi.waitFor(() => expect(flushSpy).toHaveBeenCalled(), { timeout: 5000 })
// Single-flight: the threshold crossing kicks exactly one.
expect(flushSpy.mock.calls.length).toBe(1)
})
it('idle trigger: a quiet store with dirty writes flushes itself', async () => {
const brain = await mk({ flushEveryWrites: 10_000, flushIntervalMs: 600_000, flushOnIdleMs: 60 })
const flushSpy = vi.spyOn(brain, 'flush')
await brain.add({ data: 'lone write', type: NounType.Document, metadata: {} })
await vi.waitFor(() => expect(flushSpy).toHaveBeenCalled(), { timeout: 5000 })
})
it("'manual' policy: the engine NEVER flushes on its own", async () => {
const brain = await mk({ policy: 'manual', flushEveryWrites: 2, flushOnIdleMs: 30 })
const flushSpy = vi.spyOn(brain, 'flush')
for (let i = 0; i < 6; i++) {
await brain.add({ data: `m${i}`, type: NounType.Document, metadata: { i } })
}
await new Promise((r) => setTimeout(r, 150))
expect(flushSpy).not.toHaveBeenCalled()
})
it('THE ACK LAW: writes acknowledge without awaiting the background flush — even a hung one', async () => {
const brain = await mk({ flushEveryWrites: 2, flushOnIdleMs: 60_000, flushIntervalMs: 600_000 })
// A flush that NEVER resolves: if any write ack awaited it, the test
// would time out. (The engine's background flight must be fire-and-log.)
vi.spyOn(brain, 'flush').mockImplementation(() => new Promise<void>(() => {}))
for (let i = 0; i < 6; i++) {
const id = await brain.add({ data: `a${i}`, type: NounType.Document, metadata: { i } })
expect(id).toBeTruthy()
}
// All six writes acked while the "flush" hangs forever.
const rows = await brain.find({ type: NounType.Document, limit: 10 })
expect(rows.length).toBe(6)
// Un-hang before afterEach close(): restore the method AND drop the
// never-resolving in-flight promise (close() awaits the flight — with a
// real flush that is correct; here it is the test's own artifact).
vi.restoreAllMocks()
;(brain as unknown as { _persistBackgroundFlight: Promise<void> | null })._persistBackgroundFlight =
null
})
})