fix(persistence): the idle flush trigger debounces under load — deferred to the floor, never dropped, never a flush-per-gap amplifier
An internal report from cross-engine write-path instrumentation: with individual writes slower than the idle window (a contended disk), every inter-write gap looked idle and fired a background full flush — 15 extra flushes during 100 contended adds, amplifying the very pressure that slowed the writes. The law now: an idle fire landing within the spacing floor of the last flush DEFERS to the floor boundary instead of flushing; the floor is min(interval, 10× the CONFIGURED idle window) — scaled to caller intent (a tiny idle window keeps fast idle-driven durability; default 2s/30s config gets a 20s floor), derived from the configured idle, never from a deferred re-arm delay (which would compound into runaway deferral). Deferred is never dropped: a lone write on a then-quiet store still persists at the floor without any further write arriving. Pins: the contended-shape pin (six slow-spaced writes fire ≤2 idle flushes, not one per gap; then still persist) + the original quiet-store idle pin unchanged. Unit 2055/2055.
This commit is contained in:
parent
d1651f986c
commit
a50726e6a8
2 changed files with 58 additions and 2 deletions
|
|
@ -61,6 +61,30 @@ describe('persistence policy — the engine owns its flush cadence', () => {
|
|||
await vi.waitFor(() => expect(flushSpy).toHaveBeenCalled(), { timeout: 5000 })
|
||||
})
|
||||
|
||||
it('idle debounce under load: slow writes never fire a flush per inter-write gap', async () => {
|
||||
// The contended-disk amplifier: writes slower than the idle window make
|
||||
// every gap look idle — without the spacing floor this fired a full
|
||||
// flush per write (measured 15 background flushes in 100 contended adds
|
||||
// on a production-shaped box). The floor (min(interval, 10×idle)) caps
|
||||
// idle fires; deferred, never dropped.
|
||||
const brain = await mk({ flushEveryWrites: 10_000, flushIntervalMs: 600_000, flushOnIdleMs: 50 })
|
||||
const flushSpy = vi.spyOn(brain, 'flush')
|
||||
|
||||
// Six writes spaced wider than the idle window (50ms) with the whole
|
||||
// span inside ~one floor window (500ms): the old behavior fires ~an
|
||||
// idle flush per gap (≈6); the debounced behavior fires at most two
|
||||
// (one immediate boot-window fire + one at the floor boundary).
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await brain.add({ data: `slow ${i}`, type: NounType.Document, metadata: {} })
|
||||
await new Promise((r) => setTimeout(r, 70))
|
||||
}
|
||||
expect(flushSpy.mock.calls.length, 'no flush-per-gap amplifier').toBeLessThanOrEqual(2)
|
||||
|
||||
// Deferred, never dropped: the dirty writes still persist once the
|
||||
// floor elapses on the now-quiet store.
|
||||
await vi.waitFor(() => expect(flushSpy).toHaveBeenCalled(), { timeout: 5000 })
|
||||
})
|
||||
|
||||
it("'manual' policy: the engine NEVER flushes on its own", async () => {
|
||||
const brain = await mk({ policy: 'manual', flushEveryWrites: 2, flushOnIdleMs: 30 })
|
||||
const flushSpy = vi.spyOn(brain, 'flush')
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue