fix(flush): the gate settles its waiter from the machine, never from a chain
Some checks failed
CI / Node 22 (push) Successful in 12m26s
CI / Node 24 (push) Successful in 12m19s
CI / Bun (latest) (push) Successful in 12m28s
CI / Integration + conformance (Node 22) (push) Failing after 17m12s

The single-flight gate queued its follow-up as
`leader.catch().then(() => this.flush())`. That waiter is settled ONLY by
resolving the very promise the leader is being awaited through, so the moment
anything inside a flush body awaits flush(), the promise graph closes on itself
and nobody resolves — an unbounded hang, not a slow flush, presenting exactly
like a bulk write timing out. No current call site awaits a flush from inside
one, so this is a latent cycle rather than an observed one; the gate should not
depend on that staying true.

The queue is now a bare deferred. The leader's finally opens the gate and
PROMOTES the waiter to a new leader, settling the deferred from that run; the
finally returns nothing, so the leader never awaits its own follower. Every
exit runs the same promotion — the leader resolving, the leader rejecting, the
promoted run rejecting — so a queued caller is settled exactly once on every
path, and a synchronous failure starting the promoted run is reported to the
waiter instead of thrown into the leader's finally. close() drains both handles.

tests/unit/brainy/flush-single-flight.test.ts pins the invariant on each path
that must settle a waiter: many callers during one flush all resolve within a
bound (one body, one follow-up, peak concurrency 1); a REJECTING leader still
runs and settles the queued waiter; a rejecting follow-up settles its waiter
and leaves the gate open; and the leader returns without waiting for a
deliberately slower follower.
This commit is contained in:
David Snelling 2026-09-02 12:33:59 -07:00
parent ebb3a4bf13
commit dea3ec2031
3 changed files with 243 additions and 18 deletions

View file

@ -785,9 +785,25 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* "Flushing Brainy indexes and caches to disk..." runs overlapping 3s
* apart on one brain, their walls growing 295ms 4.9s as they contended
* for the same providers.
*
* THE WAITER IS SETTLED BY THE MACHINE, NEVER BY A PROMISE CHAIN. The queue
* is a BARE DEFERRED (`_flushQueued` plus its `_flushQueuedSettle` handles),
* not `leader.then(() => this.flush())`. A chained follow-up is settled only
* by resolving the very promise the leader is being awaited through, so the
* moment anything inside a flush body awaits `flush()` the graph closes on
* itself and NOBODY resolves an unbounded hang, not a slow flush. Here the
* leader never awaits the queue: its `finally` PROMOTES the waiter to a new
* leader and settles the deferred from that run, and the leader's own
* promise settles without waiting for it. Every exit the leader
* resolving, the leader REJECTING, the promoted run rejecting runs the
* same promotion, so a queued caller is always settled exactly once.
*/
private _flushInFlight: Promise<void> | null = null
private _flushFollowUp: Promise<void> | null = null
private _flushQueued: Promise<void> | null = null
private _flushQueuedSettle: {
resolve: () => void
reject: (error: unknown) => void
} | null = null
/** Flush bodies that got past the single-flight gate (pinned by tests). */
private _flushBodyRuns = 0
/** Flush bodies running right now, and the high-water mark which the
@ -12987,29 +13003,61 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// crossed BEFORE any await, so two callers in the same tick cannot both
// find the field empty.
if (this._flushInFlight) {
if (!this._flushFollowUp) {
// The running flush's failure is not this follow-up's failure: it is
// reported to ITS caller, and the queued work still gets its turn.
this._flushFollowUp = this._flushInFlight
.catch(() => {})
.then(() => {
this._flushFollowUp = null
return this.flush()
})
if (!this._flushQueued) {
// A BARE DEFERRED, not a chain off the leader — see the field's doc.
// Nothing here awaits the leader, so no waiter can ever be reachable
// only through the promise it is itself blocking.
this._flushQueued = new Promise<void>((resolve, reject) => {
this._flushQueuedSettle = { resolve, reject }
})
}
return this._flushFollowUp
return this._flushQueued
}
return this.startFlushLeader()
}
/**
* @description Run one flush body as the leader and install it as
* `_flushInFlight`. On settle resolved OR rejected the gate opens and
* the ONE queued waiter (if any) is promoted. The `finally` callback returns
* nothing on purpose: a callback that returned the promoted run's promise
* would make the leader await its own follower.
* @returns The leader's own promise, settling on its own body alone.
*/
private startFlushLeader(): Promise<void> {
const run = this._runFlush()
// `finally` and not `then`: a failed flush must still open the gate, or
// one rejection would wedge every later flush behind a promise nobody
// will ever settle.
const gated = run.finally(() => {
const gated: Promise<void> = run.finally(() => {
if (this._flushInFlight === gated) this._flushInFlight = null
this.promoteQueuedFlush()
})
this._flushInFlight = gated
return gated
}
/**
* @description Promote the single queued waiter (if one is waiting) to
* leader and settle its deferred from that run. Never throws into the
* leader's `finally`: a synchronous failure starting the promoted run is
* reported to the waiter, which must be settled on every path.
* @returns Nothing.
*/
private promoteQueuedFlush(): void {
const settle = this._flushQueuedSettle
if (!settle) return
// Clear BEFORE starting, so the promoted run's own joiners queue afresh
// rather than joining a deferred that is already being settled.
this._flushQueued = null
this._flushQueuedSettle = null
try {
this.startFlushLeader().then(settle.resolve, settle.reject)
} catch (error) {
settle.reject(error)
}
}
/**
* @description The flush body everything {@link flush} promises, run
* exactly once at a time by that method's single-flight gate. Private
@ -20409,9 +20457,11 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// awaits its leader too, so the second pass is a no-op unless a writer
// raced this close.
for (let pass = 0; pass < 2; pass++) {
const chain = this._flushFollowUp ?? this._flushInFlight
if (!chain) break
await chain.catch(() => {})
const inFlight = this._flushInFlight
const queued = this._flushQueued
if (!inFlight && !queued) break
if (inFlight) await inFlight.catch(() => {})
if (queued) await queued.catch(() => {})
}
// Cancel any pending post-import background deduplication FIRST — it is a