fix(locks): live writers are never auto-evicted; evicted writers are fenced at every commit barrier
The production dev-store split-brain (two live writers alternating a store's
id-mapper between two internally-consistent truths), cured at all three of
its roots. (1) STALENESS REQUIRES PID-DEATH: the old rule evicted on
heartbeat age alone, so a >60s event-loop stall (debugger pause, GC, heavy
sync work) handed the lock to a second opener while the first kept writing;
a live process is now never auto-evicted — a wedged-but-alive holder is the
operator's call via {force:true}, and the heartbeat stays for observability.
(2) THE CLAIM IS ATOMIC: writeFile(wx)'s open→write→close left an empty-file
window a concurrent opener could read as torn, unlink a LIVE claim, and take
the lock; the claim is now tmp-write + hard-link — the lock appears with its
full contents in one step. (3) THE FENCE: every flush commit and transact
barrier verifies lock ownership first (one small read per window) — a
forced-out or lock-deleted writer fails typed (BRAINY_WRITER_FENCED) before
a single staged byte or manifest advance, instead of writing on unaware.
Pinned: live-with-ancient-heartbeat refuses typed; dead-PID self-clears
narrated; a forced-out writer's flush and transact both fence, advancing
nothing. Requested by a downstream team as single-writer guard or loud
lockout — this is both.
This commit is contained in:
parent
9ac9e70686
commit
292e7c0406
5 changed files with 225 additions and 8 deletions
|
|
@ -1394,6 +1394,10 @@ export class GenerationStore {
|
|||
// The transaction's entire canonical footprint is now durable, so the
|
||||
// counter/manifest advance below can never outrun the entity bytes.
|
||||
await this.storage.flushWriteBarrier?.()
|
||||
// THE FENCE (transact leg): verify lock ownership before the commit
|
||||
// point — an aborted-by-fence transact rolls back cleanly through the
|
||||
// catch below; a fenced writer must never advance counter or manifest.
|
||||
await this.storage.assertWriterFenceHeld?.()
|
||||
faultPoint('after-execute')
|
||||
|
||||
// Fact log (dual-write): append + fsync this generation's AFTER-IMAGE
|
||||
|
|
@ -1915,6 +1919,11 @@ export class GenerationStore {
|
|||
private async flushPendingSingleOpsUnlocked(): Promise<void> {
|
||||
return this.withMutex(async () => {
|
||||
if (this.pendingGens.length === 0) return
|
||||
// THE FENCE: an evicted writer (force-takeover, removed lock) must fail
|
||||
// HERE, before a single staged byte or manifest advance — writing on
|
||||
// after eviction is how split-brain stores are made. One small read
|
||||
// per flush window.
|
||||
await this.storage.assertWriterFenceHeld?.()
|
||||
this.clearPendingFlushTimer()
|
||||
|
||||
const gens = [...this.pendingGens].sort((a, b) => a - b)
|
||||
|
|
|
|||
|
|
@ -485,6 +485,16 @@ export interface GenerationStorage {
|
|||
*/
|
||||
syncEntityCanonical?(nouns: string[], verbs: string[]): Promise<void>
|
||||
|
||||
/**
|
||||
* OPTIONAL writer fence: throw `BRAINY_WRITER_FENCED` when this instance
|
||||
* no longer owns the store's writer lock (an operator force-takeover or a
|
||||
* removed lock file). Called at every flush commit and transact barrier —
|
||||
* one small read per commit window — so an evicted writer fails loudly on
|
||||
* its next commit instead of split-braining the store. Adapters without a
|
||||
* cross-process lock model omit it.
|
||||
*/
|
||||
assertWriterFenceHeld?(): Promise<void>
|
||||
|
||||
/** Read an entity's raw stored metadata+vector objects. */
|
||||
readNounRaw(id: string): Promise<{ metadata: any | null; vector: any | null }>
|
||||
/** Restore an entity's raw stored objects (`null` part ⇒ delete that file). */
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue