fix(recovery): walks are healers — the typed/tolerant boundary redrawn where block-layer fault injection proved it belonged
The quiet-loss cure regressed recovery: the new typed torn-record error was correct at identity-read time but threw inside init-time recovery walks, killing opens that previously survived. The boundary, redrawn: - IDENTITY READS (get-by-id of a specific record, CAS blob point-get): typed TornRecordError, unchanged — a caller who asked for THAT record can act on the answer. - SET-SHAPED READS AND WALKS (enumeration, pagination, batch hydration — the paths recovery rebuilds and finds page over): HEAL PAST the torn victim. The adapter's loud floor (error log + counted gauge) fires at the encounter; the walk serves the remaining rows. One crash casualty can no longer kill every query on its shard — or the open itself. - WRITES OVER TORN RECORDS ARE THE CURE: the save path's read-merge, the commit path's before-image capture, and the operations' rollback captures all treat a torn prior as the create sentinel, narrated — the incoming bytes replace the unreadable ones, and history for the id honestly restarts at that generation. Corruption can never block its own heal. - THE NaN SOURCE: torn mapper state (nextId/entries carrying garbage) discards with narration and re-derives via the existing rebuild path; the mint gains a source guard healing a non-integer counter from the live map. The reopen and first-write RangeError shapes are dead at the source, both authority branches. Pinned with the exact fault-injection scenarios: a torn entity record (including the VFS root) no longer kills the open — walks heal past it, the keeper rows serve, and the identity read of the victim itself is typed-or-healed; a torn mapper reopens and mints sanely on the first post-recovery write. Gates: tsc 0 · unit 2065/2065 · integration 828 · conformance 31/31.
This commit is contained in:
parent
214c98b4d5
commit
0e3facf4a8
6 changed files with 350 additions and 39 deletions
|
|
@ -129,11 +129,49 @@ export class EntityIdMapper implements EntityIdMapperProvider {
|
|||
// metadata channel as plain JSON; the `nextId` probe above identifies
|
||||
// the persisted EntityIdMapperData shape.
|
||||
const data = metadata as unknown as EntityIdMapperData
|
||||
this.nextId = data.nextId
|
||||
|
||||
// Rebuild maps from serialized data
|
||||
this.uuidToInt = new Map(Object.entries(data.uuidToInt).map(([k, v]) => [k, Number(v)]))
|
||||
this.intToUuid = new Map(Object.entries(data.intToUuid).map(([k, v]) => [Number(k), v]))
|
||||
// TORN-STATE VALIDATION (power-loss survivor): a torn mapper file
|
||||
// can carry NaN/garbage where integers belong — unvalidated, those
|
||||
// NaNs reach BigInt() on the graph's int-resolution (reopen) and
|
||||
// the mint path (first write after recovery) and kill both with
|
||||
// RangeErrors. A torn mapper is DISCARDED with narration and the
|
||||
// maps re-derive through the existing rebuild path (under log
|
||||
// authority the mint-at-append records reproduce assignments
|
||||
// exactly; under tree authority the metadata-index reconstruction
|
||||
// rebuilds them — the same path a missing mapper file takes).
|
||||
const validInt = (v: unknown): v is number =>
|
||||
typeof v === 'number' && Number.isSafeInteger(v) && v >= 0
|
||||
let torn = !validInt(data.nextId)
|
||||
const uuidToInt = new Map<string, number>()
|
||||
const intToUuid = new Map<number, string>()
|
||||
if (!torn) {
|
||||
for (const [k, v] of Object.entries(data.uuidToInt ?? {})) {
|
||||
const n = Number(v)
|
||||
if (!validInt(n)) { torn = true; break }
|
||||
uuidToInt.set(k, n)
|
||||
}
|
||||
}
|
||||
if (!torn) {
|
||||
for (const [k, v] of Object.entries(data.intToUuid ?? {})) {
|
||||
const n = Number(k)
|
||||
if (!validInt(n) || typeof v !== 'string') { torn = true; break }
|
||||
intToUuid.set(n, v)
|
||||
}
|
||||
}
|
||||
if (torn) {
|
||||
console.warn(
|
||||
`[EntityIdMapper] persisted mapper state is TORN (non-integer ids — ` +
|
||||
`power-loss survivor); discarding and re-deriving via the rebuild ` +
|
||||
`path. Never a RangeError at reopen or first write.`
|
||||
)
|
||||
this.nextId = 1
|
||||
this.uuidToInt = new Map()
|
||||
this.intToUuid = new Map()
|
||||
} else {
|
||||
this.nextId = data.nextId
|
||||
this.uuidToInt = uuidToInt
|
||||
this.intToUuid = intToUuid
|
||||
}
|
||||
} else {
|
||||
// Guard: mapper file missing but entities may exist on disk.
|
||||
// If we start from nextId=1 with existing entities, roaring bitmap
|
||||
|
|
@ -178,7 +216,19 @@ export class EntityIdMapper implements EntityIdMapperProvider {
|
|||
return existing
|
||||
}
|
||||
|
||||
// Assign new ID
|
||||
// Assign new ID. Source guard: nextId must be a finite positive integer
|
||||
// — the load path validates persisted state, but a NaN here would mint
|
||||
// poison ints that reach BigInt() downstream; heal to the map-derived
|
||||
// floor with narration rather than propagate.
|
||||
if (!Number.isSafeInteger(this.nextId) || this.nextId < 1) {
|
||||
let floor = 1
|
||||
for (const n of this.intToUuid.keys()) if (n >= floor) floor = n + 1
|
||||
console.warn(
|
||||
`[EntityIdMapper] nextId was non-integer (${String(this.nextId)}) — ` +
|
||||
`healed to ${floor} from the live map; torn-state survivor`
|
||||
)
|
||||
this.nextId = floor
|
||||
}
|
||||
if (this.nextId > U32_ENTITY_ID_MAX) {
|
||||
throw new EntityIdSpaceExceeded(this.nextId)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue