feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract
All checks were successful
CI / Node 22 (push) Successful in 12m16s
CI / Node 24 (push) Successful in 12m13s
CI / Bun (latest) (push) Successful in 12m20s

THE DEFAULT FLIP (ruled on proven evidence — at-ack survived 301/301
acked-writes-through-power-cut in block-layer fault injection; deferred
tree authority demonstrably loses flush-covered acks): a brain with NO
stored authority artifact now ADOPTS LOG AUTHORITY AT OPEN. The oracle
gates the flip exactly as the guarded adoption path always did — curable
divergences baseline-backfilled, the flip lands ONLY on a green verdict —
and a brain that cannot verify STAYS tree-authoritative loudly, with the
refusal recorded on the switch artifact so subsequent opens are cheap.
config logAuthority: 'defer' is the explicit documented opt-out (no
automatic adoption; declared flush-window loss; adoptLogAuthority() flips
later). A stored artifact always wins. RELEASES.md carries the posture.

Two standing .fails debt pins FLIP TO HOLDING under the default: the
at-ack crash-survival gap and the ack-at-log durability target — both now
permanent asserted truths, not aspirations.

POWER-CUT THROW SITES (fault-injection findings, brainy-alone config):
- A manifest-listed-but-unloadable column segment QUARANTINES at
  discovery (loud once, counted always, quarantinedSegments() exposed for
  the heal) and the field serves its remaining segments DEGRADED — never
  a raw throw killing every query on the field. Real storage faults still
  propagate untouched.
- Torn generation artifacts (NaN/garbage in manifest or counter) DISCARD
  with narration at the store's open and recovery re-derives — plus a
  defensive finite-integer guard at the init consumer. Never a RangeError
  killing an open.

THE LOUD TORN-RECORD CONTRACT: an existing-but-unparseable stored record
now surfaces as a typed, counted TornRecordError on every entity-read
surface (including fifteen previously-blind per-item batch catches);
ENOENT stays clean-absent; artifact readers with designed absent-recovery
keep null-tolerance behind the loud floor. Disk corruption can no longer
read as silent data invisibility.

Suite migration: the default's pins inverted deliberately, generation
baselines made relative, quarantine-contract pins rewritten to the ruled
behavior.

Gates: tsc 0 · unit 2065/2065 (159 files) · integration 826 (93 files) ·
conformance 31/31 · kill-matrix 15/15 · torn-open guards 2/2.
This commit is contained in:
David Snelling 2026-08-11 08:37:38 -07:00
parent 67c606be69
commit 214c98b4d5
23 changed files with 833 additions and 154 deletions

View file

@ -31,6 +31,7 @@ import { ColumnSegmentCursor, TailBufferCursor, type CursorEntry } from './Colum
import { writeSegmentToBuffer, readSegmentFromBuffer } from './ColumnSegmentFormat.js'
import { RoaringBitmap32 } from '../../utils/roaring/index.js'
import { compareCodePoints } from '../../utils/collation.js'
import { prodLog } from '../../utils/logger.js'
/**
* Configuration for the ColumnStore.
@ -612,6 +613,24 @@ export class ColumnStore implements ColumnStoreProvider {
/**
* Get all segment cursors for a field, loading from storage if needed.
*/
/**
* Per-field quarantine ledger for torn segments (power-loss survivors:
* manifest-listed but unloadable). A quarantined segment is skipped with
* per-doubling narration and the field serves its REMAINING segments as a
* DEGRADED-ANNOUNCED result never a raw throw killing the query, never
* a silent drop. Cleared when a heal/rebuild rewrites the field.
*/
private readonly segmentQuarantine = new Map<string, { error: string; hits: number }>()
/** Torn-segment quarantine entries for a field (observability + heal input). */
quarantinedSegments(field: string): Array<{ segment: string; error: string; hits: number }> {
const out: Array<{ segment: string; error: string; hits: number }> = []
for (const [key, q] of this.segmentQuarantine) {
if (key.startsWith(`${field}:`)) out.push({ segment: key.slice(field.length + 1), error: q.error, hits: q.hits })
}
return out
}
private async getSegmentCursors(field: string): Promise<ColumnSegmentCursor[]> {
const manifest = this.manifests.get(field)
if (!manifest) return []
@ -622,11 +641,38 @@ export class ColumnStore implements ColumnStoreProvider {
let cursor = this.segmentCache.get(cacheKey)
if (!cursor) {
// loadSegmentCursor either returns a cursor or THROWS — a corrupt /
// missing manifest-listed segment raises ColumnSegmentLoadError and a
// real storage fault propagates, so a listed segment is never silently
// dropped from the result set.
cursor = await this.loadSegmentCursor(field, seg)
const quarantined = this.segmentQuarantine.get(cacheKey)
if (quarantined) {
// Already-quarantined torn segment: skip, count, narrate per doubling.
quarantined.hits++
if ((quarantined.hits & (quarantined.hits - 1)) === 0) {
prodLog.warn(
`[ColumnStore] field '${field}' serving DEGRADED: torn segment ${seg.id} ` +
`quarantined (${quarantined.error}) — ${quarantined.hits} queries served ` +
`without it; heal/rebuild the metadata index to restore`
)
}
continue
}
try {
cursor = await this.loadSegmentCursor(field, seg)
} catch (err) {
if (err instanceof ColumnSegmentLoadError) {
// POWER-LOSS SURVIVOR: a manifest-listed segment whose bytes are
// torn/absent. Quarantine at DISCOVERY and serve the remaining
// segments degraded-announced — a raw throw here killed every
// query on the field forever; a silent skip hid the loss. The
// quarantine is the middle: loud once, counted always, healable.
this.segmentQuarantine.set(cacheKey, { error: (err as Error).message, hits: 1 })
prodLog.error(
`[ColumnStore] torn segment QUARANTINED at discovery: field '${field}' ` +
`segment ${seg.id}${(err as Error).message}. The field serves its ` +
`remaining segments DEGRADED until a heal/rebuild rewrites it.`
)
continue
}
throw err // real storage faults propagate — never absorbed
}
this.segmentCache.set(cacheKey, cursor)
}