feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract
All checks were successful
CI / Node 22 (push) Successful in 12m16s
CI / Node 24 (push) Successful in 12m13s
CI / Bun (latest) (push) Successful in 12m20s

THE DEFAULT FLIP (ruled on proven evidence — at-ack survived 301/301
acked-writes-through-power-cut in block-layer fault injection; deferred
tree authority demonstrably loses flush-covered acks): a brain with NO
stored authority artifact now ADOPTS LOG AUTHORITY AT OPEN. The oracle
gates the flip exactly as the guarded adoption path always did — curable
divergences baseline-backfilled, the flip lands ONLY on a green verdict —
and a brain that cannot verify STAYS tree-authoritative loudly, with the
refusal recorded on the switch artifact so subsequent opens are cheap.
config logAuthority: 'defer' is the explicit documented opt-out (no
automatic adoption; declared flush-window loss; adoptLogAuthority() flips
later). A stored artifact always wins. RELEASES.md carries the posture.

Two standing .fails debt pins FLIP TO HOLDING under the default: the
at-ack crash-survival gap and the ack-at-log durability target — both now
permanent asserted truths, not aspirations.

POWER-CUT THROW SITES (fault-injection findings, brainy-alone config):
- A manifest-listed-but-unloadable column segment QUARANTINES at
  discovery (loud once, counted always, quarantinedSegments() exposed for
  the heal) and the field serves its remaining segments DEGRADED — never
  a raw throw killing every query on the field. Real storage faults still
  propagate untouched.
- Torn generation artifacts (NaN/garbage in manifest or counter) DISCARD
  with narration at the store's open and recovery re-derives — plus a
  defensive finite-integer guard at the init consumer. Never a RangeError
  killing an open.

THE LOUD TORN-RECORD CONTRACT: an existing-but-unparseable stored record
now surfaces as a typed, counted TornRecordError on every entity-read
surface (including fifteen previously-blind per-item batch catches);
ENOENT stays clean-absent; artifact readers with designed absent-recovery
keep null-tolerance behind the loud floor. Disk corruption can no longer
read as silent data invisibility.

Suite migration: the default's pins inverted deliberately, generation
baselines made relative, quarantine-contract pins rewritten to the ruled
behavior.

Gates: tsc 0 · unit 2065/2065 (159 files) · integration 826 (93 files) ·
conformance 31/31 · kill-matrix 15/15 · torn-open guards 2/2.
This commit is contained in:
David Snelling 2026-08-11 08:37:38 -07:00
parent 67c606be69
commit 214c98b4d5
23 changed files with 833 additions and 154 deletions

View file

@ -18,6 +18,11 @@ import {
} from '../baseStorage.js'
import { getBrainyVersion } from '../../utils/index.js'
import { isAbsentError } from '../../utils/errorClassification.js'
import {
TornRecordError,
isUnparseablePayloadError,
registerTornRecordEncounter
} from '../tornRecordError.js'
// Node.js modules - dynamically imported to avoid issues in browser environments
let fs: any
@ -410,8 +415,22 @@ export class FileSystemStorage extends BaseStorage {
/**
* Primitive operation: Read object from path
* All metadata operations use this internally via base class routing
* Enhanced error handling for corrupted metadata files (Bug #3 mitigation)
* Supports reading both compressed (.gz) and uncompressed files for backward compatibility
*
* Read contract (loud errors, never quiet losses):
* - Genuine absence (ENOENT on every variant) `null`. Only a missing file
* is "not found".
* - TORN record (a file EXISTS but its bytes cannot be decoded invalid
* JSON, truncated/garbled gzip) the encounter is registered (production
* ERROR log + per-process gauge) and a typed {@link TornRecordError} is
* thrown. Corruption must NEVER read as absence: callers that can degrade
* (manifest recovery, rebuildable statistics) catch the typed error at
* their sites; entity reads surface it.
* Legacy dual-format exception: when the `.gz` variant is torn but the
* uncompressed fallback decodes, the recovered object is returned AFTER
* the torn `.gz` was logged and counted (loud recovery, not a silent skip).
* - Real storage fault (EIO/EACCES/EMFILE/) propagates as itself; a
* fault is neither absence nor corruption and must not be reshaped.
*/
protected async readObjectFromPath(pathStr: string): Promise<any | null> {
await this.ensureInitialized()
@ -419,7 +438,10 @@ export class FileSystemStorage extends BaseStorage {
const fullPath = path.join(this.rootDir, pathStr)
const compressedPath = `${fullPath}.gz`
// Try reading compressed file first (if compression is enabled or file exists)
// Try reading compressed file first (if compression is enabled or file exists).
// A torn .gz is remembered so the uncompressed fallback can either recover
// (legacy dual-format installs) or surface the corruption typed.
let tornCompressed: TornRecordError | null = null
try {
const compressedData = await fs.promises.readFile(compressedPath)
const decompressed = await new Promise<Buffer>((resolve, reject) => {
@ -430,9 +452,16 @@ export class FileSystemStorage extends BaseStorage {
})
return JSON.parse(decompressed.toString('utf-8'))
} catch (error: any) {
// If compressed file doesn't exist, fall back to uncompressed
if (error.code !== 'ENOENT') {
console.warn(`Failed to read compressed file ${compressedPath}:`, error)
if (error.code === 'ENOENT') {
// No compressed variant — fall through to the uncompressed path.
} else if (isUnparseablePayloadError(error)) {
// The .gz EXISTS but cannot be decoded (zlib Z_* error or JSON
// SyntaxError after gunzip): torn record. Register NOW (log + gauge),
// then attempt the uncompressed fallback as a recovery read.
tornCompressed = registerTornRecordEncounter(`${pathStr}.gz`, error)
} else {
// Real storage fault on an existing .gz (EIO/EACCES/…): propagate.
throw error
}
}
@ -442,24 +471,26 @@ export class FileSystemStorage extends BaseStorage {
return JSON.parse(data)
} catch (error: any) {
if (error.code === 'ENOENT') {
// No uncompressed file. If the .gz variant existed but was torn, the
// object EXISTS and is unreadable — that must surface typed, never as
// "absent". Otherwise this is genuine absence.
if (tornCompressed !== null) {
throw tornCompressed
}
return null
}
// Enhanced error handling for corrupted JSON files (race condition from Bug #3)
if (error instanceof SyntaxError || error.name === 'SyntaxError') {
console.warn(
`⚠️ Corrupted metadata file detected: ${pathStr}\n` +
` This may be caused by concurrent writes during import.\n` +
` Gracefully skipping this entry. File may be repaired on next write.`
)
return null
// The file EXISTS but its content cannot be parsed: torn record.
// Register (production ERROR + gauge) and throw typed — a corrupt row
// must be distinguishable from a missing row, or nothing ever heals it.
if (isUnparseablePayloadError(error)) {
throw registerTornRecordEncounter(pathStr, error)
}
// A real storage fault (EIO/EACCES/EMFILE/…) is NOT "object absent". The
// ENOENT branch (above) already returns null, and the corrupted-JSON
// branch (above) is a deliberate concurrent-write tolerance; a genuine
// fault reaching here must propagate loudly rather than masquerade as a
// missing object — which would corrupt reads and drive needless rebuilds.
// ENOENT branch (above) already returns null; a genuine fault reaching
// here must propagate loudly rather than masquerade as a missing object
// — which would corrupt reads and drive needless rebuilds.
throw error
}
}