soulcraftlabs/open-brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 343 Packages Wiki Activity Actions

feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract
All checks were successful
CI / Node 22 (push) Successful in 12m16s
Details
CI / Node 24 (push) Successful in 12m13s
Details
CI / Bun (latest) (push) Successful in 12m20s
Details

Browse source 
THE DEFAULT FLIP (ruled on proven evidence — at-ack survived 301/301
acked-writes-through-power-cut in block-layer fault injection; deferred
tree authority demonstrably loses flush-covered acks): a brain with NO
stored authority artifact now ADOPTS LOG AUTHORITY AT OPEN. The oracle
gates the flip exactly as the guarded adoption path always did — curable
divergences baseline-backfilled, the flip lands ONLY on a green verdict —
and a brain that cannot verify STAYS tree-authoritative loudly, with the
refusal recorded on the switch artifact so subsequent opens are cheap.
config logAuthority: 'defer' is the explicit documented opt-out (no
automatic adoption; declared flush-window loss; adoptLogAuthority() flips
later). A stored artifact always wins. RELEASES.md carries the posture.

Two standing .fails debt pins FLIP TO HOLDING under the default: the
at-ack crash-survival gap and the ack-at-log durability target — both now
permanent asserted truths, not aspirations.

POWER-CUT THROW SITES (fault-injection findings, brainy-alone config):
- A manifest-listed-but-unloadable column segment QUARANTINES at
  discovery (loud once, counted always, quarantinedSegments() exposed for
  the heal) and the field serves its remaining segments DEGRADED — never
  a raw throw killing every query on the field. Real storage faults still
  propagate untouched.
- Torn generation artifacts (NaN/garbage in manifest or counter) DISCARD
  with narration at the store's open and recovery re-derives — plus a
  defensive finite-integer guard at the init consumer. Never a RangeError
  killing an open.

THE LOUD TORN-RECORD CONTRACT: an existing-but-unparseable stored record
now surfaces as a typed, counted TornRecordError on every entity-read
surface (including fifteen previously-blind per-item batch catches);
ENOENT stays clean-absent; artifact readers with designed absent-recovery
keep null-tolerance behind the loud floor. Disk corruption can no longer
read as silent data invisibility.

Suite migration: the default's pins inverted deliberately, generation
baselines made relative, quarantine-contract pins rewritten to the ruled
behavior.

Gates: tsc 0 · unit 2065/2065 (159 files) · integration 826 (93 files) ·
conformance 31/31 · kill-matrix 15/15 · torn-open guards 2/2.
This commit is contained in:
David Snelling 2026-08-11 08:37:38 -07:00
parent 67c606be69
commit 214c98b4d5
23 changed files with 833 additions and 154 deletions
Download patch file Download diff file Expand all files Collapse all files

99
src/storage/baseStorage.ts
View file

@ -32,6 +32,7 @@ import { BlobStorage, type BlobStoreAdapter } from './blobStorage.js'
import { unwrapBinaryData } from './binaryDataCodec.js'
import { prodLog } from '../utils/logger.js'
import { isAbsentError } from '../utils/errorClassification.js'
import { isTornRecordError } from './tornRecordError.js'
import { BrainyError, ProtectedArtifactError, DerivedArtifactMissingError } from '../errors/brainyError.js'
import { MetadataWriteBuffer } from '../utils/metadataWriteBuffer.js'
import {
@ -674,6 +675,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// — hash verification must run on the original content bytes.
return unwrapBinaryData(data)
} catch (error) {
// A TORN blob object (exists but undecodable) must not read as
// "blob absent" — that would misdiagnose disk corruption as a
// missing blob. Propagate the typed error to the blob layer.
if (isTornRecordError(error)) throw error
return undefined
}
},
@ -768,6 +773,20 @@ export abstract class BaseStorage extends BaseStorageAdapter {
if (m) hashes.add(m[1])
}
// Recovery-path read: a TORN object here maps to "not usable" (null) BY
// DESIGN — the adapter has already logged + counted the encounter, and
// treating a torn `_cas/` copy as absent lets the re-copy from `_cow/`
// OVERWRITE the corrupt file with the good original (the heal), while a
// torn `_cow/` original is reported via `incomplete`. Real faults propagate.
const readOrNullIfTorn = async (p: string): Promise<any | null> => {
try {
return await this.readObjectFromPath(p)
} catch (error) {
if (isTornRecordError(error)) return null
throw error
}
}
let adopted = 0
let alreadyPresent = 0
let incomplete = 0
@ -775,15 +794,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// A blob counts as present only when BOTH its bytes and its metadata
// already live in `_cas/`. A half-adopted blob (bytes without meta — the
// exact "Blob metadata not found" state) is re-adopted.
const casBlob = await this.readObjectFromPath(`_cas/blob:${hash}`)
const casMeta = await this.readObjectFromPath(`_cas/blob-meta:${hash}`)
const casBlob = await readOrNullIfTorn(`_cas/blob:${hash}`)
const casMeta = await readOrNullIfTorn(`_cas/blob-meta:${hash}`)
if (casBlob !== null && casMeta !== null) {
alreadyPresent++
continue
}
const cowBlob = await this.readObjectFromPath(`_cow/blob:${hash}`)
const cowMeta = await this.readObjectFromPath(`_cow/blob-meta:${hash}`)
const cowBlob = await readOrNullIfTorn(`_cow/blob:${hash}`)
const cowMeta = await readOrNullIfTorn(`_cow/blob-meta:${hash}`)
if (cowBlob === null || cowMeta === null) {
// Can't register a blob the store can't fully describe — report it so an
// operator investigates rather than silently half-adopting.
@ -1134,12 +1153,28 @@ export abstract class BaseStorage extends BaseStorageAdapter {
* cache (record-layer files are written through
* {@link BaseStorage.writeRawObject} only).
*
* TORN-record contract (deliberate, loud-by-design): this surface serves
* SYSTEM ARTIFACTS — manifests with recovery paths, markers whose verdict
* machinery treats "unreadable" as rescan, generation/transaction records
* whose recovery is built for absent artifacts. For these readers a torn
* file maps to their existing absent-artifact degrade, so a typed
* torn-record error from the adapter is caught here and returned as `null`
* — AFTER the adapter has already logged a production ERROR and counted
* the per-process torn-record gauge (never silent). Entity reads do NOT go
* through this surface; they use the canonical read paths, which propagate
* the typed error. Real storage faults (EIO/EACCES/…) still propagate.
*
* @param path - Storage-root-relative object path (e.g. `_system/manifest.json`).
* @returns The parsed object, or `null` if absent.
* @returns The parsed object, or `null` if absent (or torn — logged + counted).
*/
public async readRawObject(path: string): Promise<any | null> {
await this.ensureInitialized()
return this.readObjectFromPath(path)
try {
return await this.readObjectFromPath(path)
} catch (error) {
if (isTornRecordError(error)) return null
throw error
}
}
/**
@ -2146,6 +2181,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
if (!metadata) return null
return { deserialized, metadata }
} catch (error) {
// A TORN record must surface typed — a paginated read that
// silently skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip nouns that fail to load
return null
}
@ -2175,6 +2213,8 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -2283,7 +2323,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
batch.map(async (id) => {
try {
return { id, metadata: await this.getNounMetadata(id) }
} catch {
} catch (error) {
// A TORN record must surface typed, never as a skipped id.
if (isTornRecordError(error)) throw error
return null
}
})
@ -2305,6 +2347,8 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards with no data
}
}
@ -2515,10 +2559,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// reserved fields top-level, ONLY custom fields in `metadata`.
collected.push({ verb: this.hydrateVerbWithMetadata(verb, metadata), shard })
} catch (error) {
// A TORN record must surface typed — a paginated read that
// silently skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip verbs that fail to load
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -3669,8 +3718,17 @@ export abstract class BaseStorage extends BaseStorageAdapter {
)
for (const result of chunkResults) {
if (result.status === 'fulfilled' && result.value.data !== null) {
results.set(result.value.path, result.value.data)
if (result.status === 'fulfilled') {
if (result.value.data !== null) {
results.set(result.value.path, result.value.data)
}
} else {
// A rejected read is a torn record or a real storage fault — NOT an
// absent object. Batch hydration backs entity reads (getNounBatch /
// getVerbsBatch / find hydration); swallowing the rejection would
// silently drop a row the caller cannot distinguish from "never
// existed". Propagate the typed/real error loudly instead.
throw result.reason
}
}
}
@ -4636,10 +4694,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
}
} catch (error) {
// A TORN record must surface typed — an enumeration that silently
// skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip nouns that fail to load
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -4825,11 +4888,16 @@ export abstract class BaseStorage extends BaseStorageAdapter {
results.push(this.hydrateVerbWithMetadata(verb, metadata))
}
} catch (error) {
// A TORN record must surface typed — an enumeration that silently
// skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip verbs that fail to load
prodLog.debug(`[BaseStorage] Failed to load verb from ${verbPath}:`, error)
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -4945,6 +5013,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
sourceVerbs.push(hydratedVerb)
}
} catch (error) {
// A TORN record propagates (typed) — batch hydration must not
// silently drop a corrupt row. Only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -5030,10 +5101,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
results.push(this.hydrateVerbWithMetadata(verb, metadata))
}
} catch (error) {
// A TORN record must surface typed — an enumeration that silently
// skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip verbs that fail to load
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}
@ -5078,10 +5154,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
)
)
} catch (error) {
// A TORN record must surface typed — an enumeration that silently
// skips a corrupt row hides data loss from the caller.
if (isTornRecordError(error)) throw error
// Skip verbs that fail to load
}
}
} catch (error) {
// A TORN record propagates (typed) — only shard-listing absence is skippable.
if (isTornRecordError(error)) throw error
// Skip shards that have no data
}
}