feat: generation-segment store — the D1+D3 packed-tier file format
First stage of the co-frozen D1+D3+repacking unit: the format core, self-contained under _generations/segments/. - seg-<firstGen,20pad>.bgs: append-once packs of consecutive generations (magic BGS1; frame = u32 len + u32 crc32c + msgpack [generation, timestamp, delta, records, flags]; flags reserves compressed-payload evolution without a format break). Sealed segments are immutable — fold refuses overlap with sealed ranges. - seg-<firstGen>.idx: DERIVED sidecar (per-generation frame offsets + per-id generation postings + checksums); lost/corrupt sidecars rebuild from their segment loudly; a damaged segment (frame CRC mismatch) fails loudly, never serves wrong bytes. - manifest.json: the one discovery path — open() reads it and never lists the packed backlog (the scan-wedge class's cure); refuses a newer manifest version rather than serving partial history. - D3 semantics: dropSegmentsBelow reclaims WHOLE segments at boundaries only and bumps compactedBelow durably; archival-profile enforcement stays with the caller per the co-freeze. - D8 rider: digestThroughPacked(g) — deterministic crc32c chain over sealed-segment checksums (+ frame-level prefix mid-segment), O(segments), reopen-stable. Also fixes a cross-adapter contract bug the suite caught: memory storage's deleteObjectFromPath ignored the raw-bytes store, so deleteRawObject on a raw-bytes path (fact-log or segment files) silently no-op'd — deletes now match filesystem unlink semantics. Six pins. Wiring into GenerationStore (two-tier reads, the repacker, cold-open manifest path) lands with the rest of the unit before its release; cortex's fact-record/stamp shapes reconcile the sidecar keying when they post.
This commit is contained in:
parent
f8e6da2b66
commit
d8acb3776b
3 changed files with 614 additions and 0 deletions
|
|
@ -133,6 +133,11 @@ export class MemoryStorage extends BaseStorage {
|
|||
*/
|
||||
protected async deleteObjectFromPath(path: string): Promise<void> {
|
||||
this.objectStore.delete(path)
|
||||
// Filesystem parity: on disk, objects and raw BYTE files are both just
|
||||
// files — unlink removes whichever exists. Without this, deleteRawObject
|
||||
// on a raw-bytes path (fact-log/generation segments) silently no-ops on
|
||||
// memory storage: the delete "succeeds" and the bytes remain.
|
||||
this.rawBytesStore.delete(path)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue