Commit graph

1 commit

Author SHA1 Message Date
d8acb3776b feat: generation-segment store — the D1+D3 packed-tier file format
First stage of the co-frozen D1+D3+repacking unit: the format core,
self-contained under _generations/segments/.

- seg-<firstGen,20pad>.bgs: append-once packs of consecutive
  generations (magic BGS1; frame = u32 len + u32 crc32c + msgpack
  [generation, timestamp, delta, records, flags]; flags reserves
  compressed-payload evolution without a format break). Sealed
  segments are immutable — fold refuses overlap with sealed ranges.
- seg-<firstGen>.idx: DERIVED sidecar (per-generation frame offsets +
  per-id generation postings + checksums); lost/corrupt sidecars
  rebuild from their segment loudly; a damaged segment (frame CRC
  mismatch) fails loudly, never serves wrong bytes.
- manifest.json: the one discovery path — open() reads it and never
  lists the packed backlog (the scan-wedge class's cure); refuses a
  newer manifest version rather than serving partial history.
- D3 semantics: dropSegmentsBelow reclaims WHOLE segments at
  boundaries only and bumps compactedBelow durably; archival-profile
  enforcement stays with the caller per the co-freeze.
- D8 rider: digestThroughPacked(g) — deterministic crc32c chain over
  sealed-segment checksums (+ frame-level prefix mid-segment),
  O(segments), reopen-stable.

Also fixes a cross-adapter contract bug the suite caught: memory
storage's deleteObjectFromPath ignored the raw-bytes store, so
deleteRawObject on a raw-bytes path (fact-log or segment files)
silently no-op'd — deletes now match filesystem unlink semantics.

Six pins. Wiring into GenerationStore (two-tier reads, the repacker,
cold-open manifest path) lands with the rest of the unit before its
release; cortex's fact-record/stamp shapes reconcile the sidecar
keying when they post.
2026-07-19 15:14:27 -07:00