feat(8.0): Model-B per-write generation-stamping + adaptive retention knob

Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.

Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
  deferred durability. Wired into add/update/remove/relate/updateRelation/
  unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
  immediately; its before-image is buffered in an in-memory pending tier that
  resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
  synchronous now() freezes with no forced flush. One fsync per window
  (triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
  groupCommit:true): a crash mid-flush discards the partial generation and
  never restores its before-images, which would otherwise revert the
  already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
  baseline: a fresh brain reports generation()===0 and an empty
  transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
  (generation()), so un-flushed single-op writes are overlaid too.

Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
  { maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
  adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
  caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
  reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
  (a coordinator's fair-share input). Per-generation bytes recorded in each
  delta enable historyBytes() introspection without a storage size API.

Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
This commit is contained in:
David Snelling 2026-06-22 15:19:58 -07:00
parent afac7f9662
commit 5c3bb2c864
15 changed files with 1207 additions and 218 deletions

View file

@ -149,12 +149,22 @@ overlay entities carry no embeddings (`with()` never invokes the embedder),
so a "full" index query over an overlay would silently exclude the
overlay's own entities. Commit with `transact()` to get the full surface.
**History granularity.** Generation *records* are written per `transact()`
batch only. Single-operation writes advance the counter (so watermarks and
CAS stay sound) but do not stage before-images: they remain visible through
earlier pins and are not reported by `db.since()`. Code that needs pinned
isolation across its writes uses `transact()` — that is the documented 8.0
contract, stated rather than papered over.
**History granularity (Model-B).** EVERY write is its own immutable generation
`transact()` batches AND single-operation `add`/`update`/`remove`/`relate`.
Single-ops stage a before-image and are reported by `db.since()`/`asOf()`/
`diff()`/`history()` exactly like transacts; a pin always freezes against later
writes. `transact()` groups several operations into ONE atomic generation.
Single-op history durability is **async group-commit**: the live write hits
canonical storage immediately (acknowledged), while its before-image is buffered
and persisted to disk in one batched fsync on a size/timer trigger (or forced by
`flush()`/`close()`/`transact()`/`compactHistory()`). The buffer participates in
point-in-time resolution exactly like on-disk generations, so the synchronous
`now()` freezes with no forced flush. A hard crash before the flush loses only
the buffered *history* of the last window — never live data — and a crash
*mid-flush* is recovered by **drop-without-restore** (the partial generation's
before-images are discarded, never replayed, because the live write was already
acknowledged; restoring them would silently revert it).
### Pinning, retention, compaction
@ -162,11 +172,17 @@ contract, stated rather than papered over.
`pin(generation)` on every registered `VersionedIndexProvider`, whose
explicit pin lifetime overrides any time-based snapshot retention the
provider has).
- `compactHistory({ retainGenerations?, retainMs? })` reclaims record-sets.
A record-set `N` is reclaimed only when `N` is at or below **every** live
- The constructor **`retention`** knob governs auto-compaction (on `flush()`/
`close()`): unset → ADAPTIVE (disk/RAM-pressure byte budget, zero-config;
driven by a coordinator's `budgetBytes` or a local `os.freemem` probe) ·
`'all'` → unbounded · `{ maxGenerations?, maxAge?, maxBytes? }` → explicit
CAPS. `compactHistory({ maxGenerations?, maxAge?, maxBytes? })` reclaims
manually on the same caps — the oldest unpinned record-sets are reclaimed
while ANY supplied cap is exceeded.
- A record-set `N` is reclaimed only when `N` is at or below **every** live
pin — deleting `N` can only break readers pinned *below* `N`, because
resolution reads before-images from generations strictly greater than the
pin. With both retention options supplied, both must allow the reclaim.
pin. Live pins are ALWAYS exempt, in every retention mode.
- The manifest records the **horizon** (highest reclaimed generation).
Generations below the horizon are unreachable; `asOf()` on them throws
`GenerationCompactedError`. The horizon itself stays reachable, resolved