feat(8.0): Model-B per-write generation-stamping + adaptive retention knob
Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.
Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
deferred durability. Wired into add/update/remove/relate/updateRelation/
unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
immediately; its before-image is buffered in an in-memory pending tier that
resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
synchronous now() freezes with no forced flush. One fsync per window
(triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
groupCommit:true): a crash mid-flush discards the partial generation and
never restores its before-images, which would otherwise revert the
already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
baseline: a fresh brain reports generation()===0 and an empty
transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
(generation()), so un-flushed single-op writes are overlaid too.
Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
(a coordinator's fair-share input). Per-generation bytes recorded in each
delta enable historyBytes() introspection without a storage size API.
Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
This commit is contained in:
parent
afac7f9662
commit
5c3bb2c864
15 changed files with 1207 additions and 218 deletions
31
RELEASES.md
31
RELEASES.md
|
|
@ -110,13 +110,29 @@ What you get:
|
|||
- **`brain.transactionLog({ from?, to?, limit? })`** — the commit log over an
|
||||
**inclusive** generation/`Date` window (contrast `since`'s exclusive lower
|
||||
bound); `limit` applies last.
|
||||
- **`brain.compactHistory({ retainGenerations, retainMs })`** — explicit
|
||||
retention. Compaction never breaks a pinned read. `diff`/`since` throw
|
||||
`GenerationCompactedError` below the horizon; `history` truncates to it.
|
||||
- **Every write is versioned (Model-B).** History granularity is **per-write**:
|
||||
EVERY write — `transact()` AND a single-operation `add`/`update`/`remove`/
|
||||
`relate` — is its own immutable generation. A `now()` pin always freezes
|
||||
against later writes, and `asOf`/`since`/`diff`/`history`/`transactionLog`
|
||||
reflect single-ops exactly like transacts. `transact()` groups several
|
||||
operations into ONE atomic generation. Single-op history durability is **async
|
||||
group-commit** (the live write is acknowledged immediately; its before-image
|
||||
is batched to disk in one fsync) — a hard crash can lose only the last
|
||||
un-flushed window's *history*, never live data, and a crash mid-flush is
|
||||
recovered by drop-without-restore. A freshly-initialized brain is
|
||||
`generation() === 0` with an empty `transactionLog()` (init-time
|
||||
infrastructure is the un-versioned baseline); the first user write is gen 1.
|
||||
- **`new Brainy({ retention })`** — the retention knob governs auto-compaction on
|
||||
`flush()`/`close()`: **unset → ADAPTIVE** (disk/RAM-pressure byte budget,
|
||||
zero-config; a coordinator can drive it via `brain.setRetentionBudget(bytes)`)
|
||||
· **`'all'`** → unbounded · **`{ maxGenerations?, maxAge?, maxBytes? }`** →
|
||||
explicit caps (reclaim oldest-unpinned while ANY cap is exceeded). Live pins
|
||||
are ALWAYS exempt.
|
||||
- **`brain.compactHistory({ maxGenerations?, maxAge?, maxBytes? })`** — manual
|
||||
reclaim on the same caps. Compaction never breaks a pinned read. `diff`/`since`
|
||||
throw `GenerationCompactedError` below the horizon; `history` truncates to it.
|
||||
|
||||
The precise guarantees (and the honest limits — history granularity is
|
||||
`transact()` commits; single-operation writes advance the clock but produce no
|
||||
historical records) are documented in:
|
||||
The precise guarantees are documented in:
|
||||
|
||||
- [docs/concepts/consistency-model.md](docs/concepts/consistency-model.md) —
|
||||
the guarantees, each proven by a dedicated test in
|
||||
|
|
@ -228,6 +244,9 @@ Hard renames, no aliases. Every pair below is verified against the 8.0 source.
|
|||
| `brain.stats().indexHealth.hnsw` | `brain.stats().indexHealth.vector` |
|
||||
| Storage adapter contract `saveHNSWData()` / `getHNSWData()` | `saveVectorIndexData()` / `getVectorIndexData()` |
|
||||
| Cache category `'hnsw'` (`CacheProvider` union) | `'vectors'` |
|
||||
| `config.history = { retainGenerations, retainMs, autoCompact }` | `config.retention` — `'all'` \| `'adaptive'` \| `{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }`; **unset → adaptive** (was keep-100/7-days). The fields are now **caps**, not floors. |
|
||||
| `compactHistory({ retainGenerations, retainMs })` | `compactHistory({ maxGenerations, maxAge, maxBytes })` — `retainGenerations`→`maxGenerations`, `retainMs`→`maxAge` (now upper-bound CAPS), plus new `maxBytes`. |
|
||||
| `CompactHistoryOptions.retainGenerations` / `.retainMs` | `.maxGenerations` / `.maxAge` (+ `.maxBytes`) |
|
||||
|
||||
Mechanical renames as one command (run from your repo root, review the diff):
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue