feat(8.0): Model-B per-write generation-stamping + adaptive retention knob
Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.
Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
deferred durability. Wired into add/update/remove/relate/updateRelation/
unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
immediately; its before-image is buffered in an in-memory pending tier that
resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
synchronous now() freezes with no forced flush. One fsync per window
(triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
groupCommit:true): a crash mid-flush discards the partial generation and
never restores its before-images, which would otherwise revert the
already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
baseline: a fresh brain reports generation()===0 and an empty
transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
(generation()), so un-flushed single-op writes are overlaid too.
Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
(a coordinator's fair-share input). Per-generation bytes recorded in each
delta enable historyBytes() introspection without a storage size API.
Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
This commit is contained in:
parent
afac7f9662
commit
5c3bb2c864
15 changed files with 1207 additions and 218 deletions
|
|
@ -138,21 +138,34 @@ export interface TransactReceipt {
|
|||
// ============================================================================
|
||||
|
||||
/**
|
||||
* @description Options for `brain.compactHistory()`. When both options are
|
||||
* supplied a generation must satisfy *both* retention rules to be reclaimed
|
||||
* (the stricter retention wins). With no options, every generation not
|
||||
* @description Options for `brain.compactHistory()` — explicit retention
|
||||
* **caps** (Model-B `retention` knob). Each supplied field is an upper bound:
|
||||
* the oldest unpinned generations are reclaimed while **ANY** supplied cap is
|
||||
* exceeded (predictable ops — the more you constrain, the more is reclaimed).
|
||||
* Live `Db` pins are ALWAYS exempt. With no options, every generation not
|
||||
* protected by a live pin is eligible.
|
||||
*
|
||||
* (8.0 rename: the pre-release `retainGenerations`/`retainMs` *floors* became
|
||||
* `maxGenerations`/`maxAge` *caps* — the `max*` naming signals the semantics.)
|
||||
*/
|
||||
export interface CompactHistoryOptions {
|
||||
/**
|
||||
* Keep at least this many of the most recent committed generations
|
||||
* (`0` = keep none beyond what live pins protect).
|
||||
* Keep at most this many of the most recent committed generations — reclaim
|
||||
* the oldest unpinned generations while the count exceeds it (`0` = reclaim
|
||||
* everything not protected by a live pin).
|
||||
*/
|
||||
retainGenerations?: number
|
||||
maxGenerations?: number
|
||||
/**
|
||||
* Keep every generation committed within the last `retainMs` milliseconds.
|
||||
* Keep only generations committed within the last `maxAge` milliseconds —
|
||||
* reclaim unpinned generations older than the window.
|
||||
*/
|
||||
retainMs?: number
|
||||
maxAge?: number
|
||||
/**
|
||||
* Keep total generational-history bytes at or below this — reclaim the oldest
|
||||
* unpinned generations while the total exceeds it. Byte accounting is the sum
|
||||
* of each surviving generation's serialized record set (`GenerationDelta.bytes`).
|
||||
*/
|
||||
maxBytes?: number
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -176,11 +189,10 @@ export interface CompactHistoryResult {
|
|||
|
||||
/**
|
||||
* @description Result of `db.since(priorDb)` — the ids touched by committed
|
||||
* transactions in the generation interval `(priorDb.generation,
|
||||
* db.generation]`. Granularity note: single-operation writes performed
|
||||
* outside `transact()` bump the generation counter but do not produce
|
||||
* generation records, so they are not reflected here (documented 8.0
|
||||
* history granularity — see `docs/ADR-001-generational-mvcc.md`).
|
||||
* generations in the interval `(priorDb.generation, db.generation]`. Under
|
||||
* Model-B EVERY write is its own generation, so single-operation writes
|
||||
* (`add`/`update`/`remove`/`relate` outside `transact()`) ARE reflected here,
|
||||
* exactly like `transact()` commits.
|
||||
*/
|
||||
export interface ChangedIds {
|
||||
/** Entity (noun) ids touched in the interval, sorted. */
|
||||
|
|
@ -219,7 +231,8 @@ export interface DiffResult {
|
|||
* @description One version of a single entity or relationship in its
|
||||
* {@link EntityHistory} — the materialized state at a committed generation that
|
||||
* changed it. `value` is `null` when the id did not exist at that version
|
||||
* (created-after / removed). Granularity is `transact()` commits.
|
||||
* (created-after / removed). Granularity is per-write (every `transact()` AND
|
||||
* single-op write is its own generation — Model-B).
|
||||
*/
|
||||
export interface HistoryVersion<T = any> {
|
||||
/** The committed generation that produced this version. */
|
||||
|
|
@ -271,9 +284,11 @@ export interface GenerationRecord {
|
|||
|
||||
/**
|
||||
* @description The per-generation delta persisted at
|
||||
* `_generations/<N>/tx.json` — written and fsynced *before* any operation of
|
||||
* the transaction executes, so crash recovery always knows exactly which ids
|
||||
* to restore from before-images.
|
||||
* `_generations/<N>/tx.json`. For a `transact()` generation it is written and
|
||||
* fsynced *before* any operation of the transaction executes, so crash
|
||||
* recovery always knows exactly which ids to restore from before-images. For a
|
||||
* single-operation (Model-B) generation it is written at group-commit flush
|
||||
* time with `groupCommit: true` — see that flag.
|
||||
*/
|
||||
export interface GenerationDelta {
|
||||
/** The generation this delta belongs to. */
|
||||
|
|
@ -286,6 +301,26 @@ export interface GenerationDelta {
|
|||
nouns: string[]
|
||||
/** Relationship ids touched by this generation. */
|
||||
verbs: string[]
|
||||
/**
|
||||
* `true` for a Model-B single-operation generation persisted by the async
|
||||
* group-commit flush (`GenerationStore.flushPendingSingleOps`). It marks the
|
||||
* **drop-without-restore** crash-recovery contract: the live write already
|
||||
* mutated canonical storage and was acknowledged to the caller BEFORE the
|
||||
* flush, so an uncommitted (crashed-mid-flush) generation with this flag set
|
||||
* must be DISCARDED — its before-images must NEVER be restored, or recovery
|
||||
* would silently revert acknowledged user writes. Absent/`false` on a
|
||||
* `transact()` generation, whose before-images ARE the durable undo log and
|
||||
* are restored on rollback (the before-image + execute are one atomic unit).
|
||||
*/
|
||||
groupCommit?: boolean
|
||||
/**
|
||||
* Serialized byte size of this generation's record set (the `prev/<id>.json`
|
||||
* before-images plus this delta), recorded at write time so `maxBytes` /
|
||||
* adaptive retention can bound total history without a storage size API. Read
|
||||
* back lazily by `GenerationStore.historyBytes()`. Absent on generations
|
||||
* written before byte accounting existed (treated as 0).
|
||||
*/
|
||||
bytes?: number
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue