feat(8.0): Model-B per-write generation-stamping + adaptive retention knob
Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.
Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
deferred durability. Wired into add/update/remove/relate/updateRelation/
unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
immediately; its before-image is buffered in an in-memory pending tier that
resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
synchronous now() freezes with no forced flush. One fsync per window
(triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
groupCommit:true): a crash mid-flush discards the partial generation and
never restores its before-images, which would otherwise revert the
already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
baseline: a fresh brain reports generation()===0 and an empty
transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
(generation()), so un-flushed single-op writes are overlaid too.
Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
(a coordinator's fair-share input). Per-generation bytes recorded in each
delta enable historyBytes() introspection without a storage size API.
Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
This commit is contained in:
parent
afac7f9662
commit
5c3bb2c864
15 changed files with 1207 additions and 218 deletions
|
|
@ -1538,33 +1538,49 @@ export interface BrainyConfig {
|
|||
}
|
||||
|
||||
/**
|
||||
* Generational-history retention policy (8.0 MVCC).
|
||||
* Generational-history retention policy (8.0 MVCC — the `retention` knob).
|
||||
*
|
||||
* Every `transact()` produces an immutable generation record-set that
|
||||
* serves historical reads (`asOf()`, pinned `Db` values). Without
|
||||
* compaction those record-sets accumulate forever, so Brainy
|
||||
* **auto-compacts on every `flush()` and `close()`** with this policy.
|
||||
* A generation is reclaimed only when it is older than BOTH bounds
|
||||
* (and never while a live `Db` pin protects it):
|
||||
* Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
|
||||
* `remove`/`relate`) produces an immutable generation record-set serving
|
||||
* historical reads (`asOf()`, pinned `Db` values). Without compaction those
|
||||
* accumulate, so Brainy **auto-compacts on every `flush()` and `close()`**.
|
||||
* Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
|
||||
*
|
||||
* - `retainGenerations` — keep at least the N most recent committed
|
||||
* generations (default: `100`).
|
||||
* - `retainMs` — keep everything committed within the window
|
||||
* (default: 7 days).
|
||||
* Modes:
|
||||
* - **unset → ADAPTIVE (default):** the retention horizon tracks
|
||||
* disk/RAM pressure. A machine-level byte budget governs how much history
|
||||
* is kept — driven by `budgetBytes` when a coordinator (e.g. cor's
|
||||
* `ResourceManager`, fair-shared across co-located instances) sets it, else
|
||||
* a local `os.freemem`/filesystem-free probe. Oldest-unpinned history is
|
||||
* reclaimed when the budget is exceeded. Zero-config.
|
||||
* - **`'all'` → unbounded:** never reclaim history (index compaction for
|
||||
* query speed still runs — the decouple). The legacy keep-everything
|
||||
* behavior, now explicit opt-in.
|
||||
* - **`{ maxGenerations?, maxAge?, maxBytes? }` → explicit CAPS:** reclaim
|
||||
* the oldest unpinned generations while ANY supplied cap is exceeded
|
||||
* (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
|
||||
*
|
||||
* Escape hatches: raise either bound for longer time-travel windows,
|
||||
* or set `autoCompact: false` to manage history manually via
|
||||
* `brain.compactHistory()`. Long-term archives belong in
|
||||
* `db.persist(path)` snapshots, which compaction never touches.
|
||||
* `autoCompact: false` disables the automatic flush/close compaction (manage
|
||||
* manually via `brain.compactHistory()`). `budgetBytes` is the settable
|
||||
* adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
|
||||
* Long-term archives belong in `db.persist(path)` snapshots, which compaction
|
||||
* never touches.
|
||||
*/
|
||||
history?: {
|
||||
/** Keep at least this many recent committed generations (default: 100). */
|
||||
retainGenerations?: number
|
||||
/** Keep generations committed within this window in ms (default: 604800000 = 7 days). */
|
||||
retainMs?: number
|
||||
/** Run compaction automatically on flush()/close() (default: true). */
|
||||
autoCompact?: boolean
|
||||
}
|
||||
retention?:
|
||||
| 'all'
|
||||
| 'adaptive'
|
||||
| {
|
||||
/** Keep at most this many recent committed generations (cap). */
|
||||
maxGenerations?: number
|
||||
/** Keep generations committed within this window in ms (cap). */
|
||||
maxAge?: number
|
||||
/** Keep total generational-history bytes at or below this (cap). */
|
||||
maxBytes?: number
|
||||
/** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
|
||||
budgetBytes?: number
|
||||
/** Run compaction automatically on flush()/close() (default: true). */
|
||||
autoCompact?: boolean
|
||||
}
|
||||
|
||||
// Memory management options
|
||||
maxQueryLimit?: number // Override auto-detected query result limit (max: 100000)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue