feat(8.0): Model-B per-write generation-stamping + adaptive retention knob
Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.
Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
deferred durability. Wired into add/update/remove/relate/updateRelation/
unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
immediately; its before-image is buffered in an in-memory pending tier that
resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
synchronous now() freezes with no forced flush. One fsync per window
(triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
groupCommit:true): a crash mid-flush discards the partial generation and
never restores its before-images, which would otherwise revert the
already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
baseline: a fresh brain reports generation()===0 and an empty
transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
(generation()), so un-flushed single-op writes are overlaid too.
Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
(a coordinator's fair-share input). Per-generation bytes recorded in each
delta enable historyBytes() introspection without a storage size API.
Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
This commit is contained in:
parent
afac7f9662
commit
5c3bb2c864
15 changed files with 1207 additions and 218 deletions
|
|
@ -58,7 +58,7 @@ async function mkBrain(backend: 'memory' | 'filesystem', dir?: string): Promise<
|
|||
storage,
|
||||
eagerEmbeddings: false, // never load WASM — we pass precomputed vectors
|
||||
// Keep history fully retained for the measurement (no mid-run compaction skewing depth).
|
||||
history: { autoCompact: false } as any,
|
||||
retention: { autoCompact: false } as any,
|
||||
index: { m: 16, efConstruction: 200, efSearch: 50 },
|
||||
cache: { maxSize: 1000, ttl: 3600 }
|
||||
} as BrainyConfig)
|
||||
|
|
@ -176,13 +176,13 @@ async function main() {
|
|||
await b.flush()
|
||||
const beforeBytes = allocBytes(`${dir}/_generations`)
|
||||
const t = process.hrtime.bigint()
|
||||
const res = await b.compactHistory({ retainGenerations: 100 } as any)
|
||||
const res = await b.compactHistory({ maxGenerations: 100 } as any)
|
||||
const compMs = Number(process.hrtime.bigint() - t) / 1e6
|
||||
await b.flush()
|
||||
const afterBytes = allocBytes(`${dir}/_generations`)
|
||||
await b.close()
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
say(`N=${N.toLocaleString()} gens → compactHistory({retainGenerations:100}): ${compMs.toFixed(0)} ms, removed ${(res as any)?.removedGenerations ?? '?'} gens`)
|
||||
say(`N=${N.toLocaleString()} gens → compactHistory({maxGenerations:100}): ${compMs.toFixed(0)} ms, removed ${(res as any)?.removedGenerations ?? '?'} gens`)
|
||||
say(` _generations footprint: ${(beforeBytes / 1024 / 1024).toFixed(1)} MiB → ${(afterBytes / 1024 / 1024).toFixed(1)} MiB allocated`)
|
||||
results.compaction.push({ N, compactMs: compMs, removed: (res as any)?.removedGenerations, beforeBytes, afterBytes })
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue