2026-06-22 13:47:33 -07:00
/ * *
* @module tests / unit / db / generation - chain
* @description Locks in the Model - B scalability fixes in ` generationStore.ts ` :
* the per - id history chains that make ` resolveAt ` O ( log ) instead of an
* O ( database - age ) scan of the global ` committedGens ` , the bounded ` deltaCache `
* ( evicted deltas are transparently re - read from storage ) , and chain
* maintenance across commit + compaction . These assert the OBSERVABLE behavior
* — ` asOf() ` correctness — across the code paths those fixes touch , including the
* edge cases the existing temporal suites don ' t reach ( eviction past the cap ,
* and chain rebuild after compaction ) .
* /
import { describe , it , expect , beforeEach , afterEach } from 'vitest'
import { Brainy } from '../../../src/index.js'
import { NounType } from '../../../src/types/graphTypes.js'
import { createTestConfig } from '../../helpers/test-factory.js'
const X = '11111111-1111-4111-8111-111111111111'
const Y = '22222222-2222-4222-8222-222222222222'
describe ( 'generation history chains (Model-B scalability)' , ( ) = > {
let brain : Brainy
beforeEach ( async ( ) = > {
brain = new Brainy ( createTestConfig ( ) )
await brain . init ( )
} )
afterEach ( async ( ) = > {
await brain . close ( )
} )
/** Seed X (its own committed generation), then return g0 = that generation. */
async function seedX ( ) : Promise < number > {
const db = await brain . transact ( [
{ op : 'add' , id : X , type : NounType . Document , subtype : 'note' , data : 'x' , metadata : { v : 0 } }
] )
await db . release ( )
return brain . generation ( )
}
async function bumpX ( v : number ) : Promise < number > {
const db = await brain . transact ( [ { op : 'update' , id : X , metadata : { v } } ] )
await db . release ( )
return brain . generation ( )
}
const vAt = async ( gen : number ) : Promise < number | undefined > = > {
const db = await brain . asOf ( gen )
const e = ( await db . get ( X ) ) as any
await db . release ( )
return e ? . metadata ? . v
}
it ( 'resolveAt returns the value as-of each pinned generation (chain binary search)' , async ( ) = > {
const g0 = await seedX ( )
const g1 = await bumpX ( 1 )
const g2 = await bumpX ( 2 )
expect ( await vAt ( g0 ) ) . toBe ( 0 ) // before-image of the g1 update = the seed value
expect ( await vAt ( g1 ) ) . toBe ( 1 ) // before-image of the g2 update
expect ( ( ( await brain . get ( X ) ) as any ) ? . metadata ? . v ) . toBe ( 2 ) // live head
expect ( g2 ) . toBeGreaterThan ( g1 )
expect ( g1 ) . toBeGreaterThan ( g0 )
} )
it ( 'stays correct when an UNCHANGED entity is read at an old pin (no chain → O(1) current)' , async ( ) = > {
const g0 = await seedX ( )
// Churn a DIFFERENT entity many times; X is never touched again.
const db = await brain . transact ( [ { op : 'add' , id : Y , type : NounType . Document , subtype : 'note' , data : 'y' , metadata : { v : 0 } } ] )
await db . release ( )
for ( let i = 1 ; i <= 30 ; i ++ ) {
const d = await brain . transact ( [ { op : 'update' , id : Y , metadata : { v : i } } ] )
await d . release ( )
}
// X has no chain entry after g0 → resolveAt returns 'current' without scanning the 31 Y-generations.
expect ( await vAt ( g0 ) ) . toBe ( 0 )
expect ( ( ( await brain . get ( X ) ) as any ) ? . metadata ? . v ) . toBe ( 0 )
} )
it ( 'survives deltaCache eviction (cap lowered; evicted deltas re-read from storage)' , async ( ) = > {
; ( brain as any ) . generationStore . deltaCacheMax = 4 // force eviction well before the gen count
const g0 = await seedX ( )
for ( let v = 1 ; v <= 12 ; v ++ ) await bumpX ( v ) // 12 gens > cap 4 → evictions
perf(8.0): bound per-id generation history chains (O(W+L) resident, was O(N))
The MVCC time-travel layer kept nounChains/verbChains as unbounded
Map<id, number[]> — one resident chain per id ever touched across retained
history (O(N) RAM, defeating billion-scale time travel). Replace with a hot-tail
window + bounded cold LRU + a mutex-free bulk resolver, keeping resolveAt exactly
correct.
The most-recent W generations stay resident as full chains (the common recent-pin
read is O(log), zero scan); deeper pins reconstruct one id's chain on demand into
an L-bounded LRU. Reconstruction is LOCK-LIGHT — it holds no commit mutex, so a
historical read never stalls writers; correctness holds because a live pin's
answer is always > minPinnedGeneration, which compaction can never reclaim, and a
concurrently-reclaimed gen below that is skipped. materializeAtGeneration routes
through a new mutex-free resolveManyAt (one forward pass, O(R + |ids|)) — without
it a deep-pin materialize both regresses to O(N*R) and deadlocks on snapshotWith's
mutex. The cold cache is invalidated on re-touch to stay coherent. Resident RAM is
O(W*d + L), independent of N. 11-case test (oracle-vs-bruteforce, held-Db across
eviction + concurrent compaction, no-deadlock, write-path I/O-free); unit 1735 +
integration 613 (db-mvcc/db-temporal/db-asof green).
2026-06-30 10:30:17 -07:00
// asOf still resolves correctly (the window build re-read every evicted delta)…
2026-06-22 13:47:33 -07:00
expect ( await vAt ( g0 ) ) . toBe ( 0 )
// …and a range op (since scans committedGens via getDelta, re-reading evicted deltas) still works.
const now = await brain . now ( )
const changed = await now . since ( g0 )
await now . release ( )
expect ( changed . nouns ) . toContain ( X )
} )
it ( 'rebuilds chains after compaction and keeps asOf correct above the horizon' , async ( ) = > {
const g0 = await seedX ( )
const g1 = await bumpX ( 1 )
const g2 = await bumpX ( 2 )
await bumpX ( 3 )
await bumpX ( 4 )
// Reclaim everything except the 2 most recent committed generations.
feat(8.0): Model-B per-write generation-stamping + adaptive retention knob
Every write — transact() AND single-op add/update/remove/relate — is now its
own immutable generation (Model-B), so a now() pin always freezes and
asOf/since/diff/history/transactionLog reflect single-ops exactly like
transacts. Closes the Model-A hole where pins did not freeze against single-op
writes.
Generation-stamping:
- GenerationStore.commitSingleOp: a one-operation commitTransaction with
deferred durability. Wired into add/update/remove/relate/updateRelation/
unrelate + removeMany (the *Many and VFS paths delegate to these).
- Async group-commit (flushPendingSingleOps): the live write is acknowledged
immediately; its before-image is buffered in an in-memory pending tier that
resolveAt/chains/changedBetween/tx-log read like on-disk generations, so the
synchronous now() freezes with no forced flush. One fsync per window
(triggers: size / 50ms timer / flush / close / transact / compactHistory).
- Crash recovery is drop-without-restore for group-commit generations (marked
groupCommit:true): a crash mid-flush discards the partial generation and
never restores its before-images, which would otherwise revert the
already-acknowledged live write.
- Init-time infrastructure (the VFS root) is the un-versioned generation-0
baseline: a fresh brain reports generation()===0 and an empty
transactionLog(); the first user write is generation 1.
- Historical find()/related() overlay bound is the full reserved watermark
(generation()), so un-flushed single-op writes are overlaid too.
Retention knob:
- config `history` -> `retention`: 'all' | 'adaptive' |
{ maxGenerations?, maxAge?, maxBytes?, budgetBytes?, autoCompact? }; unset ->
adaptive (disk/RAM pressure, zero-config). CompactHistoryOptions floors ->
caps (retainGenerations->maxGenerations, retainMs->maxAge, +maxBytes):
reclaim oldest-unpinned while ANY cap is exceeded; pins always exempt.
- brain.setRetentionBudget(bytes) drives the adaptive byte budget at runtime
(a coordinator's fair-share input). Per-generation bytes recorded in each
delta enable historyBytes() introspection without a storage size API.
Tests: per-write generation resolution, pin freeze vs add/update/remove,
drop-without-restore corruption-trap (fault injector), clean-reopen replay,
maxBytes/maxAge/no-cap reclamation, retention-then-reopen. 107 db/generation/
temporal tests green, tsc clean. Docs (ADR-001, consistency-model, snapshots
guide, api reference, RELEASES) updated to per-write granularity + retention.
2026-06-22 15:19:58 -07:00
const res = await brain . compactHistory ( { maxGenerations : 2 } )
2026-06-22 13:47:33 -07:00
expect ( res . removedGenerations ) . toBeGreaterThan ( 0 )
// A pin ABOVE the new horizon still resolves (chains were rebuilt from the survivors)…
const live = ( ( await brain . get ( X ) ) as any ) ? . metadata ? . v
expect ( live ) . toBe ( 4 )
// …and the reclaimed-depth pins are below the horizon now.
expect ( res . horizon ) . toBeGreaterThanOrEqual ( g0 )
expect ( g2 ) . toBeGreaterThan ( g1 )
} )
} )