soulcraft/brainy

Fork 0

David Snelling cc8037db10 docs(8.0): consistency-model concept + snapshots guide — Db API replaces branching docs

2026-06-11 08:37:26 -07:00

10 KiB

Raw Blame History

title

slug

public

Snapshots & Time Travel

Brainy 8.0 treats the database as a value: brain.now() pins the current state as an immutable Db, brain.transact() commits an atomic batch and hands you the resulting value, brain.asOf() opens past state, and db.persist() cuts a self-contained snapshot. This guide is the recipe book. The precise guarantees behind every recipe live in the consistency model.

Instant backup

Pin the current state, persist it, release:

const db = brain.now()
try {
  await db.persist('/backups/2026-06-11')
} finally {
  await db.release()
}

On filesystem storage the snapshot is built from hard links: every data file in Brainy is immutable-by-rename, so the snapshot is created without copying entity data and shares disk space with the live store. Later writes can never alter it — a rewrite swaps the inode, the snapshot keeps the old bytes. Cross-device targets fall back to per-file byte copies, and persisting an in-memory brain serializes it to the same directory layout — a real, durable store.

Two things to know:

persist() requires the view to still be the store's latest generation. If something committed after your pin, it throws GenerationConflictError instead of snapshotting the wrong state — pin and persist before further writes, or retry with a fresh brain.now().
The target directory must be empty or absent.

For scheduled backups, this loop is the whole job:

const db = brain.now()
try {
  await db.persist(`/backups/${new Date().toISOString().slice(0, 10)}`)
} finally {
  await db.release()
}

Restore

restore() replaces the store's entire current state from a snapshot — entities, relationships, indexes, history. It is deliberately loud about it:

await brain.restore('/backups/2026-06-11', { confirm: true })

{ confirm: true } is mandatory — current state is destroyed.
The snapshot is copied in (never linked), so it stays independent and can be restored again later.
All indexes are rebuilt from the restored records.
The generation counter is floored at its pre-restore value, so generation numbers you observed before the restore are never reissued.
Live Db pins do not survive a restore — release them first.

Open a snapshot read-only

You do not have to restore to look inside a snapshot. Brainy.load() opens it as a self-contained read-only store with the full query surface, including vector search:

const db = await Brainy.load('/backups/2026-06-11')

const hits = await db.search('unpaid invoices from the spring campaign')
const orders = await db.find({ type: NounType.Document, subtype: 'order' })

await db.release()   // closes the underlying read-only instance

brain.asOf('/backups/2026-06-11') does the same from an existing brain. This is also the 8.0 answer to "named branches": a branch is a name → path mapping your application keeps, where each path is a persisted snapshot. Need a writable copy? Restore the snapshot into a fresh data directory and open a writer on it — instead of switching a shared store between branches in place, every line of code always sees exactly the store it opened.

Time-travel debugging

When production data looks wrong, query the past directly — by wall-clock time or by generation:

// What did this order look like yesterday?
const yesterday = await brain.asOf(new Date(Date.now() - 86_400_000))
const before = await yesterday.get(orderId)

// Full queries work at any reachable generation — search, graph, filters:
const thenActive = await yesterday.find({
  type: NounType.Document,
  subtype: 'order',
  where: { status: 'active' }
})

await yesterday.release()

Pin two points in time and diff them:

const before = await brain.asOf(1041)
const after = brain.now()

const changed = await after.since(before)
changed.nouns   // entity ids touched by transactions in between
changed.verbs   // relationship ids touched in between

await before.release()
await after.release()

Three things to remember:

History granularity is transact() commits — single-operation writes advance the clock but do not produce historical records (see the consistency model). Use transact() for writes you want to travel back through.
The first index-accelerated query (semantic search, traversal, cursors, aggregation) at a historical generation builds an in-memory index materialization — O(n at that generation), once per Db, freed on release(). Metadata-level reads are free.
Generations reclaimed by compactHistory() throw GenerationCompactedError — persist anything you need to keep forever.

Safe schema migration

brain.migrate() integrates with snapshots directly: pass backupTo and a hard-link snapshot of the current generation is persisted before any transform runs:

const result = await brain.migrate({ backupTo: '/backups/pre-migration-8.0' })
console.log(result.migrationsApplied, result.backupPath)

// If the migration went wrong, roll the whole store back:
await brain.restore('/backups/pre-migration-8.0', { confirm: true })

The same persist-before-mutate pattern works for any risky bulk operation, not just migrations:

const pin = brain.now()
try {
  await pin.persist('/backups/pre-bulk-edit')
} finally {
  await pin.release()
}
await runRiskyBulkEdit(brain)

What-if analysis

db.with(ops) applies a transaction speculatively, in memory — nothing touches disk, the generation counter, or the indexes. Ask "what would the store look like if…", then commit the same operations for real:

const ops = [
  { op: 'update', id: employeeId, metadata: { team: 'platform' } },
  { op: 'relate', from: employeeId, to: milestoneId, type: VerbType.ParticipatesIn, subtype: 'assignment' }
]

const base = brain.now()
const whatIf = await base.with(ops)

await whatIf.get(employeeId)                          // sees the change
await whatIf.find({ where: { team: 'platform' } })    // metadata finds work
await whatIf.related(employeeId)                      // overlay relations included

await whatIf.release()
await base.release()

// Looks right — make it real, atomically:
await brain.transact(ops)

The boundary: speculative entities carry no embeddings (with() never invokes the embedder), so semantic search, traversal, cursors, aggregation, and persist() throw SpeculativeOverlayError on overlay views instead of returning silently incomplete results. get(), metadata-filter find(), and filter-based related() are fully supported. Overlays chain — calling with() on an overlay stacks another layer.

Audit trails

transact() reifies transaction metadata: whatever you pass as meta is recorded durably alongside the committed generation and timestamp, readable via brain.transactionLog():

await brain.transact(
  [{ op: 'update', id: invoiceId, metadata: { status: 'approved' } }],
  { meta: { author: 'approvals-service', actor: 'jane@example.com', reason: 'PO-7741' } }
)

const log = await brain.transactionLog({ limit: 20 })   // newest first
// [{ generation: 1042, timestamp: 1765432100000, meta: { author: 'approvals-service', ... } }]

Combine the log with asOf() to reconstruct exactly what any transaction did:

const [entry] = await brain.transactionLog({ limit: 1 })

const after = await brain.asOf(entry.generation)
const before = await brain.asOf(entry.generation - 1)

const touched = await after.since(before)
for (const id of touched.nouns) {
  console.log(id, await before.get(id), '→', await after.get(id))
}

await before.release()
await after.release()

For per-entity write coordination (rather than whole-store history), the _rev counter and ifRev CAS remain the right tool — see optimistic concurrency.

Keeping history bounded

Historical records cost disk space. Reclaim what no live pin protects:

await brain.compactHistory({
  retainGenerations: 100,            // keep the 100 most recent commits
  retainMs: 7 * 24 * 60 * 60 * 1000  // and everything from the last 7 days
})

Compaction never breaks a pinned read — record-sets are reclaimed only when no live Db could need them. Release views you are done with (including the ones transact() returns), and persist() any generation you want to keep beyond the retention window: snapshots are self-contained and unaffected by compaction.

From branches to values

If you used the pre-8.0 fork/checkout/commit/versions surface, every use case maps to a sharper tool:

Pre-8.0 habit	8.0 recipe
`fork()` to experiment safely	`db.with(ops)` for speculation in memory; a restored snapshot in a fresh directory for a long-lived writable copy
`commit()` checkpoints	`transact(ops, { meta })` — every batch is an atomic, logged, time-travelable commit
`checkout()` to switch branches	Open the snapshot you want — `Brainy.load(path)` read-only, or restore into its own directory. No in-place switching: every handle always sees one unambiguous store.
`getHistory()`	`brain.transactionLog()` + `db.since(priorDb)`
`versions.save()` per-entity snapshots	A pinned `Db` or persisted snapshot captures every entity at that moment; `asOf()` reads any entity's past state
`versions.restore()`	`brain.restore(snapshot, { confirm: true })` for the whole store, or read the old entity via `asOf()` and write it back with `transact()`
Backup branches	`db.persist(path)` — instant, hard-link-shared, self-contained

10 KiB Raw Blame History