feat: two-tier history reads + the repacker + generationDigest — D1+D3 wired end-to-end

The packed tier goes live inside GenerationStore:

- Two-tier reads: getDelta / readBeforeImage / readGenerationRecords
  fall through live-tier → sealed segments (live-tier-wins: a crash
  mid-fold leaves a duplicate representation, never a gap). Cold-open
  seeds committedRanges from the segment manifest via interval merge —
  packed generations resolve without their directories existing.
- repackHistory({timeBudgetMs, batchGenerations}): folds cold
  generations (older than the newest 1024) oldest-first into sealed
  segments, deleting per-generation directories only after segment +
  manifest are durable. Public API + automatic time-bounded pass at
  close() (before compaction, so reclaim can drop whole segments);
  re-representation only — the sole history transform under the
  archival profile. Early stop = consistent prefix, next pass resumes.
- compact(): packed generations reclaim logically in the loop and
  physically at whole-segment boundaries via dropSegmentsBelow (the
  frozen partial-segments-wait rule).
- generationDigest(g) (D8): deterministic content digest through g —
  sealed-segment checksum chain + live-tier delta hashes; O(segments +
  live window); RangeError out of range, GenerationCompactedError
  below the horizon (a gate can never silently pin reclaimed history).

Four end-to-end pins: asOf answers byte-identical across fold + cold
reopen with folded dirs physically gone; repack+reclaim composition;
digest reopen-stability/divergence/loud-horizon; budget no-op+resume.

CHANGELOG.md

@@ -2,6 +2,191 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19)
+
+- docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78)
+- fix: release drains in-flight writer-lock heartbeat — no phantom lock after unlink (70e4bc8)
+- feat: flush() never compacts — history maintenance moves to close() with bounded passes (300d9f2)
+
+
+### [8.8.2](https://github.com/soulcraftlabs/brainy/compare/v8.8.1...v8.8.2) (2026-07-19)
+
+- fix: one field-resolution law across aggregation hooks, source.where, removeMany, and find() spellings (945d92d)
+- chore: push public docs to the soulcraft.com ingest door on release (42037d0)
+
+
+### [8.8.1](https://github.com/soulcraftlabs/brainy/compare/v8.8.0...v8.8.1) (2026-07-18)
+
+- fix: O(1) adaptive retention accounting + historyStats fleet audit (6207e48)
+- fix: import dedup off-switch honesty + brain-owned lifecycle for the background pass (4fcef7b)
+
+
+### [8.8.0](https://github.com/soulcraftlabs/brainy/compare/v8.7.1...v8.8.0) (2026-07-17)
+
+- feat: OS-limit detection for pool-scale deployments (16a73b8)
+
+
+### [8.7.1](https://github.com/soulcraftlabs/brainy/compare/v8.7.0...v8.7.1) (2026-07-17)
+
+- fix: race-proof writer-lock acquisition + machine-readable conflict through init (01a3b46)
+
+
+### [8.7.0](https://github.com/soulcraftlabs/brainy/compare/v8.6.0...v8.7.0) (2026-07-17)
+
+- feat: scaled transact budgets + labeled timeout diagnostics + envelope docs (6ef9fcb)
+
+
+### [8.6.0](https://github.com/soulcraftlabs/brainy/compare/v8.5.2...v8.6.0) (2026-07-17)
+
+- feat: brain.auditGraph() — read-only graph-truth audit (2a03fae)
+
+
+### [8.5.2](https://github.com/soulcraftlabs/brainy/compare/v8.5.1...v8.5.2) (2026-07-17)
+
+- fix: exception-safe aggregation backfill + generation-verified adoption + loud open-path guards (a77b064)
+
+
+### [8.5.1](https://github.com/soulcraftlabs/brainy/compare/v8.5.0...v8.5.1) (2026-07-17)
+
+- fix: aggregation state adoption on reopen + single-flight backfill + query-cap ratchet removal (da55be7)
+- docs: external-backups/sparse-storage guide + generation fact log concept (593bb8b)
+
+
+### [8.5.0](https://github.com/soulcraftlabs/brainy/compare/v8.4.0...v8.5.0) (2026-07-15)
+
+- test: tolerant timing assertion in the execution-time measure test (4dc0a92)
+- feat: committedGeneration capability + pinned durability/stability contracts (d1ecee1)
+- docs: RELEASES.md entry for 8.5.0 (provider fact-log access + shared verifier) (e4f37cd)
+- feat: provider access to the fact log + shared stamp verifier via internals (352e356)
+
+
+### [8.4.0](https://github.com/soulcraftlabs/brainy/compare/v8.3.3...v8.4.0) (2026-07-15)
+
+- docs: RELEASES.md entry for 8.4.0 (generation fact log + family stamp) (4a60b43)
+- feat: entity-tree family stamp — sourceGeneration + rollup coherence at open (2888ae6)
+- feat: generation fact log — after-image commit records, dual-written at every commit point (38b0041)
+
+
+### [8.3.3](https://github.com/soulcraftlabs/brainy/compare/v8.3.2...v8.3.3) (2026-07-15)
+
+- docs: RELEASES.md entry for 8.3.3 (rename containment fix + repair) (c3feafd)
+- test: lens-consistency regression — combined vs subtype-only vs canonical ground truth (4fb41f9)
+- fix: VFS rename moves the containment edge — no ghost in the old directory (af8c179)
+
+
+### [8.3.2](https://github.com/soulcraftlabs/brainy/compare/v8.3.1...v8.3.2) (2026-07-14)
+
+- docs: RELEASES.md entry for 8.3.2 (honest counters) (0932ecd)
+- fix: honest counters — removal never re-reads the removed record + repairIndex recounts and persists all rollups (2e2ba9c)
+
+
+### [8.3.1](https://github.com/soulcraftlabs/brainy/compare/v8.3.0...v8.3.1) (2026-07-14)
+
+- docs: RELEASES.md entry for 8.3.1 (full-removal deletes + family-scoped gate) (c0c68ac)
+- fix: full-removal canonical deletes + family-scoped migration gate (366f9a9)
+- docs: cite the cross-layer integrity contract generically in comments and notes (1d26988)
+
+
+### [8.3.0](https://github.com/soulcraftlabs/brainy/compare/v8.2.8...v8.3.0) (2026-07-13)
+
+- docs: RELEASES.md entry for 8.3.0 (heal-cost + cross-layer integrity contract) (7692c6f)
+- perf: parallel + id-only canonical enumeration (heal-cost dominant term) (ec5b933)
+- feat: registered-blob family contract — declared index blobs are undeletable (bfa1762)
+- feat: validateIndexConsistency delegates to provider invariants (6bcb54f)
+
+
+### [8.2.8](https://github.com/soulcraftlabs/brainy/compare/v8.2.7...v8.2.8) (2026-07-13)
+
+- fix: honest index readiness — no silently-empty queries on a cold index (d0f69c7)
+
+
+### [8.2.7](https://github.com/soulcraftlabs/brainy/compare/v8.2.6...v8.2.7) (2026-07-13)
+
+- fix: restore loadBinaryBlob fault-propagation (native column-store lockstep) (b6c7039)
+
+
+### [8.2.6](https://github.com/soulcraftlabs/brainy/compare/v8.2.5...v8.2.6) (2026-07-13)
+
+- docs: RELEASES.md entry for 8.2.6 (write/index-spine hardening) (a873852)
+- chore: hold loadBinaryBlob fault-propagation for the cortex column-store lockstep (36c10c1)
+- fix: aggregation surfaces materialize/state-load failures loudly (02eff64)
+- fix: surface a degraded derived index on reads instead of serving it silently (ba958d9)
+- fix: saveBinaryBlob never acks a durable write that stored nothing (7feba49)
+- fix: refuse writes when single-op history cannot be made durable (54c1836)
+- fix: clear() wipes the full native/derived footprint, not a subset (d8301f8)
+- fix: surface segment/entity read faults loudly instead of masking as absent (af5d2f3)
+- fix: spine hardening pass 1 (part) — count symmetry, honest partial-load, flush durability, read-fault propagation (119087a)
+- test: pin the read-your-writes contract under the single writer (eb9c4eb)
+
+
+### [8.2.5](https://github.com/soulcraftlabs/brainy/compare/v8.2.4...v8.2.5) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.5 (honest rollback-failure response) (a7c7aa5)
+- fix: honest response when a transaction rollback cannot complete (711d2f0)
+
+
+### [8.2.4](https://github.com/soulcraftlabs/brainy/compare/v8.2.3...v8.2.4) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.4 (non-destructive restore) (4574695)
+- fix: non-destructive, crash-resumable restore (a2f4f6a)
+
+
+### [8.2.3](https://github.com/soulcraftlabs/brainy/compare/v8.2.2...v8.2.3) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.3 (transact durability barrier) (be5ce0b)
+- fix: transact durability barrier — committed transactions are durable on return (3b8fa51)
+
+
+### [8.2.2](https://github.com/soulcraftlabs/brainy/compare/v8.2.1...v8.2.2) (2026-07-11)
+
+- docs: RELEASES.md entry for 8.2.2 (transaction timeout rollback) (ed97006)
+- fix: transaction timeout rolls back applied operations (no torn state) (508a8e3)
+
+
+### [8.2.1](https://github.com/soulcraftlabs/brainy/compare/v8.2.0...v8.2.1) (2026-07-10)
+
+- test: update graph-index operation constructors to the VerbEndpointInts signature (62a449d)
+- docs: RELEASES.md entry for 8.2.1 (transact forward-ref parity fix) (7089782)
+- fix: transact forward references resolve graph endpoint ints at execute time (a175406)
+
+
+### [8.2.0](https://github.com/soulcraftlabs/brainy/compare/v8.1.0...v8.2.0) (2026-07-10)
+
+- docs: RELEASES.md entry for 8.2.0 (temporal VFS) (98ceadc)
+- feat: temporal VFS — file content joins the Model-B immutability model (a3467e1)
+- docs: pin the write-path invariant in the plugin contract (the onChange change-feed guarantee) (4af8fb3)
+
+
+### [8.1.0](https://github.com/soulcraftlabs/brainy/compare/v8.0.17...v8.1.0) (2026-07-10)
+
+- docs: RELEASES.md entry for 8.1.0 (brain.onChange change feed) (4e9be08)
+- feat: brain.onChange — the in-process change feed for every committed mutation (fd5edb5)
+
+
+### [8.0.17](https://github.com/soulcraftlabs/brainy/compare/v8.0.16...v8.0.17) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.17 (canonical count recovery + dead-machinery sweep) (6b8b9cb)
+- fix: count recovery scans the canonical layout; remove the dead 7.x hnsw sharding machinery (352e2da)
+
+
+### [8.0.16](https://github.com/soulcraftlabs/brainy/compare/v8.0.15...v8.0.16) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.16 (atomic ifAbsent/upsert + exact blob refCounts) (54e7c0e)
+- fix: atomic ifAbsent/upsert inserts + exact blob reference counts under concurrency (867939e)
+
+
+### [8.0.15](https://github.com/soulcraftlabs/brainy/compare/v8.0.14...v8.0.15) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.15 (atomic ifRev CAS) (b1fe25a)
+- fix: ifRev CAS is atomic — the revision check now runs under the commit mutex (9a3d1bd)
+
+
+### [8.0.14](https://github.com/soulcraftlabs/brainy/compare/v8.0.13...v8.0.14) (2026-07-07)
+
+- docs: RELEASES.md entry for 8.0.14 (migration preserves branch-scoped non-entity state) (64188a3)
+- fix: 7→8 migration preserves branch-scoped non-entity state instead of deleting it (a93bb4e)
+
+
 ### [8.0.13](https://github.com/soulcraftlabs/brainy/compare/v8.0.12...v8.0.13) (2026-07-07)
 
 - docs: RELEASES.md entry for 8.0.13 (accurate boot log for established stores) (38e8de5)

RELEASES.md

@@ -8,9 +8,853 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so
 - Debugging data, query, or storage behaviour
 - A new Brainy feature is available that you want to adopt
 
+## Removed APIs — 7.x → 8.x (the complete ledger)
+
+Every public API removed at the 8.0 major, with its sanctioned replacement. If your code
+still calls a left-column name on 8.x it throws (or the config key is rejected) — the
+replacement is always a one-line change. (Standing contract from 8.9.0 forward: removals
+happen only at majors, after ≥1 minor of loud runtime deprecation naming the replacement.)
+
+| Removed (7.x) | Replacement (8.x) |
+|---|---|
+| `brain.search(query, k)` | `find({ query })` — semantic; `find({ query, searchMode })` for hybrid |
+| `brain.getRelations({...})` | `related(id, opts)` for adjacency; `find({ connected: {...} })` for scoped traversal |
+| `brain.neural()` clustering | `find({ vector })` + aggregation `GROUP BY` |
+| `Db.search()` | `db.find({ vector })` |
+| Pre-8.0 storage path aliases (`directory`, `basePath`, …) | one `storage.path` key (old aliases throw) |
+| Reserved keys inside `metadata` bags (silently remapped in 7.x) | top-level params (`subtype`, `visibility`, `confidence`, `weight`, …) — reserved-in-bag throws |
+| 7.x COW branches layout (`branches/main/`) | generational MVCC (`asOf()`, `now()`, `db.persist(path)`) — on-disk migration is automatic at first 8.x open |
+
+The fork/snapshot family (`brain.snapshot()`, `createSnapshot()`, `restoreSnapshot()`)
+is sometimes cited as a 7.x removal — those methods never existed on 7.x; the 8.0 Db API
+(`asOf`/`persist`/`restore({confirm})`) is their first real implementation.
+
 ---
 
-## v8.0.13 — 2026-07-07 (accurate boot log — established stores no longer report "New installation")
+## v8.9.0 — 2026-07-19 (flush is durability-only: history maintenance moves to close())
+
+The write path stops paying maintenance costs — the last structural piece of the
+flush-storm class (a production deployment measured single writes blocked 25–191s behind
+history reclaim running inline on flush under memory pressure):
+
+- **`flush()` never compacts history.** It persists the current window's deltas and
+  nothing else — its cost no longer depends on history backlog or retention mode, in any
+  configuration. **`close()` is the auto-compaction site** (time-bounded per pass, ~5s;
+  an early stop is a consistent prefix and the next pass resumes).
+- **`compactHistory()` gains `timeBudgetMs`** — bound your own maintenance windows; the
+  same resumable-prefix guarantee applies.
+- **The documented trade**: a long-lived writer that never closes accumulates history
+  until its next explicit `compactHistory()`. Predictable writes, explicit maintenance.
+  If you run bounded retention on an always-on service, schedule a periodic
+  `compactHistory({ ...caps, timeBudgetMs })` in your maintenance window.
+- **New public doc: `docs/performance-envelopes.md`** — measured per-op envelopes
+  (p50/p95 at stated scales, hardware, and backend, with the measuring script cited).
+  Refresh rule going forward: any release touching a measured path re-runs that op's
+  benchmark and updates the envelope in the same release.
+- **New in this file: the Removed APIs 7.x→8.x table** (top of this document) — every
+  removal with its sanctioned replacement, one place, per the engine-currency contract.
+  Standing from here: removals only at majors, after ≥1 minor of loud runtime deprecation.
+
+## v8.8.2 — 2026-07-19 (one field-resolution law: reserved-field aggregates stop drifting)
+
+Four fixes from a consumer conformance audit, all rooted in the same disease — two field-resolution
+regimes where there must be one:
+
+- **Aggregates grouped by a RESERVED field (`subtype`, `visibility`, …) now decrement on
+  delete.** The delete/update hooks fed the aggregation engine a partial entity view (type,
+  service, data, metadata only), so a reserved-field `groupBy` resolved to a nonexistent group
+  on the way DOWN — counts drifted upward forever after any delete, and updates that moved an
+  entity between reserved-field groups double-counted it. The hooks now pass the full-fidelity
+  entity view (every reserved field top-level, the same shape the add path uses). If your
+  deployment derives stats from reserved-field aggregates, re-define those aggregates once
+  after upgrading (a changed definition triggers one rescan) or run them fresh — the drifted
+  persisted counts do not self-heal retroactively.
+- **Aggregation `source.where` on reserved fields now filters** instead of silently matching
+  nothing: the matcher resolves fields through the same resolver `groupBy` uses (top-level
+  standard fields + custom metadata), so `where: { subtype: 'note' }` means what it says.
+- **`removeMany()` refuses empty/invalid selectors loudly.** A bare array passed positionally
+  (`removeMany([id])` instead of `removeMany({ ids: [id] })`), an empty params object, or
+  `ids: []` used to resolve successfully having deleted nothing. All three now throw.
+- **`find()` accepts both where-key spellings.** Metadata is flattened at index time
+  (`metadata.entry.title` indexes as `entry.title`); a `metadata.`-prefixed where key now
+  falls back to its flattened spelling when the prefixed one isn't indexed — the
+  "unindexed field(s), returning []" confusion for storage-shaped spellings is gone. (A
+  literal nested custom key named `metadata` still wins when indexed as spelled.)
+
+## v8.8.1 — 2026-07-18 (flush no longer walks the whole generation history + the import dedup off-switch is now honest)
+
+### The flush-storm fix (production incident, reported by a long-running deployment)
+
+Under the default adaptive retention, **every `flush()` re-walked the entire committed
+generation history** to compute total history bytes for the budget check — O(all
+generations) with disk re-reads past the 4,096-entry delta-cache bound. On a brain with
+70,000+ accumulated generations that turned every write into a full-tail scan (60-100s
+writes), even though the budget (free-RAM-based) never tripped and nothing was ever
+reclaimed. Fixed:
+
+- `historyBytes()` now maintains a **running total**: seeded by one walk on first use,
+  then updated incrementally at every commit and reclaim — the adaptive retention check
+  on every flush is O(1). Invariant regression-pinned (running total ≡ fresh walk through
+  both commit paths and compaction).
+- New **`brain.historyStats()`** (read-only, exported `HistoryStats`): generation count,
+  total on-disk bytes, generation/timestamp range, compaction horizon, retention mode,
+  and the effective adaptive budget — the one-call fleet-audit for sizing retention
+  exposure per brain.
+- Interim guidance for keep-everything deployments already affected: `retention: 'all'`
+  skips the adaptive accounting entirely (and is the correct policy if you never want
+  history reclaimed). The accumulated files are harmless at rest; this release removes
+  the per-write cost of their existence.
+
+### The import dedup off-switch (lifecycle honesty)
+
+The post-import background deduplication pass (a merge-DELETE writer that runs ~5 minutes
+after an import, merging entities judged duplicates by id / name / vector similarity) had
+three lifecycle defects, all fixed:
+
+- **`enableDeduplication: false` now actually disables it.** The background pass was
+  scheduled unconditionally — an import that explicitly opted out could still have
+  entities auto-removed 5 minutes later. The flag now gates BOTH the inline merge and
+  the background pass (regression-pinned).
+- **One deduplicator per brain, owned by the brain.** Each `import()` call constructed its
+  own coordinator + deduplicator, so the "debounced" timer never actually debounced across
+  imports (N imports = N delete timers). The brain now owns a single instance — the
+  debounce genuinely spans imports — and `close()` cancels pending work, so a delete pass
+  can never fire against a closed brain.
+- **The 5-minute timer is unref'd** — a pending pass no longer holds the process open
+  (the exit-hang class; this timer had escaped the earlier sweep).
+
+Retention note for keep-everything deployments: with `enableDeduplication: false` on
+import calls and `retention: 'all'` in config, no engine path removes records
+automatically.
+
+## v8.8.0 — 2026-07-17 (OS-limit detection for pool-scale deployments)
+
+Small minor: brains now detect the two OS limits that bite at pool scale and warn **before**
+the incident instead of during it.
+
+- At open (once per process, Linux-only, measurement-only), Brainy reads `RLIMIT_NOFILE`
+  (soft/hard, from `/proc/self/limits`) and `vm.max_map_count`, and warns loudly when either
+  sits below the pool-scale floors (soft NOFILE < 65 536; max_map_count < 262 144) — with the
+  exact raise commands (`ulimit -n` / `LimitNOFILE=` / `sysctl vm.max_map_count`). On stock
+  defaults the failure otherwise arrives as `EMFILE` or a failed mmap deep inside an index
+  open, long after the real cause stopped being visible. An unreadable limit produces **no**
+  warning — no measurement, no claim (non-Linux platforms stay silent).
+- Exported for ops doors: `checkOsLimits()` returns the full `OsLimitsReport`
+  (values + warnings) programmatically, with the floors exported as constants.
+
+## v8.7.1 — 2026-07-17 (writer-lock acquisition is race-proof + machine-readable through init)
+
+Two hardenings of the multi-process writer lock (the `locks/_writer.lock` lease that makes a
+second writer on the same brain directory fail loudly):
+
+- **Lock acquisition claims atomically.** The acquire path used read-then-write, leaving a
+  window where two processes racing an *absent* lock could both "succeed" — and the loser
+  kept running unlocked, silently. The claim is now an atomic create-exclusive write
+  (`O_EXCL`): exactly one racer wins; the loser re-evaluates and either fails loudly with
+  the winner's details or performs a verified stale-takeover. Bounded retries; contention
+  beyond them fails loudly rather than degrading into a lockless open.
+- **`BRAINY_WRITER_LOCKED` survives `init()`.** The conflict error documents a
+  machine-readable contract (`err.code`, `err.lockInfo` with the holder's pid/host/
+  heartbeat), but init's error wrapping silently stripped both, leaving consumers a message
+  to regex against. The error now passes through unwrapped.
+
+Measured while verifying (for operators sizing audits): `brain.auditGraph()` at a
+production-consumer scale of ~2,600 relationships / 800 entities costs ~0.1 s warm and
+~0.5 s cold, with exact scar counting across reopen.
+
+## v8.7.0 — 2026-07-17 (bulk-transact ergonomics: scaled budgets + timeout telemetry)
+
+The bulk-import ergonomics release, from a consumer's measured production incident (a serial
+import on network-attached storage at ~2 s/op met a flat 30 s transaction budget):
+
+- **The transact apply budget now scales with the batch** — `max(30 s, opCount × 2 s)` — or
+  is exactly what you pass as the new `TransactOptions.timeoutMs`. A flat 30 s cap silently
+  limited honest bulk work to ~15 operations on slow disks while looking generous for small
+  batches. Internal batch paths (e.g. `removeMany` chunks) get the same scaling.
+- **`TransactionTimeoutError` is a diagnosis, not just a failure**: it now reports the
+  operation it stopped at as `i/N` with the operation's name, elapsed vs budgeted time, and
+  states the batch rolled back atomically and is retryable. Its `context` carries the same
+  fields programmatically.
+- **The transact envelope is documented** — batch sizing, budget math, chunking with
+  `ifAbsent` idempotency, and the precompute pattern (`embedBatch` + per-op `vector`) that
+  keeps model inference out of the commit path. Guide: `docs/guides/optimistic-concurrency.md`.
+- Note: `brain.embed()` / `brain.embedBatch()` (the precompute APIs) already ship — public,
+  with native-provider passthrough, verified end-to-end (batch and single paths produce
+  bit-identical vectors; a vector-supplied `add` is fully searchable). Honest measurement:
+  on the default WASM engine, batch throughput ≈ sequential (~160 ms/text) — the precompute
+  win is keeping inference out of the budgeted commit path, not raw embedding speed.
+
+## v8.6.0 — 2026-07-17 (brain.auditGraph — the graph-truth verification instrument)
+
+A minor release adding one new public API, from the fleet's graph-trust program: a read-only
+audit that **proves whether relationship reads return stored truth** on a given brain.
+
+- **`brain.auditGraph(options?)`** walks every canonical relationship record, queries the same
+  read path applications use (`related()` / VFS `readdir`) with all visibility tiers included,
+  and classifies every discrepancy into its failure family: `missingFromReads` (records the
+  read path omits — a stale adjacency index), `danglingEndpoints` (relationships whose endpoint
+  entity no longer exists — the historical partial-delete scar class), and `readOnlyVerbIds`
+  (read-path edges with no stored record — ghosts). Design-hidden internal/system edges are
+  counted separately so intentional hiding is never misclassified as loss. Counts are exact;
+  example lists cap at `maxExamples` with an explicit `truncatedExamples` flag; the result is
+  narrated loudly on incoherence. Mutates nothing — safe on a live brain.
+- The operational pairing: audit → if incoherent, `repairIndex()` → audit again. A `coherent`
+  report after the repair is the verified all-clear. Run it after any engine upgrade, restore,
+  or migration. Guide: `docs/guides/inspection.md`.
+- Also: `getNounIds` pagination now refuses an undecodable resume cursor loudly (the same
+  contract `getNouns`/`getVerbs` gained in 8.5.2 — the third and final walk brought under it).
+- Types exported: `GraphAuditReport`, `GraphAuditDiscrepancy`.
+
+## v8.5.2 — 2026-07-17 (aggregation backfill: exception-safe, generation-verified, and loud)
+
+Hardening patch from a migration incident (a byte-copied store on new hardware; the service
+entered a silent full-CPU loop at boot). Four changes, all in the aggregation engine's
+backfill/adoption path:
+
+- **Backfill walks are exception-safe and non-destructive.** A rescan now builds into a
+  staging map and swaps in atomically on completion; a mid-walk failure drops the staging map,
+  keeps the previous live state serving, and surfaces the storage error to the failing query.
+  Previously the walk wiped live state *before* a scan that could throw, never cleared the
+  pending flag on failure, and re-ran a full walk on every subsequent query — a silent
+  wipe/walk/throw loop at the caller's retry rate.
+- **Failed walks are latched.** After a walk fails, retries within a 30-second cooldown rethrow
+  the recorded error instantly instead of re-walking — a tight caller-side retry loop now costs
+  one loud error per query, never a full store walk per query.
+- **Adoption is generation-verified.** Persisted aggregation state is stamped with the store's
+  committed generation at flush; reopen adoption requires the stamp to equal the current
+  watermark. Stale state (unclean shutdown) or over-counting state (a fact-log truncation on a
+  copied store pulled the watermark back) triggers exactly one loud rescan — never a silent
+  adopt. Pre-8.5.2 state on generation-aware stores rescans once after upgrade, then is stamped.
+- **The path narrates.** Adoption decisions, no-adoptable-state outcomes, walk start/finish
+  (entity count + duration), and walk failures all log by default; a non-advancing storage
+  pagination cursor aborts the walk loudly instead of looping forever.
+
+Plus three guards from a full audit of every loop in the open/init path:
+
+- **Invalid pagination cursors fail loudly.** A supplied-but-undecodable resume token to
+  `getNouns`/`getVerbs` used to silently restart the walk at offset 0 — to a `while(hasMore)`
+  caller that re-serves page 1 forever (an unbounded silent CPU loop). It now throws with a
+  clear message instead.
+- **The graph cold-load verb walk has a stall guard.** `hasMore=true` with a missing or
+  non-advancing cursor aborts loudly instead of re-reading the same page forever.
+- **A derived index AHEAD of the store is named at open.** Brainy already surfaced a provider
+  generation *behind* the committed watermark; the *ahead* direction (the signature of a
+  byte-copy of a live service, or a log truncation during crash recovery) now logs a loud
+  warning explaining what happened and that `brain.repairIndex()` forces a heal — instead of
+  passing unnamed into whatever the derived index does next.
+
+## v8.5.1 — 2026-07-17 (aggregation state survives restarts + the query-cap ratchet removed)
+
+Patch release from a production incident (aggregate/count paths taking 40–90 s on an idle box
+while vector search stayed fast, and every `find({ limit: 5000 })` suddenly failing against an
+"auto-configured query limit of 1000"). Three fixes, one cosmetic:
+
+- **Aggregation state is actually adopted on reopen.** The boot pattern `defineAggregate()` →
+  query raced the engine's async state load: the synchronous define always won, flagged a
+  backfill, and the first query then wiped the just-loaded persisted state and re-walked the
+  entire store — every restart, forever. Reopening with an unchanged definition now adopts the
+  persisted state directly (zero scans); a backfill runs only on a real definition change, a
+  missing/failed state load, or a write that landed before adoption (exactness wins). Apps that
+  rely on persisted definitions without re-defining at boot also no longer race a spurious
+  "Aggregate not defined".
+- **Backfills are single-flight and batched.** Concurrent queries on a cold aggregate used to
+  each wipe the others' partial state and start their own full store walk — under steady query
+  arrival the store never converged (the 40–90 s loop). Now all concurrent queries share one
+  walk, and one walk fills every aggregate pending backfill (M aggregates ≠ M scans).
+- **The query-cap "learning" ratchet is removed.** `maxLimit` is a memory-protection bound, but
+  a hidden tuner shrank it 20 % per recorded query while the lifetime-average query time
+  exceeded 1 s — down to a floor of 1000, below the documented 10 000 auto floor, with no
+  practical recovery, and the resulting error blamed "available free memory" (stale basis
+  label). The cap now comes from its construction-time basis (or your explicit
+  `maxQueryLimit` / `reservedQueryMemory`) alone and never changes at runtime; query timing is
+  recorded for diagnostics only.
+- **Cosmetic:** the engine's own persistence keys (`__aggregation_*`, `brainy:entityIdMapper`)
+  no longer log `[Storage] Unknown key format` at boot — they were always routed correctly;
+  they're now recognized before the warning fires.
+
+Operationally: if a host was bitten, upgrading and restarting is the whole fix — no repair
+ritual needed. Setting `maxQueryLimit` explicitly remains the valve that bypasses auto-detection
+entirely.
+
+## v8.5.0 — 2026-07-15 (provider access to the fact log + the shared stamp verifier)
+
+Small additive follow-up to 8.4.0, from the native accelerator's first consumption pass:
+
+- **Index providers can now reach the fact log through the storage adapter** — new optional
+  capability `storage.scanFacts()` / `storage.factLogHeadGeneration()` / `storage.factSegmentPaths()`,
+  wired by the host brain at init as a closure over its live log. Providers hold only `storage` and
+  must never construct their own fact-log reader (the log's open path is writer-side); `null` means
+  "no fact log here — use the enumeration walk."
+- **The family-stamp verifier is shared** via `@soulcraft/brainy/internals`
+  (`readFamilyStamp` / `writeFamilyStamp` / `verifyFamilyStamp` + types) so first-party native
+  providers run literally the same verification function, never a synchronized copy.
+- **Rollup stamp invariants accept strings** (`number | string`) — content fingerprints such as a
+  per-tree SHA-256 are valid invariant values; a type mismatch reads as incoherence, never a pass.
+- **`storage.committedGeneration()`** — the committed watermark as a capability, so a provider
+  compares its stamp's `sourceGeneration` against the store's truth without parsing the private
+  manifest format.
+- **Two durability/stability contracts pinned in the suite:** fsync-before-ack (holds for
+  `transact()` today; the single-op path is pinned as the documented future target — group commit
+  becomes latency batching, never durability skipping) and scan-stability-under-rotation (a scan
+  snapshot yields exactly its facts — no gaps, duplicates, or bleed-in — while segments rotate
+  beneath it).
+
+No behavior change for applications; all additions.
+
+## v8.4.0 — 2026-07-15 (the generation fact log — a sequential, self-verifying commit stream)
+
+A minor release, fully backward-compatible (all additions; no behavior change for existing APIs).
+This is infrastructure: it changes nothing about how you query today, and lays the substrate that
+makes index heals and incremental catch-up sequential-read problems instead of directory walks.
+
+- **Every committed write now also appends a "fact" — an after-image commit record.** Alongside the
+  existing before-image history, each committed generation (single-op and `transact()` alike) appends
+  what each touched entity/relationship *became* — or a body-less tombstone for a removal — to an
+  append-only, checksummed segment log under `_generations/facts/`. Crash-safe by construction: a
+  torn tail is detected and ignored; on open the log is reconciled to committed truth, so an absent
+  generation always means "never committed." `transact()` facts are durable when `transact()`
+  returns; single-op facts ride the same group-commit flush as their history.
+
+- **New: `brain.scanFacts()`** — stream committed facts in commit order, in batches, with heal-grade
+  telemetry (total scope up front; per-batch generation range, byte size, and segment; a summary
+  cross-check at the end; loud abort on any gap — never a silent skip). **`brain.factSegmentPaths()`**
+  hands zero-copy consumers the immutable sealed segment files directly. New exported types:
+  `CommitFact`, `FactOp`, `FactScanBatch`, `FactScanHandle`. Facts accumulate from the first write
+  after upgrading — pre-existing history is not retroactively converted (enumeration remains the
+  fallback for old data).
+
+- **New: the entity-tree family stamp.** At every flush/close, brainy stamps which committed
+  generation the canonical entity files reflect plus the rollup invariants (entity/relationship
+  counts) that verify the tree whole. At open, coherence is a comparison — a genuine divergence is
+  loud and names the failing invariant; `repairIndex()` recounts from canonical and re-stamps. New
+  exports: `readFamilyStamp`, `verifyFamilyStamp`, `ENTITY_TREE_STAMP_PATH`, `FamilyStamp` types.
+
+- **Storage adapters** gain optional binary raw-byte primitives (`appendRawBytes`, `readRawBytes`,
+  `writeRawBytes`, `rawByteSize`) — feature-detected; the filesystem and memory adapters implement
+  them; an adapter without them simply hosts no fact log. The fact-log namespace is registered as a
+  protected family: no sweeper or GC can delete under it.
+
+No API breaks. 24 new tests; the full commit-path regression suite is green.
+
+## v8.3.3 — 2026-07-15 (rename moves the containment edge — no ghost in the old directory)
+
+One production-reported fix plus a repair path, completing the delete/move hygiene arc (8.3.1 fixed
+deletes, 8.3.2 fixed counters, this fixes moves).
+
+- **A cross-directory `vfs.rename()` now MOVES the containment edge instead of accumulating one per
+  parent.** The old parent's `Contains` edge was never removed on a move, leaving the entity a child
+  of **both** directories: `readdir(oldDir)` kept listing it after the move, re-creating the old path
+  showed the same name twice, and any tree-walking consumer (sync engines, file browsers) saw the
+  file in two places. The old edge is now removed by edge id, resolved from the graph's own adjacency
+  — a removal never requires reading the thing being removed. Bonus fix in the same seam: a move **to
+  the root** now gets its containment edge (it was previously skipped, orphaning the file out of
+  `readdir('/')`).
+
+- **`repairIndex()` now also reconciles VFS containment** (new `vfs.repairContainment()`): every VFS
+  entity's containment edges are checked against its canonical `metadata.path` — stale old-parent
+  ghosts and duplicate edges are removed, a missing expected edge is restored, and user
+  knowledge-graph edges are never touched (only `vfs-contains` edges are candidates). Loud per
+  repair. Stores that performed cross-directory renames under ≤8.3.2 should run `brain.repairIndex()`
+  once after upgrading — the same single ritual now heals orphan directories, counters, **and**
+  containment edges.
+
+- Also ships a permanent lens-consistency regression suite (combined type+subtype vs subtype-only vs
+  canonical ground truth, id-for-id, warm and after a cold reopen), ported from the field
+  investigation that closed the historical lens-drop report.
+
+No API changes beyond the new optional `vfs.repairContainment()` (also invoked by `repairIndex()`).
+
+## v8.3.2 — 2026-07-14 (honest counters — the recount + removal-without-re-reading)
+
+Completes 8.3.1's delete-hygiene story at the counter layer, from a production proof chain reported
+by a downstream deployment: persisted entity totals were permanently **inflated** — deletes whose
+count decrement was silently skipped — and because paginated `totalCount` serves
+`Math.max(persistedTotal, scanned)`, the inflated number always won and **no disk cleanup could ever
+lower it**.
+
+- **A removal's count decrement no longer requires re-reading the record being removed.** The
+  decrement was sourced from re-reading the entity's metadata inside the delete; if that read
+  returned `null` (a replace race, or a ghost left by a pre-8.3.1 partial delete) the decrement was
+  silently skipped while the paired add had counted — minting drift on every write→delete→re-create
+  cycle. The caller's pre-delete read now rides through the whole delete path
+  (`remove()`/`removeMany()` → the delete operation → `deleteNoun`/`deleteVerb` →
+  `deleteNounMetadata`/`deleteVerbMetadata`, both sides symmetric): a null internal read falls back
+  to the known prior record instead of skipping. The `StorageAdapter` signatures gain an optional
+  `priorMetadata` parameter (additive; existing adapters unaffected).
+
+- **`repairIndex()` is the sanctioned counter recount — unconditional, and it actually persists.**
+  `rebuildTypeCounts()` previously rebuilt only the type-statistics arrays and computed the total
+  *just to log it* — the persisted scalar (`counts.json`) survived every "rebuild" untouched, so an
+  already-inflated brain could never be corrected. One canonical walk now rebuilds **every** counter
+  rollup — scalar totals, per-type maps, and type statistics — and persists them, and `repairIndex()`
+  runs it unconditionally (not only when orphan directories are found: counters can be inflated over
+  perfectly clean shelves). Brains with delete history should run `brain.repairIndex()` once after
+  upgrading; the correction survives reopen.
+
+No API breaks (optional-parameter additions only). Regression tests cover the drift cycle, the
+null-read decrement fallback, and the persisted recount across reopen.
+
+## v8.3.1 — 2026-07-14 (full-removal deletes + family-scoped migration gate)
+
+Two production-reported fixes in the write/index spine, plus an operator repair path. No API changes;
+all behavior changes make previously-wrong states honest.
+
+- **Deleting an entity now removes it completely — no more "ghost" leftovers on disk.** A canonical
+  noun delete removed the metadata (content) leg but left the entity's `vectors.json` and its `<id>/`
+  directory behind. Consequences observed in a production deployment: deleted rows were
+  indistinguishable on disk from damage scars, enumerated counts inflated monotonically with every
+  delete (the leftovers were counted forever), and locator-style reads hit unreadable ghost rows.
+  `remove()`/`removeMany()`/`deleteNoun`/`deleteVerb` now remove **both legs and the entity
+  container**, with a full two-leg before-image rollback inside the transaction. The generation log
+  still holds the delete's before-image, so `asOf()` time-travel reconstructs deleted entities exactly
+  as before — this is live-HEAD hygiene, not a history change. This also fixes **duplicate `readdir`
+  entries for re-created VFS paths** at the root: with no ghost state, a delete always unposts its
+  index rows, so a delete→recreate cycle lists the path exactly once (regression-tested across
+  repeated cycles).
+
+- **`repairIndex()` prunes ghost/scar directories left by earlier versions.** Stores that deleted
+  entities under ≤8.3.0 may hold orphaned entity directories (a vector-only leg, or an empty dir).
+  `brain.repairIndex()` now sweeps them: it removes only containers with **no metadata content leg**
+  (never a directory that still holds content), logs every removal, and recomputes type/subtype
+  counts afterward so totals stop counting ghosts.
+
+- **Reads no longer hang behind an unrelated index migration (family-scoped gate).** During a native
+  provider's one-time background migration, *every* read — including plain `get()`, VFS
+  `readdir`/`readFile`, and metadata-only `find({ where })` — blocked on the whole-brain migration
+  lock until timeout, even when the migrating index was irrelevant to the read. The gate is now
+  scoped to the index families a read actually consults: canonical reads (`get`, `batchGet`, VFS
+  content) never wait; a `find` waits only on the families its query shape needs (vector for
+  semantic, metadata for `where`/type, graph for `connected`); graph traversals wait only on the
+  graph family. Writes and unclassified operations keep the conservative whole-brain wait. A read
+  that *does* need the migrating family still blocks (bounded by `migrationWaitTimeoutMs`) and
+  surfaces the retryable `MigrationInProgressError` — never a partial result.
+
+No breaking API change. Each fix ships with regression tests.
+
+## v8.3.0 — 2026-07-13 (faster index heals + the cross-layer integrity contract)
+
+Three additive changes. The first is an immediate, standalone performance win; the other two are the
+brainy side of the write/index-spine integrity contract, inert until a native accelerator
+that implements the matching hooks is present — so this release changes nothing for a JS-only brain
+beyond the speedup.
+
+- **Canonical enumeration is up to ~16× faster — the dominant term in an index heal.** The paginated
+  entity walk (`getNounsWithPagination`) hydrated each entity's vector + metadata one-at-a-time; since
+  every index rebuild enumerates canonical storage, that serial per-item latency dominated multi-minute
+  heals. Hydration is now 16-way bounded-concurrency, with the pagination contract (order, cursor
+  resume, filters, totalCount) byte-identical to before. New **`getNounIdsWithPagination()`** returns
+  ids without hydrating anything (zero per-entity reads when unfiltered) for callers that own their own
+  IO schedule.
+
+- **Cross-layer integrity — `validateIndexConsistency()` is no longer blind to native providers.** It
+  only ran the JS metadata index's own check, so a native provider whose manifest/segments/counts had
+  diverged still read as "healthy". It now feature-detects and aggregates each provider's optional
+  `validateInvariants()` self-report, names every failing invariant with its numbers, and exposes the
+  per-provider reports. `repairIndex()` now reconciles native derived state from canonical too
+  (rebuilding any provider whose failing invariant asks for it). New exported types
+  `ProviderInvariantReport` / `InvariantResult` / `InvariantHeal`.
+
+- **Registered-blob families — declared index files are undeletable through the storage layer.** A
+  provider can declare a derived-index blob *family* (a set of members that are load-bearing together);
+  once declared, `deleteBinaryBlob` / `removeRawPrefix` refuse to remove a member (new exported
+  `ProtectedArtifactError`), so a stray in-process sweeper cannot delete a load-bearing index file. The
+  declaration persists across reopen; `checkDerivedFamiliesPresent()` names any member missing on open.
+  New optional `StorageAdapter` surface (`registerDerivedFamily` / `unregisterDerivedFamily` /
+  `listDerivedFamilies` + `DerivedFamilyDeclaration`) and exported `DerivedArtifactMissingError`.
+
+No breaking API change (all additions are optional/new). Each change ships with regression tests.
+
+## v8.2.8 — 2026-07-13 (honest index readiness — no more silently-empty queries on a cold index)
+
+Closes the last of the three spine anti-patterns: the "dishonest readiness proxy," where `size() > 0`
+was treated as "this index actually serves queries." On a cold open (fresh boot, restart, crash
+recovery) a native index can load its **count** before its **serving structure** — so for a brief
+window it has data but cannot answer, and a query returned a silent empty result indistinguishable
+from "no such data." Every fix here asks the index whether it can *actually serve*, and if not,
+self-heals or fails loudly instead of returning `[]`.
+
+- **Semantic search no longer returns a silent `[]` on a cold vector index.** A pure semantic
+  `find({ query })` has no filter, so nothing previously guarded the vector index. A new one-shot
+  guard verifies the vector index serves a known persisted vector on the first semantic/proximity
+  search — preferring the provider's honest `isReady()` signal, else a known-vector self-match probe.
+  It rebuilds from canonical records if the serving structure did not load, and throws the new
+  **`VectorIndexNotReadyError`** only if a rebuild still cannot serve — never a silent empty result.
+
+- **Relationship reads fall back to the canonical scan instead of an empty result on a cold graph
+  index.** `getVerbsBySource`/`getVerbsByTarget` (used by relationship queries and virtual-filesystem
+  traversal) skipped the fast path only on `isInitialized` — which reads true once the manifest loaded
+  even if the source→target adjacency did not. They now consult the honest readiness signal and, when
+  the adjacency is not serving, take the correct-but-slower canonical shard scan. A one-shot self-heal
+  probe covers providers that expose no readiness signal.
+
+- **`getIndexStatus()` tells the truth for readiness probes.** It reported `populated: size>0` only, so
+  a Kubernetes readiness check could route traffic to a brain still warming up. It now folds in the
+  honest per-index `ready` signal (making `populated` honest), plus the degraded states already
+  surfaced by `checkHealth()`/`validateIndexConsistency()` (`rebuildFailed`/`rebuildError` and a
+  `degradedIds` count) — so a probe never reports 200-ready over a known-degraded index.
+
+This completes the write/index-spine hardening end to end. New export: **`VectorIndexNotReadyError`**;
+`getIndexStatus()` gains additive fields (`rebuildFailed`, `rebuildError?`, `degradedIds`, per-index
+`ready?`). No breaking API change. Each fix ships with a dedicated regression test.
+
+## v8.2.7 — 2026-07-13 (loadBinaryBlob fault-propagation — the lockstep completion of 8.2.6)
+
+Completes the Pass-1 spine hardening. `loadBinaryBlob` (the raw-blob read a native accelerator mmaps
+for its index files) previously returned `null` on ANY read error, so a real IO fault
+(EIO/EACCES/EMFILE) on a present-but-unreadable index blob masqueraded as "the blob is absent" —
+driving a needless full rebuild or an empty read. It now distinguishes genuine absence (ENOENT →
+`null`, the documented contract) from a real fault (→ throw), so a transient disk fault surfaces
+loudly instead of silently degrading the index.
+
+This one change was deliberately held out of 8.2.6 and ships now, in lockstep with the native
+accelerator release that hardened its two column-store read sites to handle the throw (a faulted
+segment read marks the field unavailable and throws a named error, instead of relying on
+null-on-error). A consumer on new brainy + an older accelerator was never at risk: 8.2.6 kept the
+prior swallow-on-fault behavior for this method until the accelerator was ready.
+
+No API change. Regression: `tests/unit/storage/blob-save-durability.test.ts` gains the loadBinaryBlob
+leg (absent → null; present → bytes; real fault → throws).
+
+## v8.2.6 — 2026-07-13 (write/index-spine hardening — loud errors, never quiet losses)
+
+Durability + integrity hardening across the write and index paths. Every fix converts a place that
+could *silently* lose data, serve a partial result, or acknowledge a write that did not land into a
+loud, observable failure. No breaking API changes; one new exported error.
+
+- **A durable blob write that stored nothing is no longer acknowledged.** The atomic blob write
+  (`saveBinaryBlob`, used for vector/graph index segments and the native index files) uses a unique
+  per-writer temp file, so a rename `ENOENT` can only mean *our* temp vanished before the rename —
+  the bytes never landed. It previously returned success on that path; it now retries once with a
+  fresh temp and, if the temp vanishes again, throws instead of acknowledging a write that persisted
+  nothing. A downstream deployment that mmaps these files no longer finds a "successfully written"
+  blob missing on the next open.
+
+- **Single-op history durability is enforced, not assumed.** The asynchronous group-commit flush
+  that persists single-op generation history previously swallowed a persist failure as a warning
+  while writes kept succeeding and their history piled up, undurable, in memory. It now tolerates a
+  transient blip (retry with capped backoff) and, after repeated failures, **refuses further writes**
+  with the new **`PendingFlushDurabilityError`** rather than promise a durability it cannot deliver.
+  Live canonical data is untouched; the latch self-heals the moment a flush succeeds. Callers needing
+  hard per-write durability should keep using `transact()` (or `flush()` after a single-op).
+
+- **A degraded derived index is surfaced on reads, not served silently.** When a non-fatal index
+  rebuild fails at open, or a write commits via adopt-forward recovery with an incomplete derived
+  index, `find()` and `get()` now emit one loud warning per degraded window (reads still return —
+  canonical is the source of truth), the state folds into `checkHealth()` /
+  `validateIndexConsistency()`, and `repairIndex()` reconciles and clears it.
+
+- **`clear()` removes the full derived footprint.** It previously left raw index blobs, the native
+  id-mapper, and column-index manifests on disk, so a cleared brain could re-read stale native state
+  on reopen. It now wipes them together as a set.
+
+- **Counts stay honest across deletes.** A delete decremented the per-type breakdown but not the
+  scalar total, so the total inflated permanently (and won pagination). Delete now decrements both;
+  the invariant `total === Σ per-type` holds across any interleaving of add / visibility-flip /
+  delete and across a reopen.
+
+- **Index-maintenance and aggregation failures are loud.** A partial LSM/segment load no longer
+  publishes the manifest's full count as if healthy; an HNSW flush that can't persist a node throws
+  (`HnswFlushError`) instead of returning a lying count and dropping the node; a corrupt or missing
+  manifest-listed column segment throws (`ColumnSegmentLoadError`) instead of silently dropping its
+  entities from every query; and aggregation materialization / state-load failures now warn instead
+  of vanishing into an empty catch.
+
+New export: **`PendingFlushDurabilityError`** (with `.cause` and `.failedAttempts`). No other public
+API change. Each fix ships with a dedicated regression test.
+
+## v8.2.5 — 2026-07-12 (honest response when a transaction rollback can't complete)
+
+Data-integrity fix. When a transaction failed and its rollback then *also* failed to undo a
+canonical write (retries exhausted), the old behavior logged, continued, and threw
+`TransactionRollbackError` — while the record it couldn't undo stayed durable on disk, and the
+transaction even reported its state as `'rolled_back'`. The caller got an error implying the write
+was undone; a read-back showed the record. A failed rollback had no truthful response.
+
+Rollback now tells the truth, with a two-branch contract:
+
+- **Adopt-forward (safe case).** A single-op write (`add`/`update`/…) whose only damage is a
+  durably-present record — the write the caller asked for — is **adopted**: its generation is
+  committed, the write returns success, and a loud warning records that the derived index may be
+  incomplete for that id until the next rebuild/`repairIndex()`. No error, no double-write; the
+  record is immediately durable and retrievable by `get()`.
+- **Fail loud + quarantine (unsafe case).** A multi-operation batch, or *any* case where a record
+  was lost (a remove/update whose restore-undo failed), throws the new **`StoreInconsistentError`**
+  naming every unreconciled record and its disposition (`orphan` vs `loss`), and puts the brain into
+  **write-quarantine**: reads keep working, but writes are refused until `repairIndex()` reconciles
+  the derived indexes against canonical storage and lifts the quarantine. The generation counter is
+  not advanced, and a transaction whose rollback failed is now `'inconsistent'`, never the
+  `'rolled_back'` lie.
+
+The decision is made by *observation*, not guesswork: both commit paths already hold byte-identical
+before-images, so after a failed rollback the store compares current canonical state to them to
+classify exactly which records are orphaned or lost.
+
+New export: `StoreInconsistentError` (with `.records` and `.cause`) and the `UnreconciledRecord` type.
+No other API change; `repairIndex()` gains the quarantine-lift behavior. Regression
+(`tests/integration/rollback-trapdoor.test.ts`) injects the exact failure (index add throws →
+canonical delete-undo fails) and pins adopt-forward (durable, get-able, not quarantined), fail-loud
+(`StoreInconsistentError` + quarantine + reads work + `repairIndex()` lifts it), and the error's
+record naming.
+
+## v8.2.4 — 2026-07-12 (restore can no longer destroy the store it's recovering)
+
+Recovery-safety fix. `restore()` removed the entire live brain directory and THEN copied the
+snapshot in — so any copy failure left the store destroyed with only a partial copy. The sharpest
+edge: the copy (`fs.cp`) materialized the holes of sparse mmap blob files, so a snapshot that fits
+on disk could balloon and `ENOSPC` mid-copy, and the recovery tool would have just destroyed the
+brain it was asked to recover. Reported from a downstream incident's recovery forensics.
+
+Restore is now **non-destructive and crash-resumable**:
+
+- The snapshot is copied into a staging area (`_restore_staging/`) **before any live data is
+  touched**. The copy is **sparse-aware** — all-zero regions are left as holes, so a store of
+  mostly-hole blobs restores at its true allocated size instead of its apparent size.
+- A copy failure (including `ENOSPC`) removes only the half-written staging area and throws; the
+  live store is left **exactly as it was**.
+- Only after the copy succeeds and a completion marker is `fsync`'d does an **atomic per-entry
+  swap** move the staged data into place — same-filesystem renames that cannot fail for disk space.
+- A crash mid-swap is finished **forward** on the next open: startup resumes a committed-but-
+  incomplete swap, or discards an uncommitted staging area (live data still authoritative).
+
+No API change — `restore(path, { confirm: true })` is unchanged. `persist()` was already safe
+(hard-link snapshot). Regression (`tests/integration/restore-nondestructive.test.ts`): a forced
+copy failure leaves live data fully intact, a normal restore round-trips, an interrupted-but-
+committed restore completes on reopen, an uncommitted staging area is discarded, and the sparse
+copy is byte-identical with allocation far below apparent size.
+
+## v8.2.3 — 2026-07-12 (a committed transaction is durable on return)
+
+Durability fix. A `transact()` reported "committed" while its canonical entity writes were still
+only in the OS page cache (written via tmp+rename, not yet `fsync`'d), even though the generation
+counter and manifest WERE fsync'd. A hard kill (power loss, SIGKILL) in that window could leave the
+durable generation counter **ahead of** the persisted entity bytes — so a consumer resuming from the
+counter would see "phantom progress": a generation that claims writes the disk never kept. Reported
+from a downstream migration's crash-lifecycle forensics.
+
+`commitTransaction` now runs a **durability barrier**: it records every canonical write and delete
+the batch's operations make, then `fsync`s that entire footprint (file contents, the rename
+directory entries, and the parent directories of any deletes) **before** advancing the generation
+counter and manifest. A committed transaction is therefore durable the moment `transact()` returns —
+the counter can never outrun the entity bytes.
+
+**Durability contract, now explicit.** `transact()` is durable-on-return (above). A **single-op**
+write (`add`/`update`/`remove`/`relate`/…) is Model-B group-commit: its live bytes are written but
+become durable at the next `flush()` or `close()`, not the instant the call resolves — the counter
+is buffered alongside the data, so a crash loses both together (never a torn counter-ahead-of-state
+store). Need per-write durability? Use `transact()` (even for one op), or `flush()` after the write.
+This deliberately trades single-op fsync latency (a 3-5x write regression) for throughput.
+
+No API change; no accelerator involvement (the barrier is in the filesystem storage adapter). In-memory
+and durable-per-call (cloud object-PUT) adapters treat the barrier as a no-op. Regression:
+`tests/integration/transact-durability-barrier.test.ts` proves entity writes fsync in an earlier
+batch than the manifest, for single-op and multi-op (add+relate) transactions, and that a
+precommit-rejected batch opens no barrier and advances nothing.
+
+## v8.2.2 — 2026-07-11 (P0: a timed-out transaction now rolls back — no torn state)
+
+Data-integrity fix. A transaction that exceeded its time budget **mid-flight** (e.g. a bulk
+`transact()` on slower hardware crossing the 30s ceiling) threw a timeout error WITHOUT rolling
+back the operations it had already applied. The budget check sat outside the per-operation
+rollback path, so only per-operation *failures* rolled back — a timeout stranded the partial
+writes in canonical storage while the generation was never stamped, leaving torn, generation-less
+state. This was caught by a downstream migration that crossed the ceiling on a large batch.
+
+`Transaction.execute()` now has a **single rollback point**: any error that escapes the operation
+loop — an operation failure OR a mid-flight timeout — rolls back every applied operation in
+reverse order, then surfaces the original error (a rollback failure supersedes it, loudly, as
+before). This restores the invariant the generation-store commit path already depended on: a throw
+from execute means the applied operations were undone byte-identically, and the aborted
+transaction leaves `generation()` unchanged and storage byte-identical to its pre-transaction
+state.
+
+No API change. Regression pins the reported requirement — after a mid-flight timeout,
+storage is byte-identical and the transaction ends in the `rolled_back` terminal state
+(`tests/unit/transaction/timeout-rollback.test.ts`), plus the operation-failure and
+rollback-failure paths through the same single rollback point.
+
+**Note on the 30s ceiling itself** (configurable/scaled timeout, batched embedding precompute,
+timeout telemetry) — that ergonomics work is tracked separately; this release fixes only the
+correctness bug (a timeout must never leave partial state), independent of where the ceiling sits.
+
+## v8.2.1 — 2026-07-11 (transact forward references work on the native accelerator)
+
+Parity fix. `transact()` has always promised atomic forward references — `add` an entity and
+`relate` to it in one batch — but the transaction planner resolved relationship endpoint ids at
+**plan** time, before the batch's adds had applied. Asking the id mapper about an entity that
+doesn't exist yet made the accelerator's (correctly strict) native mapper throw in
+`EntityIdMapper.getOrAssign`, so `transact([{op:'add', id:X}, {op:'relate', to:X}])` failed on
+native deployments while the permissive JS mapper masked the bug — and silently leaked an id
+assignment whenever a batch was later rejected by a commit precondition.
+
+Endpoint resolution is now **lazy** — evaluated when the graph operation executes inside the
+commit, after the batch's adds have applied (the same lazy pattern the operation's generation
+stamp already used). Fixed across all transact-planned graph operations: relate (including the
+bidirectional reverse edge), remove's relationship cascade, and unrelate. Single-operation writes
+are unchanged. Also fixed as a byproduct: a rejected batch no longer pollutes the id mapper.
+
+Regression suite covers the exact reported shape, both-endpoints-in-batch, bidirectional,
+add+relate+remove in one batch, and the mapper-cleanliness proof on rejected batches
+(`tests/integration/transact-forward-ref-graph.test.ts`). No API changes; no accelerator version
+pairing required.
+
+## v8.2.0 — 2026-07-10 (temporal VFS — file content joins the immutability model)
+
+Time travel now covers Virtual Filesystem **content**. Previously the temporal model had a hole
+exactly where files were concerned: every entity write was an immutable generation, but the
+content *bytes* lived under an eager reference-count GC — deleting a file could physically destroy
+bytes that in-window history still referenced, while overwriting never released the old content at
+all (an unbounded silent leak that only *accidentally* preserved history). A production consumer's
+recovery from a bad deploy succeeded only because a stale field happened to hold the good value —
+luck, not a guarantee. Both directions are now fixed by making blob reclamation a **history**
+decision instead of a **liveness** decision:
+
+- **Content blobs are retention-protected.** Each blob tracks live references AND history
+  references (one per persisted generation record that carries its hash). Deleting/overwriting a
+  file drops only the live reference; bytes are physically reclaimed in exactly one place —
+  history compaction — once no live reference and no retained generation references the hash.
+  Pinned views are exempt automatically (compaction already respects pins). Crash ordering is
+  over-count-only (never under-count), so a crash can leak until the built-in scrub recounts, but
+  can never reclaim bytes history still needs. Existing stores get a one-time, marker-gated
+  backfill on open; cross-file content dedup is handled exactly.
+- **`vfs.readFile(path, { asOf })`** — the file's exact bytes as of a generation or `Date`,
+  guaranteed present within the retention window.
+- **`vfs.history(path)`** — the file's versions (`{ generation, timestamp, hash, size,
+  mimeType? }`, ascending). Restore = read the old bytes `asOf` and write them back (a new write;
+  history is never rewritten).
+- **Overwrites now refresh the file entity's `data`/embedding text** — previously semantic search
+  and the `data` field served the FIRST version's text forever.
+
+Lifecycle note: deleting a file no longer frees its bytes immediately — old content lives until
+compaction reclaims its generations, under the same `retention` budget as all Model-B history
+(`retention: 'all'` keeps every version forever). Guide: the new "Time travel for files" section in
+`docs/guides/snapshots-and-time-travel.md`. Native accelerator: unaffected (canonical write path
+only) — no version pairing required.
+
+## v8.1.0 — 2026-07-10 (`brain.onChange` — the in-process change feed)
+
+New public API: subscribe to every committed mutation with
+`brain.onChange(cb) → unsubscribe`. One event per affected record, for **every**
+canonical write regardless of origin — direct calls, batch methods, `transact()`,
+imports, and Virtual Filesystem writes all funnel through the same commit point the
+feed is emitted from. This is the authoritative in-process signal for live UIs,
+cache invalidation, and realtime sync layers (downstream SDKs can forward it over
+their own transports to make local and remote brains uniform).
+
+Event shape (`BrainyChangeEvent`, exported): `kind` (`entity`/`relation`/`store`),
+`op` (`add`/`update`/`remove`/`relate`/`unrelate`/`updateRelation`/`clear`/`restore`),
+the post-commit `entity` or `relation` view (type + full custom metadata), the
+committed `generation`, and `timestamp`. Notable properties:
+
+- **Post-commit only** — an aborted write (a losing `ifRev` CAS, a rejected
+  transaction) never emits. If you got the event, the write is durable.
+- **Deletes fully described** — `remove`/`unrelate` carry the record's last
+  committed state (from the commit's own history record), not just an id; batch
+  deletes included.
+- **Batches emit per item**; a `transact()` batch's events share its single
+  generation. Entity deletes also emit `unrelate` for each cascaded relationship.
+- **Commit-ordered, asynchronous, isolated** — events arrive in commit order,
+  dispatched in a microtask so a slow listener never delays a write and a throwing
+  listener never affects the write or other listeners. Zero overhead with no
+  subscribers.
+- **`kind: 'store'`** events for `clear()`/`restore()` mean "refetch everything".
+- Fire-and-forget by design; each event's `generation` composes with
+  `asOf()`/`transactionLog()` for catch-up after a gap.
+
+Guide: `docs/guides/reacting-to-changes.md`. No behavior changes for existing code.
+
+## v8.0.17 — 2026-07-08 (count recovery scans the real layout · ~1,100 lines of dead 7.x machinery removed)
+
+A cleanup of the vestigial 7.x "hnsw sharding" machinery that turned up two real fixes.
+
+**1 — Count recovery now scans the layout the database actually uses.** When `counts.json` is
+lost or corrupted (container restarts, partial copies), the recovery scan rebuilt the entity/
+relationship counters by counting files in `entities/*/hnsw/` — directories the 8.0 write path
+never populates — so an established store recovered to **zero counts** on real data: wrong
+`getNounCount()`/stats, a "New installation"-style boot log, and a mis-sized rebuild-strategy
+decision at open. The scan now counts the canonical `entities/<kind>/<shard>/<id>/` tree.
+Regression-tested: delete `counts.json` from a populated store → reopen → exact counts.
+
+**2 — A latent init-time deadlock removed.** The recovery scan's type-distribution sampler called
+a guarded accessor (`getNounMetadata` → `ensureInitialized`) from **inside** `init()`, which
+re-enters `init()` and hangs the open. It was unreachable before only because the old scan never
+found anything to sample; the fix reads the sampled metadata files directly.
+
+**3 — The dead machinery itself is gone (−1,138 lines).** Twenty-three methods with zero callers:
+the 7.x hnsw-layout entity/edge CRUD, the sharding-depth prober + depth-migration engine, and an
+orphaned streaming paginator. Verified by reachability analysis before deletion; no public API
+touched; the full suite is green. New stores no longer carry the always-empty legacy directories'
+scan cost, and the boot log keeps the truthful new-vs-established wording from v8.0.13.
+
+## v8.0.16 — 2026-07-08 (concurrency fast-follow: atomic `ifAbsent`/`upsert` + exact blob reference counts)
+
+Closes the two remaining check-then-act races found in the sweep that followed v8.0.15's CAS fix
+(disclosed in that release's notes). Same bug class — a check performed before the serialization
+point that guards the apply — same cure.
+
+**1 — `add({ ifAbsent })` and `add({ upsert })` are now atomic.** The absence check ran before the
+commit mutex, so N concurrent same-id creates could all pass it and all write — the second
+silently overwriting the first, violating ifAbsent's "returns the existing id **without writing**"
+contract and upsert's "merge, never clobber" contract. The insert leg now carries a must-be-absent
+precondition (the same conditional-commit primitive as `ifRev`), verified under the commit mutex:
+exactly one concurrent create wins; every other caller takes its documented resolution — ifAbsent
+returns the existing id with zero writes, upsert merges into the now-existing entity (with a
+bounded retry if a concurrent delete intervenes). Verified: 8 concurrent `ifAbsent` creates
+advance the store by exactly ONE generation; 8 concurrent upserts on an absent id produce one
+create + seven merges (`_rev` lands at exactly 8). Note: `transact()`'s batch-level
+ifAbsent/upsert keep planning-time semantics (the batch converges to a valid entity; the window is
+documented, not silent).
+
+**2 — Blob reference counts are exact under concurrency.** The content-addressed blob store's
+`write()` (dedup: exists → add a reference, absent → create) and `delete()` (decrement → remove at
+zero) were unserialized read-modify-writes over the blob's metadata. Concurrent writes of
+identical content could lose references — making a later delete remove bytes **another file still
+referenced** (data loss), or leak unreferenced blobs. All reference-count-bearing mutations are
+now serialized per content hash (distinct content never contends): N concurrent identical writes
+yield exactly N references, and a blob is physically removed only when the true last reference
+drops. Verified with concurrent write/delete storms.
+
+Both fixes are in-process complete by construction — storage enforces single-writer-per-directory,
+so the process is the whole concurrency domain. No API changes.
+
+## v8.0.15 — 2026-07-08 (`ifRev` CAS is now atomic — exactly one winner under concurrency)
+
+Correctness fix for optimistic concurrency, reported from a production cutover rehearsal. N
+**concurrent** `update({ ifRev })` calls carrying the same expected revision ALL succeeded — zero
+`RevisionConflictError`s, last-writer-wins, the other N−1 writes silently lost. (Sequential calls
+conflicted correctly.) The revision check ran before the commit mutex, so interleaved callers all
+passed it before any apply landed — breaking the exactly-one-winner semantics the
+[optimistic-concurrency guide](docs/guides/optimistic-concurrency.md) promises, which advisory
+locks and per-entity ledgers/counters build on.
+
+The fix makes the check-and-apply a **conditional commit**: the generation store's commit paths
+accept a precondition that runs under the commit mutex against the just-read authoritative
+before-images — the per-record analogue of `ifAtGeneration`, which always ran there. `update()`
+and `transact()` per-op `ifRev` both re-verify at that point (the earlier check remains as a cheap
+fast-fail). A conflict aborts atomically: nothing staged, nothing applied, the same
+`RevisionConflictError` as before. Two adjacent behaviors also became honest:
+
+- **`_rev` is now monotonic under concurrency.** The winner's stamp derives from the
+  authoritative before-image, so N concurrent plain updates advance `_rev` by N (previously they
+  could all stamp the same stale value).
+- **CAS against a concurrently-deleted entity** now throws `EntityNotFoundError` instead of
+  silently resurrecting it (plain updates keep their last-writer-wins re-create semantics).
+
+Verified with a concurrency regression suite: 8 parallel same-rev updates → exactly 1 winner + 7
+conflicts; the documented read→CAS→retry ledger loop converges exactly (8 workers, 8 decrements,
+0 lost) — `tests/integration/ifrev-concurrent-cas.test.ts`. If you serialized writes in your own
+adapter as a workaround, it can come out after this upgrade.
+
+## v8.0.14 — 2026-07-07 (7→8 migration preserves branch-scoped non-entity state instead of deleting it)
+
+Defense-in-depth for the one-time 7→8 layout migration. The migration rescues the head branch's
+entities (`branches/<head>/entities/*` → `entities/*`) and then drained the whole `branches/<head>/`
+directory. If a 7.x engine had written durable state under the head branch *outside* `entities/`
+(a branch-scoped index, blob area, or field registry), that drain would have silently deleted it —
+the same failure class as the VFS content blobs a 7.x store kept in `_cow/`. The migration now drains
+the branch only when nothing but the moved entities remains; if any non-entity object survives, it
+**preserves the branch** (no delete) and logs a loud warning naming the leftover keys, so the state is
+recoverable rather than lost. No effect on a normal migration (a clean branch still drains); purely a
+guard against silent data loss.
 
 Cosmetic boot-log fix. Every persisted 8.0 store logged `📁 New installation: using depth 1
 sharding` on **every** open — even established brains holding thousands of entities — which is

docs/PLUGINS.md

@@ -149,6 +149,17 @@ context.registerProvider('embedBatch', async (texts: string[]) => {
 
 ### Index Providers
 
+> **Write-path invariant (the change-feed contract).** Every canonical
+> mutation flows through Brainy's generation-store commit points — index
+> providers are invoked *inside* that commit and never originate canonical
+> writes of their own. The `brain.onChange` change feed is emitted from those
+> commit points and relies on this: **a plugin must never introduce a write
+> path that bypasses the generation-store commit.** If a future provider ever
+> needs a direct native ingest path, it must either route through the commit
+> or emit equivalent change events — otherwise every `onChange` consumer
+> (live UIs, cache invalidation, realtime sync) silently develops a blind
+> spot.
+
 #### `vector`
 **Type:** `(config: object, distanceFunction: Function, options: object) => VectorIndexProvider-compatible`
 

docs/concepts/generation-fact-log.md

@@ -0,0 +1,117 @@
+---
+title: The Generation Fact Log
+slug: concepts/generation-fact-log
+public: true
+category: concepts
+template: concept
+order: 6
+description: Every committed write also appends a self-verifying "fact" — an after-image commit record — to an append-only log. What facts are, the crash-safety model, the scanFacts() streaming surface, family stamps, and how index providers consume the log for sequential heals.
+next:
+  - concepts/consistency-model
+  - guides/snapshots-and-time-travel
+---
+
+# The Generation Fact Log
+
+Since 8.4.0, every committed generation also appends a **fact** — a compact record of what each
+touched entity or relationship *became* — to an append-only, checksummed log under
+`_generations/facts/`. Where the generational history answers *"what did things look like
+before?"* (before-images, powering `asOf()` and rollback), the fact log answers *"what happened,
+in order?"* — one sequential, self-verifying stream of the store's present being written.
+
+Nothing about querying changes. The fact log exists for three consumers:
+
+1. **Index heals and rebuilds** — one sequential read in commit order replaces a per-entity
+   directory walk over millions of files.
+2. **Incremental catch-up** — a derived index that knows which generation it reflects reads *just
+   the gap*, instead of rebuilding from scratch.
+3. **Replay and audit tooling** — anything that wants the store's committed timeline as a stream.
+
+## What a fact is
+
+One fact per committed generation:
+
+- **`generation`** and **`timestamp`** — which commit, when.
+- **`ops`** — every write in that commit: `{ kind: 'noun' | 'verb', id, record }` where `record`
+  holds the entity's full after-image (both stored legs), or **`null` for a tombstone** — a
+  removal carries no body, by design.
+- **`meta`** — the transaction metadata `transact()` was submitted with, when present.
+- **`blobHashes`** — content-blob references, for exact reclamation accounting.
+
+Facts accumulate **from the first write after upgrading** — pre-existing history is not
+retroactively converted, and consumers fall back to the enumeration walk when no log exists.
+
+## Crash safety, in one paragraph
+
+Facts are appended and fsynced **inside the same durability window as the commit itself**, before
+the commit point — so after a crash, the log can only ever be *ahead* of committed truth, never
+behind it with a hole. On open, the store reconciles the log back to the committed watermark:
+torn tails are detected by per-record checksums and cut; whole records beyond the watermark are
+truncated. The invariant every reader can rely on: **an absent generation was never committed; a
+present fact was.** `transact()` facts are durable the moment `transact()` returns; single-op
+facts share the same group-commit flush as the rest of their generation, so a hard kill loses the
+fact and the generation *together* — never a torn state.
+
+## Reading the log
+
+```typescript
+const scan = brain.scanFacts({ fromGeneration: 1 })
+if (scan) {
+  // Telemetry up front — progress bars get a denominator from second zero.
+  console.log(scan.headGeneration, scan.segmentCount, scan.approxFactCount)
+
+  for await (const batch of scan.batches()) {
+    // Each batch: { facts, firstGeneration, lastGeneration, factCount, byteSize, segmentId }
+    for (const fact of batch.facts) {
+      for (const op of fact.ops) {
+        if (op.record === null) {
+          // a tombstone: op.id was removed in this generation
+        }
+      }
+    }
+  }
+
+  console.log(scan.summary()) // { factsYielded, segmentsRead } — the cross-check
+}
+```
+
+- `scanFacts()` returns `null` when the store hosts no fact log (older store, or a storage adapter
+  without binary append support) — fall back to enumerating entities.
+- Scans run against a **snapshot**: facts appended after the scan opens never bleed in, each fact
+  is yielded exactly once, and a detected gap aborts loudly — never a silent skip.
+- `brain.factSegmentPaths()` returns the immutable, *sealed* segment files for zero-copy consumers
+  (the append-mutable tail is excluded — read it through `scanFacts()`).
+
+## Family stamps: how a projection proves it's current
+
+Anything derived from the store — an index, the entity file tree itself — carries a **family
+stamp**: a small JSON record of *which committed generation the projection reflects*
+(`sourceGeneration`) plus the invariants that verify it whole (exact per-file byte sizes for
+bounded families; rollup invariants like entity counts for unbounded ones). At open, coherence is
+a **comparison**, not a walk:
+
+- stamp equals the committed watermark and invariants hold → serve;
+- stamp is behind → the projection reads just the gap from the fact log;
+- invariants fail → loud, named divergence — `brain.repairIndex()` rebuilds from canonical and
+  re-stamps.
+
+The verifier is exported (`verifyFamilyStamp`) so every projection — TypeScript or native — runs
+literally the same check.
+
+## For plugin authors: the storage capability
+
+Index providers receive the storage adapter, not the brain — so the host wires the log onto it.
+Feature-detect and prefer the stream; fall back to enumeration:
+
+```typescript
+const scan = storage.scanFacts?.({ fromGeneration: stamp.sourceGeneration + 1 })
+if (scan) {
+  // sequential catch-up from the log
+} else {
+  // enumeration walk (older store or adapter)
+}
+const committed = storage.committedGeneration?.() // the watermark stamps compare against
+```
+
+Providers must never construct their own reader over the log's files — the open path belongs to
+the single writer (it reconciles the log at open); the capability is the sanctioned seam.

docs/guides/aggregation.md

@@ -11,7 +11,13 @@ No batch jobs. No scheduled recalculations. Aggregates stay current with every w
 **Defining over existing data:** if you define an aggregate on a store that already holds
 matching entities, Brainy backfills it from those entities on the first query (a one-time scan,
 then purely incremental). So `defineAggregate()` behaves the same whether you define it before
-or after the data exists — including when a persisted brain reopens already populated.
+or after the data exists.
+
+**Reopening a persisted brain:** aggregate state persists across restarts. Re-defining the
+same aggregate at boot (the normal declarative pattern) adopts the persisted state directly —
+no rescan. A backfill scan runs only when the definition actually changed, when no persisted
+state exists, or when the state failed to load; and however many aggregates need backfilling,
+they share a single scan.
 
 ## Quick Start
 

docs/guides/external-backups-and-sparse-storage.md

@@ -0,0 +1,99 @@
+---
+title: External Backups & Sparse Storage
+slug: guides/external-backups
+public: true
+category: guides
+template: guide
+order: 10
+description: How to back up a brain directory with external tools (tar, rsync, cp) without exploding sparse files — why a store can show 100+ GB "apparent" size on a small disk, which files are sparse, and how persist()/restore() handle it for you.
+next:
+  - guides/snapshots-and-time-travel
+  - concepts/storage-adapters
+---
+
+# External Backups & Sparse Storage
+
+The built-in snapshot path — [`db.persist()` and `brain.restore()`](/docs/guides/snapshots-and-time-travel) —
+already handles everything on this page for you. Read this when you back up a brain directory with
+**external tools**: `tar`, `rsync`, `cp`, `scp`, or a filesystem-level backup agent.
+
+## The one-sentence rule
+
+> **Always use the sparse-aware flag**: `tar czSf` (capital `S`), `rsync --sparse`,
+> `cp --sparse=always`. A naive copy can turn a 2 GB store into a 100+ GB one — or fail
+> the disk entirely.
+
+## Why: some files are sparse
+
+When a native accelerator plugin is active, parts of the index live in **memory-mapped files**
+created at a large fixed virtual size — the file's *apparent* size — while the filesystem only
+allocates blocks that were actually written. A brand-new id-mapper file can report tens of
+gigabytes in `ls -l` while occupying a few megabytes on disk.
+
+Check the difference yourself:
+
+```bash
+ls -lh  brain-data/_id_mapper/    # APPARENT size (can be huge)
+du -sh  brain-data/               # ALLOCATED size (the real footprint)
+```
+
+The sparse candidates in a brain directory:
+
+| Path | What it is |
+|---|---|
+| `_id_mapper/` | The native id-mapper's mmap files (large fixed virtual size) |
+| `_blobs/` | Native index files (vector base, segments) — may be mmap-backed |
+
+Everything else (entities, `_system`, `_generations`, `_cas` content blobs) is ordinary dense data.
+
+## Doing it right
+
+**tar** — the `S` flag detects holes and stores only real data:
+
+```bash
+tar czSf brain-backup.tgz /data/brain
+# restore preserves the holes:
+tar xzSf brain-backup.tgz -C /data/
+```
+
+**rsync**:
+
+```bash
+rsync -a --sparse /data/brain/ backup-host:/backups/brain/
+```
+
+**cp**:
+
+```bash
+cp -a --sparse=always /data/brain /backups/brain
+```
+
+**What goes wrong without the flag:** the copy *materializes* every hole as real zero bytes.
+A store whose apparent size exceeds the target disk fails with `ENOSPC` partway through — and a
+copy that *does* fit silently costs the full apparent size in storage and transfer time.
+
+## What the built-in paths do (so you don't have to)
+
+- **`db.persist(path)`** snapshots via **hard links** — instant and space-shared, since every data
+  file is immutable-by-rename. The handful of append-in-place files (the transaction log, the
+  commit fact log's tail segment) and mmap-mutated directories (`_id_mapper/`) are **byte-copied**
+  instead, so a post-snapshot write can never reach through a shared inode into your backup.
+- **`brain.restore(path, { confirm: true })`** is **non-destructive and sparse-aware**: the snapshot
+  is copied into a staging area *before* any live data is touched (all-zero blocks stay holes), and
+  only after the copy fully succeeds does an atomic swap move it into place. A failed copy —
+  including `ENOSPC` — leaves the live store exactly as it was. A crash mid-swap completes forward
+  on the next open.
+
+## Live-store caveats for external tools
+
+1. **Prefer snapshotting a `persist()` output, not the live directory.** `persist()` produces a
+   crash-consistent, immutable snapshot; running `tar` against a live, actively-written directory
+   can capture a torn mid-write state. If you must archive live, stop writes first (or accept that
+   the archive is only as consistent as the moment's flush state).
+2. **Never prune or "clean up" files inside a brain directory.** Index files that look stale or
+   redundant are load-bearing; the store protects its declared index families from in-process
+   deletion, but an external `rm` bypasses that fence. If space is the concern, `du -sh` first —
+   the allocated size is usually far smaller than it looks.
+3. **Verify restores by opening them.** `Brainy.load(path)` opens any snapshot or restored
+   directory read-only — the store verifies its own coherence at open and reports loudly if
+   anything is missing or torn.

docs/guides/find-limits.md

@@ -40,6 +40,10 @@ Brainy picks `maxLimit` from the first of these that's available:
 
 Worked example: a 4 GB Cloud Run container picks priority 3 → `floor(4 GB × 0.25 / 25 KB) = floor(40 960) = 40 000` results. A 900 MB free-memory box on priority 4 gets `floor(900 MB / 25 KB) = ~36 000`.
 
+The cap is fixed at construction and never changes at runtime. Query timing is recorded
+for diagnostics only — a burst of slow queries cannot silently shrink the cap, and the
+auto-detected tiers (3 and 4) never go below a floor of 10 000.
+
 > **Calibration note.** Pre-7.30.2 used 100 KB per result instead of 25 KB, which produced caps that were 4× too tight for typical workloads (an 8 KB / result reality). 7.30.2 recalibrated to match observed entity sizes; existing `limit: 10_000` safety patterns now pass silently on any reasonably-sized box.
 
 ## What happens when you exceed the cap

docs/guides/import-anything.md

@@ -300,7 +300,10 @@ await brain.import(data, {
  // Deduplication
  enableDeduplication: true, // Check for duplicate entities (default: true)
  deduplicationThreshold: 0.85, // Similarity threshold for duplicates (0-1, default: 0.85)
- // Note: Auto-disabled for imports >100 entities
+ // Notes: false disables BOTH the inline merge and the background pass that
+ // runs ~5 min after the last import (merged duplicates are deleted).
+ // The inline pass auto-disables for imports >100 entities (O(n²) cost);
+ // the background pass still covers those unless the flag is false.
 
  // Performance
  chunkSize: 100, // Batch size for processing (default: varies by operation)

docs/guides/import-quick-reference.md

@@ -86,11 +86,22 @@ await brain.import(file, {
 
 ```typescript
 await brain.import(file, {
-  enableDeduplication: true,         // Check for duplicates (default: false)
+  enableDeduplication: true,         // Check for duplicates (default: true)
   deduplicationThreshold: 0.85       // Similarity threshold (default: 0.85)
 })
 ```
 
+Deduplication merges entities judged duplicates — the non-primary records are
+**deleted**. Set `enableDeduplication: false` to disable it entirely: the flag
+gates both the inline merge during import and the background pass that runs
+about 5 minutes after the last import.
+
+```typescript
+await brain.import(file, {
+  enableDeduplication: false         // No merging, inline or background
+})
+```
+
 ### Import Tracking
 
 Track and organize imports by project:

docs/guides/inspection.md

@@ -166,6 +166,34 @@ brainy inspect diff /data/brain-prod /data/brain-staging
 Sample-based — for a full diff, dump both with `inspect dump` and compare
 the JSONL.
 
+## Auditing graph-read truth
+
+`brain.auditGraph()` (8.6.0+) proves — or disproves — that relationship reads
+return canonical truth on a given brain, without mutating anything. It walks
+every stored relationship record, asks the same read path your application
+uses (`related()`, VFS `readdir`) with every visibility tier included, and
+classifies every discrepancy:
+
+```typescript
+const report = await brain.auditGraph()
+
+report.coherent              // true = related()/readdir can be trusted on this brain
+report.missingFromReadsCount // records the read path omits — stale index
+report.danglingEndpointsCount // relationships whose endpoint entity is gone
+report.readOnlyCount         // read-path edges with NO stored record — ghosts
+report.visibilityHiddenCount // internal/system edges hidden by design (not a fault)
+```
+
+Counts are always exact; the example lists (`missingFromReads`,
+`danglingEndpoints`, `readOnlyVerbIds`) are capped at `maxExamples`
+(default 100) and `truncatedExamples` says so when they are.
+
+Run it after any engine upgrade, restore, or migration. If it reports
+discrepancies, run `brain.repairIndex()` and audit again — a `coherent`
+report after the repair is the verified statement that the heal worked.
+Cost: one relationship-record walk plus one indexed read per distinct
+source entity — safe on a live brain.
+
 ## Repairing a corrupted store
 
 If invariants fail and you suspect index corruption, `inspect repair`

docs/guides/optimistic-concurrency.md

@@ -180,3 +180,35 @@ Brainy 8.0 has exactly two write-coordination counters, at two granularities:
 They compose: a `transact()` batch can carry per-entity `ifRev` checks *and* a whole-store `ifAtGeneration`; any failed check rejects the entire batch before anything is staged. Generations also power snapshots and time travel (`brain.now()`, `brain.asOf()`, `db.persist()`) — see the [consistency model](../concepts/consistency-model.md) and [Snapshots & Time Travel](./snapshots-and-time-travel.md).
 
 A snapshot or historical view captures each entity *including* its `_rev` at that moment, so reading the past and writing back with `ifRev` against the live state works exactly as you'd hope: the write fails if the entity moved since the state you copied from.
+
+## The transact envelope: batch size, budget, and bulk imports
+
+`transact()` applies its batch atomically under one commit — which means the whole batch
+shares one **apply budget**. Since 8.7.0 the budget scales with the batch:
+`max(30 s, opCount × 2 s)`, or exactly what you pass as `timeoutMs`. A tripped budget rolls
+the entire batch back (nothing partial survives) and throws a retryable
+`TransactionTimeoutError` that names the operation it stopped at, the batch size, and the
+elapsed vs budgeted time — a diagnosis, not just a failure:
+
+```
+Transaction timed out at operation 41/120 ('add') — 246012ms elapsed, budget 240000ms.
+The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.
+```
+
+Practical envelope guidance for bulk work:
+
+1. **Precompute embeddings outside the commit path.** Embedding inside `transact()` spends
+   the budget on model inference. Use `brain.embedBatch(texts)` and pass each vector via
+   the op's `vector` field — the commit then pays only storage costs, and a retried batch
+   never re-pays inference. (The win is *where* the inference happens, not raw embedding
+   throughput: on the default WASM engine, batch and sequential embedding measure
+   comparably, ~160 ms/text; native embedding providers may batch faster.)
+2. **Chunk very large imports** into batches of a few hundred ops with one `transact()`
+   each. You lose whole-import atomicity but keep per-chunk atomicity, bounded memory, and
+   resumability — pair with `ifAbsent` upserts so a retried chunk is idempotent.
+3. **Slow disks change the math, not the contract.** On network-attached storage a single
+   op can cost ~2 s (canonical write + fsync + index maintenance). The scaled default
+   absorbs that; pass an explicit `timeoutMs` only when you know better than the scale.
+4. **`addMany`/`relateMany` are the convenience tier** — they chunk and batch-embed for
+   you, with per-item error reporting instead of batch atomicity. Choose by what you need:
+   atomic-all-or-nothing → `transact()`; resilient bulk load → `addMany`.

docs/guides/reacting-to-changes.md

@@ -0,0 +1,117 @@
+---
+title: Reacting to Changes
+slug: guides/reacting-to-changes
+public: true
+category: guides
+template: guide
+order: 11
+description: Subscribe to every committed mutation with `brain.onChange` — the in-process change feed behind live UIs, cache invalidation, and realtime sync. Covers the event shape, delivery guarantees, and catch-up patterns.
+next:
+  - guides/optimistic-concurrency
+  - guides/snapshots-and-time-travel
+---
+
+# Reacting to Changes
+
+`brain.onChange(cb)` is Brainy's in-process change feed: subscribe once and
+receive one event per committed mutation — **every** mutation, regardless of
+how it happened. Direct calls, batch methods, `transact()`, imports, and
+Virtual Filesystem writes all funnel through the same commit point the feed is
+emitted from, so nothing slips past it.
+
+```ts
+const off = brain.onChange((e) => {
+  if (e.kind === 'entity') {
+    console.log(`${e.op} ${e.entity?.type} ${e.id} @ generation ${e.generation}`)
+  }
+})
+
+await brain.add({ data: 'Ada Lovelace', type: 'person' })
+// → "add person 0198... @ generation 42"
+
+off() // unsubscribe when done
+```
+
+## The event
+
+```ts
+interface BrainyChangeEvent {
+  kind: 'entity' | 'relation' | 'store'
+  op: 'add' | 'update' | 'remove' | 'relate' | 'unrelate' | 'updateRelation'
+    | 'clear' | 'restore'
+  id?: string
+  entity?:   { id: string; type: string; subtype?: string;
+               metadata: Record<string, unknown>; service?: string }
+  relation?: { id: string; from: string; to: string; type: string;
+               metadata?: Record<string, unknown> }
+  generation?: number
+  timestamp: number
+}
+```
+
+- **Entity events** (`add` / `update` / `remove`) carry the post-commit indexed
+  view — `type`, `subtype`, and the full custom `metadata`, so you can match
+  your own `where`-style filters against events without a read.
+- **Deletes are fully described.** A `remove` or `unrelate` event carries the
+  record's *last committed state* (sourced from the commit's own history
+  record), not just an id.
+- **Batches emit per item.** `addMany` / `updateMany` / `relateMany` /
+  `removeMany` emit one event per affected record; a `transact()` batch emits
+  one event per item, all sharing the batch's single `generation`.
+- **Cascades are visible.** Removing an entity also emits `unrelate` for each
+  relationship the delete cascaded to.
+- **Store-level events** (`kind: 'store'`) fire for the two wholesale
+  operations — `clear()` and `restore()` — and mean *"everything may have
+  changed; refetch what you care about."*
+
+## Delivery guarantees
+
+- **Post-commit only.** An aborted write — a losing
+  [`ifRev` compare-and-swap](optimistic-concurrency.md), a rejected
+  transaction — never emits. If you received the event, the write is durable.
+- **Commit-ordered.** Events arrive in the order writes committed;
+  `generation` is monotonic.
+- **Asynchronous, never blocking.** Delivery happens in a microtask after the
+  write completes. A slow listener cannot delay a write; a throwing listener
+  is logged and isolated from other listeners.
+- **Zero overhead when unused.** With no subscribers, the write path does no
+  event work at all.
+- **Fire-and-forget.** There is no replay or backpressure. For catch-up after
+  a disconnect, use the `generation` on each event together with
+  [`asOf()` / the transaction log](snapshots-and-time-travel.md): record the
+  last generation you processed, and on reconnect diff from there. For file
+  content specifically, `vfs.readFile(path, { asOf })` and
+  `vfs.history(path)` are the temporal read — see
+  [Snapshots & Time Travel](snapshots-and-time-travel.md).
+
+## Patterns
+
+**Cache invalidation** — drop cached reads for whatever changed:
+
+```ts
+brain.onChange((e) => {
+  if (e.kind === 'store') return cache.clear()
+  if (e.id) cache.delete(e.id)
+})
+```
+
+**Live queries (notify-and-refetch)** — re-run a query when a relevant change
+lands, rather than diffing incrementally:
+
+```ts
+brain.onChange((e) => {
+  if (e.kind === 'entity' && e.entity?.type === 'order') {
+    refreshOpenOrdersView() // debounce as needed
+  }
+})
+```
+
+**Forwarding to other processes** — the feed is in-process by design. To push
+changes to browsers or other services, forward events through your own
+transport (WebSocket, SSE) from the process that owns the brain.
+
+## Lifecycle
+
+`onChange` returns an unsubscribe function — call it when tearing down a
+subscriber (for example, when evicting a pooled instance). `brain.close()`
+drops all listeners; no events are delivered for or after `close()`.

docs/guides/snapshots-and-time-travel.md

@@ -9,6 +9,7 @@ description: Recipes for the Db API — instant backups with persist(), restore,
 next:
   - concepts/consistency-model
   - guides/optimistic-concurrency
+  - guides/external-backups
 ---
 
 # Snapshots & Time Travel
@@ -41,6 +42,10 @@ bytes. Cross-device targets fall back to per-file byte copies, and
 persisting an in-memory brain serializes it to the same directory layout —
 a real, durable store.
 
+> Archiving a brain directory with **external tools** (`tar`, `rsync`, `cp`)?
+> Some index files are sparse and can explode to their apparent size under a
+> naive copy — see [External Backups & Sparse Storage](/docs/guides/external-backups).
+
 Two things to know:
 
 - `persist()` requires the view to still be the store's **latest**
@@ -339,8 +344,12 @@ For per-entity write coordination (rather than whole-store history), the
 ## Keeping history bounded
 
 Under Model-B every write is a generation, so history can grow quickly —
-Brainy auto-compacts on every `flush()`/`close()` under the **`retention`**
-knob (configured on the constructor):
+Brainy auto-compacts at `close()` (time-bounded per pass) under the
+**`retention`** knob (configured on the constructor). Since 8.9.0, `flush()`
+never compacts: flushing is durability work and costs only what the current
+window's writes cost, regardless of history backlog. A long-lived writer that
+never closes keeps its history until its next explicit `compactHistory()` —
+schedule one in your maintenance window if you run bounded retention:
 
 ```typescript
 // Zero-config: ADAPTIVE — keep as much history as free disk/RAM allows,
@@ -354,10 +363,13 @@ new Brainy({ retention: 'all' })
 new Brainy({ retention: { maxGenerations: 1000, maxAge: 7 * 86_400_000, maxBytes: 512 * 1024 ** 2 } })
 ```
 
-Reclaim manually at any time (the same caps):
+Reclaim manually at any time (the same caps, plus an optional per-pass time
+budget for maintenance windows — an early stop is a consistent prefix and the
+next pass resumes):
 
 ```typescript
 await brain.compactHistory({ maxGenerations: 100, maxAge: 7 * 24 * 60 * 60 * 1000 })
+await brain.compactHistory({ maxBytes: 512 * 1024 ** 2, timeBudgetMs: 10_000 })
 ```
 
 Compaction never breaks a pinned read — record-sets are reclaimed only when
@@ -366,6 +378,61 @@ are done with (including the ones `transact()` returns), and `persist()` any
 generation you want to keep beyond the retention window: snapshots are
 self-contained and unaffected by compaction.
 
+## Time travel for files (the VFS)
+
+Since 8.2.0, time travel covers Virtual Filesystem **content**, not just
+entity records. File bytes are retention-protected: a content blob referenced
+by any generation inside the retention window is never reclaimed, so reading
+the past always returns the exact bytes — never a stale field or a
+dangling hash.
+
+**`vfs.readFile(path, { asOf })`** takes a generation number or a `Date` and
+returns the file's exact bytes as they stood then. It resolves the path's
+current entity, then materializes its state at the target generation — so it
+answers *"what did the file at this path hold at that point?"* It bypasses
+the content cache; the `encoding` option still applies. Asking about a
+generation before the file existed throws the usual not-found error, and
+asking past the retention window's compaction horizon throws a
+compacted-generation error.
+
+**`vfs.history(path)`** returns the file's versions inside the retention
+window, oldest first — one `FileVersion` per generation that wrote the file,
+the newest entry being the current state:
+
+```typescript
+// A CMS page evolves…
+await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch"}')
+await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch v2"}')
+await brain.vfs.writeFile('/pages/home.json', '{"title":""}') // bad deploy!
+
+// Every version is listed and readable:
+const versions = await brain.vfs.history('/pages/home.json')
+// → [{ generation, timestamp, hash, size, mimeType? }, …] ascending
+
+const good = versions[versions.length - 2]
+const bytes = await brain.vfs.readFile('/pages/home.json', {
+  asOf: good.generation
+})
+
+// Restore = write the old bytes back. This is a NEW write (a new
+// generation) — history is never rewritten, so the bad version stays
+// visible in the audit trail.
+await brain.vfs.writeFile('/pages/home.json', bytes)
+```
+
+Two lifecycle consequences worth stating plainly:
+
+- **Deleting or overwriting a file no longer frees its bytes immediately.**
+  Old content lives until history compaction reclaims the generations that
+  reference it — the same `retention` budget that bounds all Model-B history
+  (and pinned views are exempt, exactly as above). Size your `retention` for
+  the file-version depth you want; `retention: 'all'` keeps every version of
+  every file forever.
+- **After `compactHistory()` reclaims a generation, its file versions are
+  gone** and their bytes are physically reclaimed. (This also fixed a
+  pre-8.2.0 defect where overwritten content was never reclaimed at all — an
+  unbounded silent leak.)
+
 ## From branches to values
 
 If you used the pre-8.0 `fork`/`checkout`/`commit`/`versions` surface, every

docs/performance-envelopes.md

@@ -0,0 +1,83 @@
+---
+title: Performance Envelopes
+slug: guides/performance-envelopes
+public: true
+category: guides
+template: guide
+order: 40
+description: Measured per-operation latency envelopes at stated scales — what to expect, on what hardware, and exactly how each number was produced.
+next:
+  - guides/find-limits
+---
+
+# Performance Envelopes
+
+Every number on this page is **measured, never projected** — produced by the script
+cited at the bottom, against the built package (the artifact you install), on the stated
+hardware. Each entry says what was measured, at what scale, on which storage backend.
+When a release touches a measured path, that operation is re-measured and this page
+updates in the same release.
+
+Two scopes to keep straight:
+
+- **These envelopes are the pure-JS engine** (no native accelerator registered) on
+  filesystem storage. This is the floor every deployment gets from `npm install` alone.
+- **Accelerated deployments** (the optional native provider) publish their own numbers —
+  this page never claims them.
+
+## Read operations
+
+Reads are where the architecture pays off: after the write path has done its indexing
+work, queries answer from purpose-built indexes without scanning.
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `get(id)` (warm) | p50 < 0.1ms | p50 < 0.1ms | served from cache/metadata index |
+| `find` (metadata: indexed equality + range, limit 100) | p50 1.0ms · p95 1.8ms | p50 7.0ms · p95 8.9ms | column-store bitmap paths |
+| `related(id)` (per-node adjacency) | p50 < 0.1ms · p95 0.2ms | p50 < 0.1ms | LSM adjacency index — O(degree), scale-independent |
+| `find` (semantic: embed + HNSW, 1k docs) | p50 178ms · p95 393ms | — | dominated by WASM query embedding (measured on a machine under concurrent load — treat the p95 as an upper bound); the vector search itself is single-digit ms |
+
+## Write operations
+
+Under Model-B **every write is its own durable generation** — a single-op `add` pays
+serialization, before-image staging, and fsync before it acks. That durability is priced
+into the write path visibly, by design:
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `add` (single-op) | p50 167ms · p95 171ms | p50 165ms · p95 172ms | full durable generation per write — flat across scale |
+| `addMany` (bulk) | ~163ms/entity | ~187ms/entity | **currently per-item commits** — see the honest note below |
+| `relateMany` | ~0.8ms/edge | ~0.9ms/edge | edges batch efficiently today |
+| `flush` (steady-state, 1 pending write) | p50 8ms · p95 10ms | p50 45ms · p95 52ms | durability-only since 8.9.0 — cost no longer depends on history backlog or retention mode |
+
+**The honest note on bulk writes:** `addMany` today commits each item as its own
+generation (the same durability as single-op `add`, serialized by the single-writer
+lock), so bulk-load cost is N × single-op cost. Batched chunk commits (one generation
+and one fsync window per chunk, as `removeMany` already does) are designed into the
+unified-commit work on the current roadmap. Until that ships, size bulk imports
+accordingly — 10k entities is minutes, not seconds, on filesystem storage.
+
+## Open / close
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `open` (empty store) | ~560ms | ~190ms | includes embedder initialization |
+| `open` (warm, populated, clean shutdown) | 763ms | 4.9s | pure-JS vector index load dominates and grows with entity count; the native accelerator exists precisely to remove this |
+| `close` | bounded | bounded | auto-compaction pass is time-bounded (~5s max) since 8.9.0 |
+
+A store that was NOT cleanly closed pays index rebuilds on top of the warm-open
+number (tens of seconds at 10k) — clean shutdown is worth engineering for.
+
+## How these were produced
+
+- **Hardware**: Intel Core i9-14900HX (32 threads), 62GB RAM, NVMe, Linux, Node v22.
+- **Backend**: `storage: { type: 'filesystem' }`, pure JS (no native providers).
+- **Embeddings**: deterministic stub for non-semantic ops (isolates engine cost);
+  the real WASM embedder for the semantic row (that's what you'll run).
+- **Method**: p50/p95 over 50–200 samples per op against the built `dist/`;
+  the measuring script ships in the repo history and re-runs per release.
+
+Numbers on different hardware will differ; the *shape* (sub-2ms indexed reads,
+~160ms embedding-bound semantic queries, durability-priced writes) is the envelope
+you should hold your deployment against. If your measurements diverge from these
+shapes by an order of magnitude, something is wrong — file it.

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.0.13",
+  "version": "8.9.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "@soulcraft/brainy",
-      "version": "8.0.13",
+      "version": "8.9.0",
       "license": "MIT",
       "dependencies": {
         "@msgpack/msgpack": "^3.1.2",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.0.13",
+  "version": "8.9.0",
   "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
   "main": "dist/index.js",
   "module": "dist/index.js",

scripts/push-docs.js

@@ -0,0 +1,116 @@
+#!/usr/bin/env node
+/**
+ * @module scripts/push-docs
+ * @description Push this repo's PUBLIC docs to the soulcraft.com docs ingest
+ * door after an npm publish (VENUE-DOCS-RELEASE-PUSH — retires the old
+ * build-time docs sync).
+ *
+ * Contract (mirrors the reference implementation on the serving side):
+ *   POST {base}/api/docs/ingest
+ *   headers: x-service-secret: $DOCS_INGEST_SECRET, Content-Type: application/json
+ *   body: { docs: [{ slug, title, markdown, nav: { order, section } }] }
+ *   batches of 10, idempotent per slug.
+ *
+ * A doc is public iff its frontmatter has `public: true` AND a `slug`. The
+ * frontmatter is stripped; `category` → nav.section, `order` → nav.order.
+ *
+ * Deliberately NOT pushed: the combined /docs landing index. It spans BOTH
+ * engine corpora (this repo's and the native accelerator's), so a per-repo
+ * push would clobber the union — the index is authored on the serving side.
+ *
+ * Env: DOCS_INGEST_SECRET (required), DOCS_INGEST_BASE (default
+ * https://soulcraft.com). Exits 0 with a LOUD warning when the secret is
+ * absent (the npm publish has already happened; the serving side runs its
+ * interim sync on request) and exits 1 when a push actually fails — the docs
+ * site would silently trail npm otherwise, and that must be visible.
+ */
+import * as fs from 'node:fs'
+import * as path from 'node:path'
+
+const BASE = (process.env.DOCS_INGEST_BASE || 'https://soulcraft.com').replace(/\/+$/, '')
+const SECRET = process.env.DOCS_INGEST_SECRET
+const DOCS_DIR = path.join(path.dirname(new URL(import.meta.url).pathname), '..', 'docs')
+const BATCH = 10
+
+if (!SECRET) {
+  console.warn(
+    '⚠️  DOCS PUSH SKIPPED: DOCS_INGEST_SECRET is not set.\n' +
+      '    soulcraft.com/docs now TRAILS this npm release until docs are pushed.\n' +
+      '    Either export DOCS_INGEST_SECRET and re-run `node scripts/push-docs.js`,\n' +
+      '    or ping venue on VENUE-DOCS-RELEASE-PUSH for the interim sync.'
+  )
+  process.exit(0)
+}
+
+/** Minimal frontmatter split — returns [meta, body] or [null, raw]. */
+function parseFrontmatter(raw) {
+  const m = raw.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/)
+  if (!m) return [null, raw]
+  const meta = {}
+  for (const line of m[1].split('\n')) {
+    const kv = line.match(/^(\w[\w-]*):\s*(.*)$/)
+    if (kv) meta[kv[1]] = kv[2].trim().replace(/^["']|["']$/g, '')
+  }
+  return [meta, m[2]]
+}
+
+const docs = []
+;(function walk(dir) {
+  for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+    const full = path.join(dir, entry.name)
+    if (entry.isDirectory()) walk(full)
+    else if (entry.name.endsWith('.md')) {
+      const [meta, body] = parseFrontmatter(fs.readFileSync(full, 'utf-8'))
+      if (!meta || meta.public !== 'true' || !meta.slug) continue
+      docs.push({
+        slug: meta.slug,
+        title: meta.title || meta.slug,
+        markdown: body.trim(),
+        nav: {
+          order: Number.parseInt(meta.order || '99', 10) || 99,
+          section: meta.category || 'guides'
+        }
+      })
+    }
+  }
+})(DOCS_DIR)
+
+if (docs.length === 0) {
+  console.error('❌ DOCS PUSH FAILED: zero public docs collected — refusing to push an empty corpus.')
+  process.exit(1)
+}
+docs.sort((a, b) => a.slug.localeCompare(b.slug))
+console.log(`Pushing ${docs.length} public docs to ${BASE}/api/docs/ingest …`)
+
+let failed = false
+for (let i = 0; i < docs.length; i += BATCH) {
+  const batch = docs.slice(i, i + BATCH)
+  try {
+    const res = await fetch(`${BASE}/api/docs/ingest`, {
+      method: 'POST',
+      headers: {
+        'x-service-secret': SECRET,
+        'Content-Type': 'application/json',
+        'User-Agent': 'brainy-docs-push/1.0'
+      },
+      body: JSON.stringify({ docs: batch }),
+      signal: AbortSignal.timeout(120_000)
+    })
+    if (!res.ok) {
+      throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`)
+    }
+    console.log(`  batch ${i / BATCH + 1}: ${batch.map((d) => d.slug).join(', ')} → ok`)
+  } catch (err) {
+    failed = true
+    console.error(`  batch ${i / BATCH + 1} FAILED: ${err instanceof Error ? err.message : err}`)
+  }
+}
+
+if (failed) {
+  console.error(
+    '❌ DOCS PUSH INCOMPLETE — soulcraft.com/docs may trail npm. ' +
+      'Re-run `node scripts/push-docs.js` or ping venue on VENUE-DOCS-RELEASE-PUSH.'
+  )
+  process.exit(1)
+}
+console.log('✅ Docs pushed.')

scripts/release.sh

@@ -196,6 +196,18 @@ else
 fi
 echo -e "${GREEN}✅ GitHub release created${NC}\n"
 
+# Step 12: Push public docs to the soulcraft.com docs ingest door
+# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
+# DOCS_INGEST_SECRET is unset; fails loudly (without undoing the publish —
+# that already happened) when a push errors, so the docs site never
+# silently trails npm.
+echo -e "${BLUE}1️⃣2️⃣ Pushing public docs to soulcraft.com/docs...${NC}"
+if node scripts/push-docs.js; then
+  echo -e "${GREEN}✅ Docs push step done${NC}\n"
+else
+  echo -e "${RED}❌ Docs push FAILED — soulcraft.com/docs trails npm until re-run or interim sync${NC}\n"
+fi
+
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
 echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}"
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"

src/aggregation/AggregationIndex.ts

@@ -29,6 +29,7 @@ import { matchesMetadataFilter } from '../utils/metadataFilter.js'
 import { compareCodePoints } from '../utils/collation.js'
 import { bucketTimestamp } from './timeWindows.js'
 import { NounType } from '../types/graphTypes.js'
+import { prodLog } from '../utils/logger.js'
 
 /** Persistence key for aggregate definitions */
 const DEFINITIONS_KEY = '__aggregation_definitions__'
@@ -87,10 +88,18 @@ function matchesSource(entity: Record<string, unknown>, source: AggregateDefinit
     if (entity.service !== source.service) return false
   }
 
-  // Metadata where filter — match against the entity's metadata sub-object
+  // Where filter — resolve each filtered field through resolveEntityField,
+  // the SAME single source of truth groupBy uses (top-level standard fields
+  // + custom metadata). Matching only the metadata sub-object made
+  // where:{subtype}/{visibility}/… a silent no-op: reserved fields never
+  // live in the custom bag, so those filters could never match anything.
   if (source.where && Object.keys(source.where).length > 0) {
-    const metadata = (entity.metadata ?? entity) as Record<string, unknown>
-    if (!matchesMetadataFilter(metadata, source.where)) return false
+    const e = entity as unknown as HNSWNounWithMetadata
+    const resolved: Record<string, unknown> = {}
+    for (const key of Object.keys(source.where)) {
+      resolved[key] = resolveEntityField(e, key)
+    }
+    if (!matchesMetadataFilter(resolved, source.where)) return false
   }
 
   return true
@@ -327,6 +336,30 @@ export class AggregationIndex {
   /** Track aggregates with stale MIN/MAX (need lazy recompute) */
   private staleMinMax = new Map<string, Set<string>>()
 
+  /** Resolves when init() has finished loading persisted definitions/state. */
+  private initPromise: Promise<void> | null = null
+
+  /** True once init() has settled (success or failure). */
+  private initDone = false
+
+  /**
+   * Aggregates registered by the app before init() finished loading persisted
+   * state, awaiting reconciliation: init() adopts the persisted state when the
+   * definition hash matches; anything left unadopted when init settles resolves
+   * to a backfill. Deciding backfill eagerly at define time was the boot-order
+   * bug that wiped valid persisted state on every restart — the synchronous
+   * defineAggregate() always beats the async init().
+   */
+  private pendingAdopt = new Set<string>()
+
+  /**
+   * In-flight rescan targets. While a name has a staging map, ALL
+   * contributions (the walk's and concurrent write hooks') land there instead
+   * of the live map; the live map keeps serving until {@link finishBackfill}
+   * swaps the staging map in atomically.
+   */
+  private backfillStaging = new Map<string, Map<string, AggregateGroupState>>()
+
   constructor(storage: StorageAdapter, nativeProvider?: AggregationProvider) {
     this.storage = storage
     this.nativeProvider = nativeProvider
@@ -336,21 +369,128 @@ export class AggregationIndex {
 
   /**
    * Initialize: load persisted definitions and state, detect changes, rebuild stale.
+   *
+   * Idempotent — repeated calls return the same promise. Definitions registered
+   * *before* this completes (the normal boot order: `defineAggregate()` is
+   * synchronous and always beats this async load) are reconciled rather than
+   * clobbered: the app's definition wins, and its persisted state is adopted
+   * when the definition hash matches — backfill happens only on a real change.
    */
-  async init(): Promise<void> {
+  init(): Promise<void> {
+    if (!this.initPromise) {
+      this.initPromise = this.loadPersisted().finally(() => {
+        this.resolvePendingAdoptToBackfill()
+        this.initDone = true
+      })
+    }
+    return this.initPromise
+  }
+
+  /**
+   * Await the persisted-state load (if one was started) and settle every
+   * pending adoption decision. After this resolves, `getPendingBackfills()`
+   * is authoritative: a name is listed iff it genuinely needs a rescan.
+   * Query paths must await this before consulting backfill state.
+   */
+  async ready(): Promise<void> {
+    if (this.initPromise) {
+      try {
+        await this.initPromise
+      } catch {
+        // The owner already surfaced the load failure loudly; backfill covers.
+      }
+    }
+    this.resolvePendingAdoptToBackfill()
+  }
+
+  /**
+   * Any definition still awaiting state adoption has no persisted state to
+   * adopt (or init never ran / failed) — it must backfill.
+   */
+  private resolvePendingAdoptToBackfill(): void {
+    if (this.pendingAdopt.size > 0) {
+      prodLog.info(
+        `[Aggregation] no adoptable persisted state for: ${Array.from(this.pendingAdopt).join(', ')} — flagged for backfill`
+      )
+    }
+    for (const name of this.pendingAdopt) this.needsBackfill.add(name)
+    this.pendingAdopt.clear()
+  }
+
+  /**
+   * May this persisted state be ADOPTED? When the store exposes its committed
+   * watermark, the state's `sourceGeneration` must EQUAL it: behind means
+   * later writes are missing from the state (unclean shutdown); ahead means
+   * it counts writes that no longer exist (e.g. a fact-log truncation on a
+   * copied store pulled the watermark back). Either way: one exact rescan,
+   * said out loud — never a silent adopt. Stores without the capability (and
+   * pre-stamp state on them) fall back to hash-only adoption.
+   */
+  private stateGenerationAdoptable(name: string, stateData: unknown): boolean {
+    const committed = this.storage.committedGeneration?.() ?? null
+    if (committed === null) return true
+    const raw = (stateData as Record<string, unknown>).sourceGeneration
+    const stamped = typeof raw === 'number' ? raw : null
+    if (stamped === committed) return true
+    prodLog.warn(
+      `[Aggregation] '${name}': persisted state is at generation ${stamped ?? 'unstamped'} ` +
+        `but the store's committed generation is ${committed} — rescanning instead of adopting`
+    )
+    return false
+  }
+
+  private async loadPersisted(): Promise<void> {
     // Load persisted definitions
     const savedDefs = await this.storage.getMetadata(DEFINITIONS_KEY)
     if (savedDefs && typeof savedDefs === 'object' && savedDefs.definitions) {
       const defs = savedDefs.definitions as Array<AggregateDefinition & { _hash?: string }>
 
       for (const def of defs) {
-        this.definitions.set(def.name, def)
-        const currentHash = hashDefinition(def)
         const savedHash = def._hash || ''
 
-        // Load persisted state
+        if (this.definitions.has(def.name)) {
+          // The app re-registered this aggregate before the load finished.
+          // The app's definition wins — never clobber it with the persisted
+          // copy. Adopt the persisted state when the definition is unchanged
+          // AND no write has landed for it yet (a landed write would be lost
+          // by adoption; the hook flips such names to backfill).
+          const appHash = this.definitionHashes.get(def.name) || ''
+          if (appHash === savedHash && this.pendingAdopt.has(def.name)) {
+            const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
+            if (
+              stateData &&
+              stateData.groups &&
+              this.stateGenerationAdoptable(def.name, stateData)
+            ) {
+              const groupMap = new Map<string, AggregateGroupState>()
+              for (const group of stateData.groups as AggregateGroupState[]) {
+                groupMap.set(serializeGroupKey(group.groupKey), group)
+              }
+              this.states.set(def.name, groupMap)
+              this.pendingAdopt.delete(def.name)
+              this.needsBackfill.delete(def.name)
+              prodLog.info(
+                `[Aggregation] '${def.name}': adopted persisted state (${groupMap.size} groups) — no rescan`
+              )
+            }
+            // No/invalid persisted state: stays in pendingAdopt and resolves
+            // to backfill when init settles.
+          }
+          continue
+        }
+
+        // Not registered this session — restore definition + state from
+        // persistence.
+        this.definitions.set(def.name, def)
+        const currentHash = hashDefinition(def)
+
         const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
-        if (stateData && stateData.groups && savedHash === currentHash) {
+        if (
+          stateData &&
+          stateData.groups &&
+          savedHash === currentHash &&
+          this.stateGenerationAdoptable(def.name, stateData)
+        ) {
           // Definition unchanged — load state
           const groupMap = new Map<string, AggregateGroupState>()
           for (const group of stateData.groups as AggregateGroupState[]) {
@@ -358,6 +498,10 @@ export class AggregationIndex {
             groupMap.set(serialized, group)
           }
           this.states.set(def.name, groupMap)
+          this.needsBackfill.delete(def.name)
+          prodLog.info(
+            `[Aggregation] '${def.name}': restored definition + adopted persisted state (${groupMap.size} groups)`
+          )
         } else {
           // Definition changed or no saved state — start fresh and backfill from
           // existing entities (the owner drains needsBackfill on first query).
@@ -398,14 +542,22 @@ export class AggregationIndex {
     }))
     await this.storage.saveMetadata(DEFINITIONS_KEY, { definitions: defsToSave })
 
-    // Persist dirty states
+    // Persist dirty states, stamped with the committed generation they
+    // reflect. The stamp is what makes reopen-adoption verifiable: state at a
+    // different generation than the store's committed watermark is stale (an
+    // unclean shutdown after later writes) or over-counts (a fact-log
+    // truncation on a copied store pulled the watermark BACK below the
+    // stamp) — either way the answer is one exact rescan, never a silent
+    // adopt. Read the generation after collecting groups so any racing
+    // commit resolves toward rescan, not wrong-adopt.
     for (const name of this.dirty) {
       const stateMap = this.states.get(name)
       if (stateMap) {
         const groups = Array.from(stateMap.values())
+        const sourceGeneration = this.storage.committedGeneration?.() ?? null
         await this.storage.saveMetadata(
           `${STATE_KEY_PREFIX}${name}__`,
-          { groups }
+          sourceGeneration === null ? { groups } : { groups, sourceGeneration }
         )
       }
     }
@@ -452,10 +604,19 @@ export class AggregationIndex {
     this.definitions.set(def.name, def)
     this.definitionHashes.set(def.name, newHash)
 
+    // First sight this session, before init() settled: defer the backfill
+    // decision — init() adopts the persisted state on hash match, and anything
+    // left unadopted resolves to backfill. Deciding eagerly here wiped valid
+    // persisted state on every restart.
+    if (!this.states.has(def.name) && !this.initDone) {
+      this.states.set(def.name, new Map())
+      this.pendingAdopt.add(def.name)
+    }
     // Reset state if definition changed or doesn't exist yet, and flag it for
     // backfill so already-stored entities are counted (write-time hooks only see
     // future writes). The owner drains this on the next query via getPendingBackfills().
-    if (!this.states.has(def.name) || (oldHash && oldHash !== newHash)) {
+    else if (!this.states.has(def.name) || (oldHash && oldHash !== newHash)) {
+      this.pendingAdopt.delete(def.name)
       this.states.set(def.name, new Map())
       this.needsBackfill.add(def.name)
     }
@@ -476,6 +637,8 @@ export class AggregationIndex {
     this.definitionHashes.delete(name)
     this.states.delete(name)
     this.staleMinMax.delete(name)
+    this.pendingAdopt.delete(name)
+    this.needsBackfill.delete(name)
 
     // Notify native provider
     if (this.nativeProvider?.removeAggregate) {
@@ -513,9 +676,17 @@ export class AggregationIndex {
     return Array.from(this.needsBackfill)
   }
 
-  /** Clear an aggregate's state so a full rescan cannot double-count. */
+  /**
+   * Begin a rescan into a STAGING map. The live state is not touched — it
+   * keeps serving (possibly stale, but flagged pending) until the rescan
+   * completes and swaps in atomically. A mid-walk failure drops the staging
+   * map via {@link abortBackfill} and loses nothing: wiping live state before
+   * a scan that could throw was the destructive-before-durable defect.
+   * Contributions (walk + concurrent write hooks) land in staging while it
+   * exists, so the swapped-in result reflects writes that raced the walk.
+   */
   beginBackfill(name: string): void {
-    this.states.set(name, new Map())
+    this.backfillStaging.set(name, new Map())
     // Reset native provider state for this aggregate too, if present.
     const def = this.definitions.get(name)
     if (def && this.nativeProvider?.removeAggregate && this.nativeProvider?.defineAggregate) {
@@ -524,6 +695,15 @@ export class AggregationIndex {
     }
   }
 
+  /**
+   * Abandon an in-flight rescan after a failure: drop the staging map, keep
+   * the live state serving, leave the aggregate flagged as pending so a later
+   * attempt rescans. The failure itself must be surfaced loudly by the owner.
+   */
+  abortBackfill(name: string): void {
+    this.backfillStaging.delete(name)
+  }
+
   /** Feed one already-stored entity into a single aggregate during backfill. */
   backfillEntity(name: string, entity: Record<string, unknown>): void {
     if (isAggregateEntity(entity)) return
@@ -537,14 +717,33 @@ export class AggregationIndex {
     }
   }
 
-  /** Mark an aggregate's backfill complete; rebuilt state persists on next flush(). */
+  /** Swap the rebuilt staging state in atomically; persists on next flush(). */
   finishBackfill(name: string): void {
+    const staged = this.backfillStaging.get(name)
+    if (staged) {
+      this.states.set(name, staged)
+      this.backfillStaging.delete(name)
+    }
     this.needsBackfill.delete(name)
     this.dirty.add(name)
   }
 
   // ============= Write-Time Hooks =============
 
+  /**
+   * A write is landing for an aggregate whose persisted-state adoption is still
+   * pending — adopting after this write would lose its contribution. Settle the
+   * decision now: an exact rescan instead of adoption. The window is the few
+   * milliseconds between a boot-time defineAggregate() and init() completing,
+   * so this rarely fires; when it does, correctness wins over the walk.
+   */
+  private resolveAdoptOnWrite(name: string): void {
+    if (this.pendingAdopt.has(name)) {
+      this.pendingAdopt.delete(name)
+      this.needsBackfill.add(name)
+    }
+  }
+
   /**
    * Called when an entity is added. Updates all matching aggregates.
    */
@@ -553,6 +752,7 @@ export class AggregationIndex {
 
     for (const [name, def] of this.definitions) {
       if (!matchesSource(entity, def.source)) continue
+      this.resolveAdoptOnWrite(name)
 
       if (this.nativeProvider) {
         const results = this.nativeProvider.incrementalUpdate(name, def, entity, 'add')
@@ -579,6 +779,10 @@ export class AggregationIndex {
       const oldMatches = matchesSource(oldEntity, def.source)
       const newMatches = matchesSource(newEntity, def.source)
 
+      if (oldMatches || newMatches) {
+        this.resolveAdoptOnWrite(name)
+      }
+
       if (this.nativeProvider && (oldMatches || newMatches)) {
         const results = this.nativeProvider.incrementalUpdate(name, def, newEntity, 'update', oldEntity)
         this.applyNativeResults(name, results)
@@ -605,6 +809,7 @@ export class AggregationIndex {
 
     for (const [name, def] of this.definitions) {
       if (!matchesSource(entity, def.source)) continue
+      this.resolveAdoptOnWrite(name)
 
       if (this.nativeProvider) {
         const results = this.nativeProvider.incrementalUpdate(name, def, entity, 'delete')
@@ -757,7 +962,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: an unnest dimension makes one entity contribute to several groups.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -815,7 +1020,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: reverse the entity's contribution from every group it joined.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -871,7 +1076,7 @@ export class AggregationIndex {
    * Apply results from native provider back into the state maps.
    */
   private applyNativeResults(aggName: string, results: AggregateGroupState[]): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
     for (const group of results) {
       const serialized = serializeGroupKey(group.groupKey)
       stateMap.set(serialized, group)

src/aggregation/materializer.ts

@@ -14,6 +14,7 @@ import type {
   AggregateMetricDef
 } from '../types/brainy.types.js'
 import { serializeGroupKey } from './AggregationIndex.js'
+import { prodLog } from '../utils/logger.js'
 
 /**
  * Callback interface for the materializer to create/update Brainy entities.
@@ -86,8 +87,15 @@ export class AggregateMaterializer {
     if (existing) clearTimeout(existing)
 
     this.debounceTimers.set(key, setTimeout(() => {
-      this.materializeOne(key).catch(() => {
-        // Non-fatal — materialization is derived data
+      this.materializeOne(key).catch((err) => {
+        // Materialization is derived data (rebuildable via backfill-on-query), so
+        // a failure is non-fatal — but it leaves the materialized Measurement
+        // entity STALE. Surface it loudly rather than swallow it silently.
+        prodLog.warn(
+          `[Aggregation] Failed to materialize aggregate group '${key}': ` +
+          `${(err as Error).message}. The materialized value is stale until the ` +
+          `next successful materialization or a backfill-on-query.`
+        )
       })
     }, debounceMs))
   }

src/coreTypes.ts

@@ -754,6 +754,39 @@ export interface Change {
   data?: HNSWNounWithMetadata | HNSWVerbWithMetadata
 }
 
+/**
+ * @description A declared derived-index blob FAMILY (the
+ * registered-blob contract). A family names the set of on-disk blobs that a
+ * derived index needs AS A SET (e.g. the vector base = `main.dkann` +
+ * `main.slotmap` + `main.slotrev`): losing ANY member corrupts the index. Once a
+ * family is declared:
+ * - its members are UNDELETABLE through the storage layer — `deleteBinaryBlob` /
+ *   `removeRawPrefix` refuse with a `ProtectedArtifactError`, so an in-process
+ *   GC/sweeper cannot remove a load-bearing file (intentional retirement =
+ *   `unregisterDerivedFamily` first);
+ * - a member missing on open is a loud `DerivedArtifactMissingError` → rebuild.
+ * `COLD ≠ DEAD`: a write-once segment or a recovery-critical archive is
+ * load-bearing even when it has not been touched in a long time.
+ */
+export interface DerivedFamilyDeclaration {
+  /** Stable family id, e.g. `'vector-base'` / `'metadata-sstables'`. */
+  name: string
+  /**
+   * The logical blob keys that make up the family. When {@link namespace} is
+   * set, each entry is a PREFIX protecting every key beneath it (for growing
+   * sets like `seg-*`); otherwise each entry is an exact member key.
+   */
+  members: string[]
+  /** When true, {@link members} are prefixes (protect all keys beneath each). */
+  namespace?: boolean
+  /**
+   * Whether a missing family can be rebuilt from the canonical records (default
+   * `true`). `false` marks an irreplaceable family (a missing member is data
+   * loss, not a rebuild).
+   */
+  rebuildable?: boolean
+}
+
 export interface StorageAdapter {
   init(): Promise<void>
 
@@ -815,7 +848,17 @@ export interface StorageAdapter {
    */
   getNounsByNounType(nounType: string): Promise<HNSWNounWithMetadata[]>
 
-  deleteNoun(id: string): Promise<void>
+  /**
+   * Delete a noun — FULL canonical removal (both legs + the entity container).
+   *
+   * @param id The entity id.
+   * @param priorMetadata OPTIONAL already-known metadata of the entity being
+   *   removed (the caller's pre-delete read). The count decrement must never
+   *   REQUIRE re-reading the record being removed: when the internal read
+   *   returns `null` (replace race, or a ghost left by an earlier version) the
+   *   decrement falls back to this record instead of being silently skipped.
+   */
+  deleteNoun(id: string, priorMetadata?: NounMetadata | null): Promise<void>
 
   /**
    * Save verb - Pure HNSW verb with core fields only
@@ -872,7 +915,15 @@ export interface StorageAdapter {
    */
   getVerbsByType(type: string): Promise<HNSWVerbWithMetadata[]>
 
-  deleteVerb(id: string): Promise<void>
+  /**
+   * Delete a verb — FULL canonical removal (both legs + the container).
+   *
+   * @param id The relationship id.
+   * @param priorMetadata OPTIONAL already-known metadata of the edge being
+   *   removed (the caller's pre-delete read); keeps the count decrement honest
+   *   when the internal read returns `null` (see `deleteNoun`).
+   */
+  deleteVerb(id: string, priorMetadata?: VerbMetadata | null): Promise<void>
 
   /**
    * Save metadata
@@ -1049,6 +1100,104 @@ export interface StorageAdapter {
    */
   getBinaryBlobPath(key: string): string | null
 
+  /**
+   * @description OPTIONAL (registered-blob contract). Declare a
+   * derived-index blob {@link DerivedFamilyDeclaration | family} whose members
+   * become UNDELETABLE through this adapter — a subsequent `deleteBinaryBlob` /
+   * `removeRawPrefix` that would remove a declared member throws a
+   * `ProtectedArtifactError`. Providers declare their families on create; the
+   * declaration is persisted so protection survives a reopen. Idempotent per
+   * `name` (re-declaring replaces).
+   * @param family - The family to protect.
+   */
+  registerDerivedFamily?(family: DerivedFamilyDeclaration): Promise<void>
+
+  /**
+   * @description OPTIONAL. Remove a family's protection so its members can be
+   * deleted again — the explicit, auditable step for intentional retirement of a
+   * derived index (the ONLY way a declared member becomes deletable).
+   * @param name - The {@link DerivedFamilyDeclaration.name} to unregister.
+   */
+  unregisterDerivedFamily?(name: string): Promise<void>
+
+  /**
+   * @description OPTIONAL. List the currently-declared derived-index families —
+   * the source of truth for what `clear()` must wipe and what a
+   * missing-on-open check verifies.
+   */
+  listDerivedFamilies?(): Promise<DerivedFamilyDeclaration[]>
+
+  /**
+   * @description OPTIONAL binary raw-byte primitives — the substrate for
+   * append-only log-structured files (the generation fact log's CRC-framed
+   * segments). Feature-detected: an adapter that omits them simply hosts no
+   * fact log (readers fall back to canonical enumeration). Paths are
+   * storage-root-relative and used VERBATIM (no `.gz`/`.bin` suffixing —
+   * unlike the JSON object and blob primitives).
+   *
+   * Append to a raw binary file, creating it (and parent directories) when
+   * absent. Append durability is the CALLER's job via `syncRawObjects` —
+   * matching the commit protocol, which batches fsyncs at its barrier.
+   */
+  appendRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+
+  /**
+   * Read a raw binary file whole. Absent → `null`; a real IO fault throws
+   * (never masked as absence).
+   */
+  readRawBytes?(path: string): Promise<Uint8Array | null>
+
+  /**
+   * Replace a raw binary file atomically (write-new → fsync → rename) — the
+   * reconcile primitive (e.g. truncating a fact-log tail back to committed
+   * truth after a crash).
+   */
+  writeRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+
+  /**
+   * Byte size of a raw binary file, or `null` when absent.
+   */
+  rawByteSize?(path: string): Promise<number | null>
+
+  /**
+   * @description OPTIONAL fact-scan capability — how an index provider that
+   * holds only `storage` reaches the generation fact log (the host brain
+   * wires it at init; providers must never construct their own fact-log
+   * reader — the log's open path is writer-side). Present ⟺ this store hosts
+   * a fact log AND the host wired the capability. Returns a scan handle over
+   * committed facts (heal-grade telemetry included), or `null` when no fact
+   * log exists — callers fall back to the canonical enumeration walk.
+   */
+  scanFacts?(options?: {
+    fromGeneration?: number
+    toGeneration?: number
+    kinds?: Array<'noun' | 'verb'>
+    batchSize?: number
+  }): import('./db/factLog.js').FactScanHandle | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the fact log's
+   * head generation — the replay target for `stamp.sourceGeneration + 1`
+   * catch-ups. `null` when no fact log exists.
+   */
+  factLogHeadGeneration?(): number | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the COMMITTED
+   * generation watermark — the manifest truth a projection's
+   * `sourceGeneration` compares against. Exposed as a capability so a
+   * provider never parses the store's private manifest format. `null` when
+   * the capability is unwired.
+   */
+  committedGeneration?(): number | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the immutable,
+   * sealed fact-segment file paths covering `fromGeneration` — the zero-copy
+   * handoff. The mutable tail is excluded (read it via `scanFacts`).
+   */
+  factSegmentPaths?(options?: { fromGeneration?: number }): string[]
+
   /**
    * Save statistics data
    * @param statistics The statistics data to save

src/db/db.ts

@@ -945,6 +945,13 @@ export class Db<T = any> {
    * {@link SpeculativeOverlayError} (commit them with `brain.transact()`
    * first).
    *
+   * SPARSE FILES: a native accelerator's mmap index files can be sparse —
+   * huge apparent size, small allocated size. `persist()` handles them
+   * correctly (hard links share the allocation). But if you then archive the
+   * snapshot with EXTERNAL tools, use the sparse-aware flags (`tar czSf`,
+   * `rsync --sparse`, `cp --sparse=always`) or the copy materializes every
+   * hole — see docs/guides/external-backups-and-sparse-storage.md.
+   *
    * @param path - Absolute directory for the snapshot (created; must be
    *   empty or absent).
    * @throws GenerationConflictError when this view is no longer the latest

src/db/errors.ts

@@ -159,3 +159,133 @@ export class GenerationCompactedError extends Error {
     this.horizon = horizon
   }
 }
+
+/** One entity/relationship left in an unreconciled state by a failed rollback. */
+export interface UnreconciledRecord {
+  /** The entity or relationship id. */
+  id: string
+  /** `'noun'` (entity) or `'verb'` (relationship). */
+  kind: 'noun' | 'verb'
+  /**
+   * `'orphan'` — the record is durably PRESENT but the transaction aborted
+   * (an add whose delete-undo failed); `'loss'` — the record is durably GONE
+   * or wrong when it should have been restored (a remove/update whose
+   * restore-undo failed — actual data loss).
+   */
+  disposition: 'orphan' | 'loss'
+}
+
+/**
+ * @description Thrown when a transaction's rollback could not be fully applied
+ * and the resulting inconsistency cannot be safely adopted forward — i.e. a
+ * multi-operation batch, or ANY case where a record was lost (a remove/update
+ * whose restore-undo failed). The store is left in a known-inconsistent state:
+ * the {@link records} name every entity/relationship whose canonical state no
+ * longer matches what the aborted transaction should have produced.
+ *
+ * On throw, the brain enters **write-quarantine** — reads continue to work, but
+ * further writes are refused until {@link } `repairIndex()` reconciles the
+ * derived indexes against canonical storage and lifts the quarantine. This is
+ * the honest, loud failure: a visible inconsistency the operator must repair,
+ * never a silent partial write. The generation counter is NOT advanced.
+ *
+ * (A SINGLE-op add whose only damage is a durably-present orphan is NOT this
+ * error: it is adopted forward as a committed generation and returned as a
+ * degraded-but-successful write — the record the caller asked for exists.)
+ *
+ * @example
+ * try {
+ *   await brain.transact(ops)
+ * } catch (err) {
+ *   if (err instanceof StoreInconsistentError) {
+ *     console.error('store inconsistent:', err.records) // ids + dispositions
+ *     await brain.repairIndex() // reconcile + lift the write-quarantine
+ *   }
+ * }
+ */
+export class StoreInconsistentError extends Error {
+  /** Every record left in an unreconciled state by the failed rollback. */
+  public readonly records: UnreconciledRecord[]
+  /** The original error that triggered the (then-failed) rollback. */
+  public override readonly cause: Error
+
+  /**
+   * @param records - The unreconciled entities/relationships (ids + disposition).
+   * @param cause - The error that triggered the rollback.
+   */
+  constructor(records: UnreconciledRecord[], cause: Error) {
+    const orphans = records.filter((r) => r.disposition === 'orphan').length
+    const losses = records.filter((r) => r.disposition === 'loss').length
+    super(
+      `Store left inconsistent by a failed transaction rollback: ` +
+        `${records.length} record(s) could not be reconciled ` +
+        `(${orphans} orphaned, ${losses} lost). ` +
+        `The brain is now WRITE-QUARANTINED (reads still work) — run repairIndex() ` +
+        `to reconcile the derived indexes against canonical storage and lift the ` +
+        `quarantine. Ids: ${records.map((r) => `${r.id}:${r.disposition}`).join(', ')}. ` +
+        `Cause: ${cause.message}`
+    )
+    this.name = 'StoreInconsistentError'
+    this.records = records
+    this.cause = cause
+  }
+}
+
+/**
+ * @description Thrown by a write when the store cannot make single-op
+ * generation **history** durable: the asynchronous group-commit flush that
+ * persists buffered before-images to disk has failed repeatedly (a real
+ * storage fault — a full disk, an EIO, a permission loss — not a transient
+ * blip). Rather than keep accepting writes whose history silently piles up in
+ * memory and is never persisted — an invisible durability loss, and an
+ * unbounded leak — the store LATCHES this failure and refuses further writes
+ * until the pending tier drains.
+ *
+ * This is the loud, honest response to a broken history-durability path:
+ * - Live canonical data is unaffected (single-op live bytes are written before
+ *   the history flush; only the immutable before-image history is stuck).
+ * - The background flush keeps retrying with backoff; when the underlying fault
+ *   clears and a flush succeeds, the latch lifts and writes resume
+ *   automatically. An explicit `flush()` also clears it on success.
+ * - {@link cause} is the underlying storage error from the last failed flush.
+ *
+ * A caller seeing this should treat it exactly like a full disk: stop writing,
+ * resolve the storage fault, and retry. It is NOT a data-corruption error — no
+ * committed generation is lost — it is a refusal to *promise* durability the
+ * store currently cannot deliver.
+ *
+ * @example
+ * try {
+ *   await brain.add({ ... })
+ * } catch (err) {
+ *   if (err instanceof PendingFlushDurabilityError) {
+ *     // History can't be persisted right now (disk fault). Resolve storage,
+ *     // then retry — the store self-heals once a flush succeeds.
+ *     console.error('history durability stalled:', err.cause)
+ *   }
+ * }
+ */
+export class PendingFlushDurabilityError extends Error {
+  /** The underlying storage error from the most recent failed flush. */
+  public override readonly cause: Error
+  /** How many consecutive flush attempts had failed when the latch tripped. */
+  public readonly failedAttempts: number
+
+  /**
+   * @param cause - The storage error from the last failed pending-tier flush.
+   * @param failedAttempts - Consecutive failed flush attempts at latch time.
+   */
+  constructor(cause: Error, failedAttempts: number) {
+    super(
+      `Write refused: single-op generation history could not be made durable ` +
+        `after ${failedAttempts} consecutive flush attempts (${cause.message}). ` +
+        `Live data is intact, but buffered history is not yet persisted — the ` +
+        `store refuses further writes rather than silently accumulate ` +
+        `un-durable history. Resolve the underlying storage fault; the store ` +
+        `self-heals and resumes writes once a flush succeeds. Cause: ${cause.message}`
+    )
+    this.name = 'PendingFlushDurabilityError'
+    this.cause = cause
+    this.failedAttempts = failedAttempts
+  }
+}

src/db/factLog.ts

@@ -0,0 +1,721 @@
+/**
+ * @module db/factLog
+ * @description The generation FACT LOG — an append-only, CRC-framed record of
+ * every committed generation as an AFTER-IMAGE "fact": what each touched
+ * entity/relationship BECAME (or a body-less tombstone when it was removed).
+ * This is the dual-write half of the log-canonical transition: today the
+ * before-image history + canonical tree remain authoritative; the fact log is
+ * appended at the same commit points and reconciled to committed truth at
+ * open, so consumers (index heals, replays, scans) can read one sequential,
+ * self-verifying stream instead of walking the entity tree.
+ *
+ * ## Wire format (frozen; additive-only within a major)
+ *
+ * Fact (msgpack, POSITIONAL array — the segment header's formatVersion
+ * governs the schema):
+ *
+ *   fact := [ generation:u64, timestamp:u64, ops, meta|nil, blobHashes|nil ]
+ *   op   := [ kind:u8 (0=noun, 1=verb), id:bin16 (raw uuid bytes),
+ *             record:[metaLeg, vecLeg] | nil ]        // nil = TOMBSTONE
+ *
+ * Segment file (`_generations/facts/seg-<firstGeneration, zero-padded 20>.bfl`):
+ *
+ *   header := magic "BFACTS\0\0" (8B) | formatVersion:u32 LE |
+ *             firstGeneration:u64 LE | reserved 12B (ZEROED, verified)
+ *   frame  := length:u32 LE | crc32c:u32 LE (of payload) | payload
+ *
+ * A fact is never split across segments; a torn tail (length overrun or CRC
+ * mismatch) terminates that segment's scan — everything before it is intact.
+ * Zero-padded names make lexicographic order == generation order.
+ *
+ * ## Invariant
+ *
+ * After {@link FactLog.open}, the log contains EXACTLY the committed prefix:
+ * facts are appended BEFORE the commit point (inside the same durability
+ * window), so a crash can only leave the log AHEAD of committed truth — open
+ * truncates any fact beyond the committed generation. Absent generation =
+ * never committed; present = committed. A scan can never see an uncommitted
+ * fact.
+ *
+ * The manifest (`_generations/facts/manifest.json`, JSON — forensics stay
+ * terminal-readable) is the single source of truth for the segment SET;
+ * rotation flips it atomically (write-new → fsync → rename) BEFORE the new
+ * tail's first byte exists, so no segment file is ever unaccounted for.
+ */
+import { encode as defaultEncode, decode as defaultDecode } from '@msgpack/msgpack'
+import { crc32c } from '../utils/crc32c.js'
+import { prodLog } from '../utils/logger.js'
+
+// Swappable msgpack implementation — defaults to the JS codec; a native
+// provider (registered via the plugin registry's 'msgpack' key) may replace
+// it. Byte-compatibility is the contract (positional arrays, bin16 ids).
+let msgpackEncode: (value: unknown) => Uint8Array = defaultEncode
+let msgpackDecode: (bytes: Uint8Array) => unknown = defaultDecode
+
+/** Replace the msgpack encode/decode implementation at runtime. */
+export function setFactCodec(impl: {
+  encode: (value: unknown) => Uint8Array
+  decode: (bytes: Uint8Array) => unknown
+}): void {
+  msgpackEncode = impl.encode
+  msgpackDecode = impl.decode
+}
+
+/** Storage-root-relative home of the fact log. */
+export const FACTS_PREFIX = '_generations/facts'
+/** The facts manifest path (JSON). */
+export const FACTS_MANIFEST_PATH = `${FACTS_PREFIX}/manifest.json`
+/** Current segment format version (header field; additive-only within a major). */
+export const FACTS_FORMAT_VERSION = 1
+/** Rotation threshold: seal the tail segment once it exceeds this many bytes. */
+const SEGMENT_ROTATE_BYTES = 8 * 1024 * 1024
+/** Segment header: magic(8) + formatVersion(4) + firstGeneration(8) + reserved(12). */
+const HEADER_BYTES = 32
+const MAGIC = new Uint8Array([0x42, 0x46, 0x41, 0x43, 0x54, 0x53, 0x00, 0x00]) // "BFACTS\0\0"
+/** Frame prefix: length(4) + crc32c(4). */
+const FRAME_PREFIX_BYTES = 8
+
+/** One write inside a fact: what the id became (or a tombstone). */
+export interface FactOp {
+  kind: 'noun' | 'verb'
+  id: string
+  /** The AFTER-IMAGE legs, or `null` for a tombstone (the id was removed). */
+  record: { metadata: unknown | null; vector: unknown | null } | null
+}
+
+/** One committed generation, as scanned back out of the log. */
+export interface CommitFact {
+  generation: number
+  timestamp: number
+  ops: FactOp[]
+  meta?: Record<string, unknown>
+  blobHashes?: string[]
+}
+
+/** The telemetry a scan batch carries (frozen shape). */
+export interface FactScanBatch {
+  facts: CommitFact[]
+  firstGeneration: number
+  lastGeneration: number
+  factCount: number
+  byteSize: number
+  segmentId: string
+}
+
+/**
+ * Liveness bound on a scan's FIRST batch (Stage-2 co-freeze, D1 contract):
+ * `batches()` must yield its first batch — or fail loudly — within this many
+ * ms of the first pull. A backlogged or damaged store may be SLOW, but it may
+ * never be SILENT: a consumer awaiting the first batch is otherwise
+ * indistinguishable from a wedge (the exact failure shape a production heal
+ * hit against a generations-backlogged brain).
+ */
+export const SCANFACTS_FIRST_BATCH_MS = 10_000
+
+/** The telemetry a scan OPEN returns (frozen shape). */
+export interface FactScanHandle {
+  headGeneration: number
+  segmentCount: number
+  approxFactCount: number
+  /**
+   * Ordered batches; a detected gap aborts LOUDLY, never a silent skip.
+   * Liveness contract: the FIRST batch resolves or rejects within
+   * {@link SCANFACTS_FIRST_BATCH_MS} of the first pull — never a silent hang.
+   */
+  batches: () => AsyncGenerator<FactScanBatch>
+  /** Close telemetry — the invariant cross-check, valid after iteration ends. */
+  summary: () => { factsYielded: number; segmentsRead: number }
+}
+
+/** Manifest entry for a sealed segment. */
+interface SegmentEntry {
+  file: string
+  firstGeneration: number
+  lastGeneration: number
+  facts: number
+  bytes: number
+}
+
+/** The facts manifest (JSON on disk). */
+interface FactsManifest {
+  formatVersion: number
+  segments: SegmentEntry[]
+  /** The append target. Its true content is established by scanning (crash tolerance). */
+  tailSegment: string | null
+  updatedAt: string
+}
+
+/** The narrow byte-level storage surface the fact log rides. */
+export interface FactLogStorage {
+  appendRawBytes(path: string, bytes: Uint8Array): Promise<void>
+  readRawBytes(path: string): Promise<Uint8Array | null>
+  writeRawBytes(path: string, bytes: Uint8Array): Promise<void>
+  rawByteSize(path: string): Promise<number | null>
+  readRawObject(path: string): Promise<any | null>
+  writeRawObject(path: string, data: any): Promise<void>
+  syncRawObjects(paths: string[]): Promise<void>
+  deleteRawObject(path: string): Promise<void>
+}
+
+/** True when the storage adapter exposes every primitive the fact log needs. */
+export function storageSupportsFactLog(storage: unknown): storage is FactLogStorage {
+  const s = storage as Record<string, unknown>
+  return (
+    typeof s.appendRawBytes === 'function' &&
+    typeof s.readRawBytes === 'function' &&
+    typeof s.writeRawBytes === 'function' &&
+    typeof s.rawByteSize === 'function'
+  )
+}
+
+/** uuid string → 16 raw bytes (bin16 on the wire). */
+function uuidToBytes(id: string): Uint8Array {
+  const hex = id.replace(/-/g, '')
+  if (hex.length !== 32) {
+    // Non-uuid ids (legacy/natural keys) ride as UTF-8 with a length prefix
+    // marker impossible for uuids: we refuse instead — the write API has
+    // guaranteed uuid ids since 8.0, so anything else is a corruption signal.
+    throw new Error(`fact log: id is not a uuid: ${id}`)
+  }
+  const bytes = new Uint8Array(16)
+  for (let i = 0; i < 16; i++) {
+    bytes[i] = parseInt(hex.slice(i * 2, i * 2 + 2), 16)
+  }
+  return bytes
+}
+
+/** 16 raw bytes → canonical lowercase uuid string. */
+function bytesToUuid(bytes: Uint8Array): string {
+  let hex = ''
+  for (let i = 0; i < 16; i++) hex += bytes[i].toString(16).padStart(2, '0')
+  return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`
+}
+
+/** Zero-padded segment filename: lexicographic order == generation order. */
+function segmentFileName(firstGeneration: number): string {
+  return `seg-${String(firstGeneration).padStart(20, '0')}.bfl`
+}
+
+/** Build a segment header. Reserved bytes are ZEROED (and verified on open). */
+function buildHeader(firstGeneration: number): Uint8Array {
+  const header = new Uint8Array(HEADER_BYTES)
+  header.set(MAGIC, 0)
+  const view = new DataView(header.buffer)
+  view.setUint32(8, FACTS_FORMAT_VERSION, true)
+  view.setBigUint64(12, BigInt(firstGeneration), true)
+  // bytes 20..31 stay zero (reserved)
+  return header
+}
+
+/** Encode one fact into a framed record (length + crc32c + msgpack payload). */
+function encodeFrame(fact: CommitFact): Uint8Array {
+  const payload = msgpackEncode([
+    fact.generation,
+    fact.timestamp,
+    fact.ops.map((op) => [
+      op.kind === 'noun' ? 0 : 1,
+      uuidToBytes(op.id),
+      op.record === null ? null : [op.record.metadata, op.record.vector]
+    ]),
+    fact.meta ?? null,
+    fact.blobHashes && fact.blobHashes.length > 0 ? fact.blobHashes : null
+  ])
+  const frame = new Uint8Array(FRAME_PREFIX_BYTES + payload.length)
+  const view = new DataView(frame.buffer)
+  view.setUint32(0, payload.length, true)
+  view.setUint32(4, crc32c(payload), true)
+  frame.set(payload, FRAME_PREFIX_BYTES)
+  return frame
+}
+
+/** Decode one msgpack payload back into a CommitFact. */
+function decodeFact(payload: Uint8Array): CommitFact {
+  const raw = msgpackDecode(payload) as unknown[]
+  const [generation, timestamp, ops, meta, blobHashes] = raw as [
+    number,
+    number,
+    Array<[number, Uint8Array, [unknown, unknown] | null]>,
+    Record<string, unknown> | null,
+    string[] | null
+  ]
+  return {
+    generation: Number(generation),
+    timestamp: Number(timestamp),
+    ops: ops.map(([kind, idBytes, record]) => ({
+      kind: kind === 0 ? ('noun' as const) : ('verb' as const),
+      id: bytesToUuid(idBytes),
+      record: record === null ? null : { metadata: record[0] ?? null, vector: record[1] ?? null }
+    })),
+    ...(meta ? { meta } : {}),
+    ...(blobHashes && blobHashes.length > 0 ? { blobHashes } : {})
+  }
+}
+
+/**
+ * Parse a segment's bytes: verify the header, then walk frames until the end
+ * or a torn tail (length overrun / CRC mismatch), which terminates the walk —
+ * everything before it is intact. Returns the decoded facts plus the byte
+ * length of the VALID prefix (header + intact frames), which reconciliation
+ * uses to cut a torn tail without re-encoding.
+ */
+function parseSegment(
+  file: string,
+  bytes: Uint8Array
+): { facts: CommitFact[]; validBytes: number } {
+  if (bytes.length < HEADER_BYTES) {
+    prodLog.warn(`[FactLog] segment ${file} shorter than its header — treating as empty`)
+    return { facts: [], validBytes: 0 }
+  }
+  for (let i = 0; i < MAGIC.length; i++) {
+    if (bytes[i] !== MAGIC[i]) {
+      throw new Error(`fact log: segment ${file} has a bad magic — not a fact segment`)
+    }
+  }
+  const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength)
+  const version = view.getUint32(8, true)
+  if (version !== FACTS_FORMAT_VERSION) {
+    throw new Error(
+      `fact log: segment ${file} has formatVersion ${version}; this build reads ${FACTS_FORMAT_VERSION}`
+    )
+  }
+  for (let i = 20; i < HEADER_BYTES; i++) {
+    if (bytes[i] !== 0) {
+      // Non-zero reserved bytes = a future format this build cannot verify.
+      throw new Error(`fact log: segment ${file} has non-zero reserved header bytes — unverifiable`)
+    }
+  }
+
+  const facts: CommitFact[] = []
+  let offset = HEADER_BYTES
+  while (offset + FRAME_PREFIX_BYTES <= bytes.length) {
+    const length = view.getUint32(offset, true)
+    const expectedCrc = view.getUint32(offset + 4, true)
+    const start = offset + FRAME_PREFIX_BYTES
+    const end = start + length
+    if (end > bytes.length) break // torn tail: frame length overruns the file
+    const payload = bytes.subarray(start, end)
+    if (crc32c(payload) !== expectedCrc) break // torn tail: payload CRC mismatch
+    facts.push(decodeFact(payload))
+    offset = end
+  }
+  return { facts, validBytes: offset }
+}
+
+/**
+ * The generation fact log. One instance per open store; every method assumes
+ * the single-writer discipline the generation store already enforces (calls
+ * arrive under its commit mutex).
+ */
+export class FactLog {
+  private readonly storage: FactLogStorage
+  /** Rotation threshold (bytes); tests may lower it to exercise rotation. */
+  private readonly rotateBytes: number
+  private manifest: FactsManifest = {
+    formatVersion: FACTS_FORMAT_VERSION,
+    segments: [],
+    tailSegment: null,
+    updatedAt: new Date(0).toISOString()
+  }
+  /** Decoded facts of the TAIL segment (bounded by the rotation threshold). */
+  private tailFacts: CommitFact[] = []
+  /** Byte size of the tail segment file (valid prefix). */
+  private tailBytes = 0
+  /** Highest generation in the log (0 = empty). */
+  private head = 0
+  /** Segment paths appended since the last sync (the fsync batch). */
+  private readonly dirtySegments = new Set<string>()
+
+  constructor(storage: FactLogStorage, options?: { rotateBytes?: number }) {
+    this.storage = storage
+    this.rotateBytes = options?.rotateBytes ?? SEGMENT_ROTATE_BYTES
+  }
+
+  /** The highest committed generation the log holds (0 = empty). */
+  headGeneration(): number {
+    return this.head
+  }
+
+  /**
+   * Open the log and reconcile it to committed truth: read the manifest,
+   * establish the tail's intact content (torn-tail scan), then TRUNCATE any
+   * fact with `generation > committedGeneration` — those never committed (a
+   * crash between fact-append and the commit point). After open, the log is
+   * exactly the committed prefix.
+   */
+  async open(committedGeneration: number): Promise<void> {
+    const stored = (await this.storage.readRawObject(FACTS_MANIFEST_PATH)) as FactsManifest | null
+    if (stored && typeof stored === 'object' && Array.isArray(stored.segments)) {
+      if (stored.formatVersion !== FACTS_FORMAT_VERSION) {
+        throw new Error(
+          `fact log: manifest formatVersion ${stored.formatVersion}; this build reads ${FACTS_FORMAT_VERSION}`
+        )
+      }
+      this.manifest = stored
+    }
+
+    // Drop sealed segments that sit ENTIRELY beyond committed truth (a crash
+    // right after a rotation whose facts never committed), newest first.
+    while (this.manifest.segments.length > 0) {
+      const last = this.manifest.segments[this.manifest.segments.length - 1]
+      if (last.firstGeneration > committedGeneration) {
+        prodLog.warn(
+          `[FactLog] dropping sealed segment ${last.file} (generations ${last.firstGeneration}..` +
+            `${last.lastGeneration} never committed)`
+        )
+        await this.storage.deleteRawObject(`${FACTS_PREFIX}/${last.file}`)
+        this.manifest.segments.pop()
+        await this.persistManifest()
+      } else if (last.lastGeneration > committedGeneration) {
+        // A sealed segment STRADDLING committed truth: cut it back.
+        await this.truncateSegmentTo(last.file, committedGeneration)
+        const cut = await this.reloadSegmentEntry(last.file)
+        this.manifest.segments[this.manifest.segments.length - 1] = cut
+        await this.persistManifest()
+        break
+      } else {
+        break
+      }
+    }
+
+    // Establish the tail: scan its intact prefix, then truncate beyond
+    // committed truth (the common crash shape: buffered single-op facts whose
+    // counter never went durable).
+    if (this.manifest.tailSegment) {
+      const tailPath = `${FACTS_PREFIX}/${this.manifest.tailSegment}`
+      const bytes = await this.storage.readRawBytes(tailPath)
+      if (bytes === null) {
+        // Manifest named a tail whose first byte never landed — an empty tail.
+        this.tailFacts = []
+        this.tailBytes = 0
+      } else {
+        const { facts, validBytes } = parseSegment(this.manifest.tailSegment, bytes)
+        const kept = facts.filter((f) => f.generation <= committedGeneration)
+        if (kept.length !== facts.length || validBytes !== bytes.length) {
+          const dropped = facts.length - kept.length
+          if (dropped > 0) {
+            prodLog.warn(
+              `[FactLog] truncating ${dropped} uncommitted fact(s) beyond generation ` +
+                `${committedGeneration} from the tail (never committed)`
+            )
+          }
+          await this.rewriteTail(kept)
+        } else {
+          this.tailFacts = facts
+          this.tailBytes = validBytes
+        }
+      }
+    }
+
+    this.head = this.computeHead()
+  }
+
+  /**
+   * Append one committed generation's fact. NOT durable until {@link sync} —
+   * the caller batches durability at its commit barrier (transact syncs in
+   * the same call; Model-B group-commit syncs at flush).
+   */
+  async append(fact: CommitFact): Promise<void> {
+    if (fact.generation <= this.head) {
+      throw new Error(
+        `fact log: non-monotonic append (generation ${fact.generation} ≤ head ${this.head})`
+      )
+    }
+    if (this.manifest.tailSegment === null) {
+      await this.startTail(fact.generation)
+    } else if (this.tailBytes >= this.rotateBytes) {
+      await this.rotate(fact.generation)
+    }
+    const frame = encodeFrame(fact)
+    const tailPath = `${FACTS_PREFIX}/${this.manifest.tailSegment}`
+    await this.storage.appendRawBytes(tailPath, frame)
+    this.tailFacts.push(fact)
+    this.tailBytes += frame.length
+    this.head = fact.generation
+    this.dirtySegments.add(tailPath)
+  }
+
+  /** Fsync every segment appended since the last sync. */
+  async sync(): Promise<void> {
+    if (this.dirtySegments.size === 0) return
+    const paths = [...this.dirtySegments]
+    this.dirtySegments.clear()
+    await this.storage.syncRawObjects(paths)
+  }
+
+  /**
+   * Open a scan over committed facts. The scan runs against a MANIFEST
+   * SNAPSHOT (sealed segments + the tail's decoded facts at open) — exactly-
+   * once per fact, inclusive bounds, stable under concurrent appends. Gaps
+   * abort LOUDLY: a missing generation inside a segment's declared range is
+   * corruption, never silently skipped.
+   */
+  scanFacts(options?: {
+    fromGeneration?: number
+    toGeneration?: number
+    kinds?: Array<'noun' | 'verb'>
+    batchSize?: number
+    /** Test override for the first-batch liveness bound (default {@link SCANFACTS_FIRST_BATCH_MS}). */
+    firstBatchTimeoutMs?: number
+  }): FactScanHandle {
+    const from = options?.fromGeneration ?? 1
+    const to = options?.toGeneration ?? this.head
+    const kinds = options?.kinds
+    const batchSize = Math.max(1, options?.batchSize ?? 256)
+
+    // Snapshot: the segment list + tail content as of NOW.
+    const segments = this.manifest.segments.filter(
+      (s) => s.lastGeneration >= from && s.firstGeneration <= to
+    )
+    const tailSnapshot = this.tailFacts.filter((f) => f.generation >= from && f.generation <= to)
+    const tailId = this.manifest.tailSegment ?? 'tail'
+    const approxFactCount =
+      segments.reduce((sum, s) => sum + s.facts, 0) + tailSnapshot.length
+
+    let factsYielded = 0
+    let segmentsRead = 0
+    const storage = this.storage
+
+    async function* batches(this: void): AsyncGenerator<FactScanBatch> {
+      let expectedNext = 0 // gap detection: generations are monotonic, not necessarily dense
+      const emit = (facts: CommitFact[], segmentId: string, byteSize: number): FactScanBatch => ({
+        facts,
+        firstGeneration: facts[0].generation,
+        lastGeneration: facts[facts.length - 1].generation,
+        factCount: facts.length,
+        byteSize,
+        segmentId
+      })
+      const filterOps = (fact: CommitFact): CommitFact =>
+        kinds
+          ? { ...fact, ops: fact.ops.filter((op) => kinds.includes(op.kind)) }
+          : fact
+
+      for (const entry of segments) {
+        const bytes = await storage.readRawBytes(`${FACTS_PREFIX}/${entry.file}`)
+        if (bytes === null) {
+          throw new Error(
+            `fact log: sealed segment ${entry.file} is MISSING — the log is damaged; aborting scan`
+          )
+        }
+        const { facts } = parseSegment(entry.file, bytes)
+        segmentsRead++
+        const inRange = facts.filter((f) => f.generation >= from && f.generation <= to)
+        for (const f of inRange) {
+          if (f.generation <= expectedNext - 1) {
+            throw new Error(`fact log: out-of-order fact ${f.generation} in ${entry.file} — aborting scan`)
+          }
+          expectedNext = f.generation + 1
+        }
+        for (let i = 0; i < inRange.length; i += batchSize) {
+          const slice = inRange.slice(i, i + batchSize).map(filterOps)
+          if (slice.length === 0) continue
+          factsYielded += slice.length
+          yield emit(slice, entry.file, slice.reduce((n, f) => n + encodeFrame(f).length, 0))
+        }
+      }
+
+      if (tailSnapshot.length > 0) {
+        segmentsRead++
+        for (const f of tailSnapshot) {
+          if (f.generation <= expectedNext - 1) {
+            throw new Error(`fact log: out-of-order fact ${f.generation} in the tail — aborting scan`)
+          }
+          expectedNext = f.generation + 1
+        }
+        for (let i = 0; i < tailSnapshot.length; i += batchSize) {
+          const slice = tailSnapshot.slice(i, i + batchSize).map(filterOps)
+          factsYielded += slice.length
+          yield emit(slice, tailId, slice.reduce((n, f) => n + encodeFrame(f).length, 0))
+        }
+      }
+    }
+
+    // Liveness wrapper: the FIRST pull races the contract deadline. Only the
+    // first — the bound is time-to-first-batch (proof the producer is alive),
+    // not per-batch pacing; and it runs only while a pull is actually pending,
+    // so consumer think-time between pulls never counts against the producer.
+    const firstBatchTimeoutMs = options?.firstBatchTimeoutMs ?? SCANFACTS_FIRST_BATCH_MS
+    async function* batchesWithLiveness(this: void): AsyncGenerator<FactScanBatch> {
+      const inner = batches()
+      let timer: NodeJS.Timeout | undefined
+      try {
+        const deadline = new Promise<never>((_, reject) => {
+          timer = setTimeout(
+            () =>
+              reject(
+                new Error(
+                  `fact log: scanFacts produced no first batch within ${firstBatchTimeoutMs}ms ` +
+                    `(liveness contract) — the store is wedged or unreadably slow; aborting scan LOUDLY ` +
+                    `instead of hanging the consumer.`
+                )
+              ),
+            firstBatchTimeoutMs
+          )
+          timer.unref?.()
+        })
+        const first = await Promise.race([inner.next(), deadline])
+        if (first.done) return
+        yield first.value
+      } finally {
+        clearTimeout(timer)
+      }
+      yield* inner
+    }
+
+    return {
+      headGeneration: this.head,
+      segmentCount: segments.length + (tailSnapshot.length > 0 ? 1 : 0),
+      approxFactCount,
+      batches: batchesWithLiveness,
+      summary: () => ({ factsYielded, segmentsRead })
+    }
+  }
+
+  /**
+   * The mmap fast path (capability handoff): the immutable sealed segment
+   * files covering `fromGeneration`, in order. The TAIL is deliberately NOT
+   * included — it is append-mutable; consumers read it via {@link scanFacts}.
+   */
+  segmentPaths(options?: { fromGeneration?: number }): string[] {
+    const from = options?.fromGeneration ?? 1
+    return this.manifest.segments
+      .filter((s) => s.lastGeneration >= from)
+      .map((s) => `${FACTS_PREFIX}/${s.file}`)
+  }
+
+  /**
+   * Drop every fact with `generation > keepThrough` — the in-session abort
+   * compensation: a transact appends its fact BEFORE the commit point, so a
+   * real (non-crash) abort after the append must take the fact back out. The
+   * dropped facts can only live in the TAIL (they were just appended); the
+   * rewrite is atomic and bounded by the rotation threshold.
+   */
+  async dropAbove(keepThrough: number): Promise<void> {
+    if (this.head <= keepThrough) return
+    const kept = this.tailFacts.filter((f) => f.generation <= keepThrough)
+    if (kept.length === this.tailFacts.length) {
+      throw new Error(
+        `fact log: dropAbove(${keepThrough}) found no droppable facts in the tail ` +
+          `(head ${this.head}) — the fact to drop was already sealed; the log needs reopen`
+      )
+    }
+    await this.rewriteTail(kept)
+    this.head = this.computeHead()
+  }
+
+  // -- internals -------------------------------------------------------------
+
+  private computeHead(): number {
+    if (this.tailFacts.length > 0) return this.tailFacts[this.tailFacts.length - 1].generation
+    const sealed = this.manifest.segments
+    if (sealed.length > 0) return sealed[sealed.length - 1].lastGeneration
+    return 0
+  }
+
+  /** Create the very first tail segment (manifest-first, then header bytes). */
+  private async startTail(firstGeneration: number): Promise<void> {
+    const file = segmentFileName(firstGeneration)
+    this.manifest.tailSegment = file
+    await this.persistManifest()
+    await this.storage.appendRawBytes(`${FACTS_PREFIX}/${file}`, buildHeader(firstGeneration))
+    this.tailFacts = []
+    this.tailBytes = HEADER_BYTES
+  }
+
+  /**
+   * Seal the tail into the manifest and start a new one. Manifest-first: the
+   * flip both seals the old tail AND names the new one atomically, so no
+   * segment file ever exists unaccounted for.
+   */
+  private async rotate(nextGeneration: number): Promise<void> {
+    const sealedFile = this.manifest.tailSegment
+    if (!sealedFile) return
+    // Seal what the tail actually holds.
+    await this.sync() // sealed segments are always fully durable
+    const entry: SegmentEntry = {
+      file: sealedFile,
+      firstGeneration: this.tailFacts[0]?.generation ?? nextGeneration,
+      lastGeneration: this.tailFacts[this.tailFacts.length - 1]?.generation ?? nextGeneration - 1,
+      facts: this.tailFacts.length,
+      bytes: this.tailBytes
+    }
+    const newFile = segmentFileName(nextGeneration)
+    this.manifest.segments.push(entry)
+    this.manifest.tailSegment = newFile
+    await this.persistManifest()
+    await this.storage.appendRawBytes(`${FACTS_PREFIX}/${newFile}`, buildHeader(nextGeneration))
+    this.tailFacts = []
+    this.tailBytes = HEADER_BYTES
+  }
+
+  /** Atomically persist the manifest (write-new → fsync → rename downstream). */
+  private async persistManifest(): Promise<void> {
+    this.manifest.updatedAt = new Date().toISOString()
+    await this.storage.writeRawObject(FACTS_MANIFEST_PATH, this.manifest)
+    await this.storage.syncRawObjects([FACTS_MANIFEST_PATH])
+  }
+
+  /** Rewrite the tail segment to hold exactly `facts` (atomic replace). */
+  private async rewriteTail(facts: CommitFact[]): Promise<void> {
+    const file = this.manifest.tailSegment
+    if (!file) return
+    const first = facts[0]?.generation ?? this.segmentFirstGenerationFromName(file)
+    const parts: Uint8Array[] = [buildHeader(first)]
+    for (const f of facts) parts.push(encodeFrame(f))
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const merged = new Uint8Array(total)
+    let offset = 0
+    for (const p of parts) {
+      merged.set(p, offset)
+      offset += p.length
+    }
+    await this.storage.writeRawBytes(`${FACTS_PREFIX}/${file}`, merged)
+    this.tailFacts = facts
+    this.tailBytes = total
+  }
+
+  /** Cut a SEALED segment back to `committedGeneration` (atomic replace). */
+  private async truncateSegmentTo(file: string, committedGeneration: number): Promise<void> {
+    const path = `${FACTS_PREFIX}/${file}`
+    const bytes = await this.storage.readRawBytes(path)
+    if (bytes === null) return
+    const { facts } = parseSegment(file, bytes)
+    const kept = facts.filter((f) => f.generation <= committedGeneration)
+    prodLog.warn(
+      `[FactLog] truncating sealed segment ${file} to generation ${committedGeneration} ` +
+        `(${facts.length - kept.length} uncommitted fact(s) dropped)`
+    )
+    const first = kept[0]?.generation ?? this.segmentFirstGenerationFromName(file)
+    const parts: Uint8Array[] = [buildHeader(first)]
+    for (const f of kept) parts.push(encodeFrame(f))
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const merged = new Uint8Array(total)
+    let offset = 0
+    for (const p of parts) {
+      merged.set(p, offset)
+      offset += p.length
+    }
+    await this.storage.writeRawBytes(path, merged)
+  }
+
+  /** Re-derive a sealed segment's manifest entry from its actual bytes. */
+  private async reloadSegmentEntry(file: string): Promise<SegmentEntry> {
+    const bytes = await this.storage.readRawBytes(`${FACTS_PREFIX}/${file}`)
+    const { facts, validBytes } = bytes
+      ? parseSegment(file, bytes)
+      : { facts: [] as CommitFact[], validBytes: 0 }
+    return {
+      file,
+      firstGeneration: facts[0]?.generation ?? this.segmentFirstGenerationFromName(file),
+      lastGeneration: facts[facts.length - 1]?.generation ?? 0,
+      facts: facts.length,
+      bytes: validBytes
+    }
+  }
+
+  /** Parse the zero-padded firstGeneration back out of a segment filename. */
+  private segmentFirstGenerationFromName(file: string): number {
+    const match = /^seg-(\d{20})\.bfl$/.exec(file)
+    return match ? Number(match[1]) : 0
+  }
+}

src/db/familyStamp.ts

@@ -0,0 +1,152 @@
+/**
+ * @module db/familyStamp
+ * @description The generalized FAMILY STAMP — one JSON shape that declares,
+ * for any derived projection, WHICH source state it reflects and HOW to verify
+ * it is whole. The entity tree (canonical current-state files) carries the
+ * first brainy-side stamp; native index families carry the same shape. One
+ * verifier reads both member modes:
+ *
+ * - `enumerated` — bounded families: exact byte size per member file,
+ *   verified at open.
+ * - `rollup`     — unbounded families (the entity tree: millions of files):
+ *   the verified surface is a small set of rollup invariants (entity/
+ *   relationship counts) plus `sourceGeneration`.
+ *
+ * `sourceGeneration` is the generation of the source-of-truth log this
+ * projection reflects — open-time coherence becomes a COMPARISON (stamp vs
+ * log head), not a walk:
+ *
+ * - equal + invariants hold → coherent, serve.
+ * - behind → the projection missed the tail (crash between commit and stamp);
+ *   for the Stage-1 tree this is benign by construction (the tree is written
+ *   BY the commit), so the stamp refreshes; a DERIVED projection would replay
+ *   the gap instead.
+ * - invariants FAIL at equal generation → genuine incoherence: loud, and the
+ *   repair ritual (`repairIndex()`, whose recount rebuilds the rollups from a
+ *   canonical walk) heals it.
+ *
+ * Stamps are JSON on purpose — every incident gets debugged by reading a
+ * stamp in a terminal.
+ */
+
+/** Storage-root-relative directory holding family stamps. */
+export const FAMILY_STAMPS_PREFIX = '_system/family-stamps'
+
+/** The entity tree's stamp path. */
+export const ENTITY_TREE_STAMP_PATH = `${FAMILY_STAMPS_PREFIX}/entity-tree.json`
+
+/** One enumerated member: a file and its exact expected byte size. */
+export interface EnumeratedMember {
+  path: string
+  bytes: number
+}
+
+/**
+ * The stamp's verified surface, in one of the two member modes. Rollup
+ * invariant values may be numbers (counts, byte sizes) or strings (content
+ * fingerprints, e.g. a per-tree SHA-256) — the verifier compares by strict
+ * equality either way, so a type mismatch reads as incoherence, never a pass.
+ */
+export type StampMembers =
+  | { mode: 'enumerated'; files: EnumeratedMember[] }
+  | { mode: 'rollup'; invariants: Record<string, number | string> }
+
+/** The generalized family stamp (one shape, one verifier, both engines). */
+export interface FamilyStamp {
+  /** Which projection this stamps (e.g. `'entity-tree'`). */
+  family: string
+  /** Monotonic per-family stamp generation — bumps on every committed stamp. */
+  generation: number
+  /** ISO timestamp of the stamp write. */
+  committedAt: string
+  /** The source-of-truth generation this projection reflects. */
+  sourceGeneration: number
+  /** The verified surface. */
+  members: StampMembers
+}
+
+/** The verdict of an open-time stamp verification. */
+export type StampVerdict =
+  | { state: 'coherent' }
+  | { state: 'absent' } // legacy store — first stamp writes at the next flush
+  | { state: 'behind'; stampSource: number; head: number }
+  | { state: 'incoherent'; failures: string[] }
+  | { state: 'unverifiable'; reason: string } // a FAULT reading the stamp — never conflated with absence
+
+/** The narrow storage surface stamps ride (JSON objects + fsync). */
+export interface StampStorage {
+  readRawObject(path: string): Promise<any | null>
+  writeRawObject(path: string, data: any): Promise<void>
+  syncRawObjects(paths: string[]): Promise<void>
+}
+
+/** Read a family's stamp; `null` when none was ever written. */
+export async function readFamilyStamp(
+  storage: StampStorage,
+  path: string
+): Promise<FamilyStamp | null> {
+  const stored = (await storage.readRawObject(path)) as FamilyStamp | null
+  if (!stored || typeof stored !== 'object' || typeof stored.family !== 'string') return null
+  return stored
+}
+
+/** Write a family's stamp durably (atomic object write + fsync). */
+export async function writeFamilyStamp(
+  storage: StampStorage,
+  path: string,
+  stamp: Omit<FamilyStamp, 'generation' | 'committedAt'> & { generation?: number }
+): Promise<void> {
+  const prior = await readFamilyStamp(storage, path)
+  const full: FamilyStamp = {
+    ...stamp,
+    generation: (prior?.generation ?? 0) + 1,
+    committedAt: new Date().toISOString()
+  }
+  await storage.writeRawObject(path, full)
+  await storage.syncRawObjects([path])
+}
+
+/**
+ * The ONE verifier, both member modes. `actual` supplies the observed rollup
+ * values (rollup mode) or file sizes (enumerated mode, keyed by path);
+ * `head` is the source-of-truth generation now.
+ */
+export function verifyFamilyStamp(
+  stamp: FamilyStamp | null,
+  head: number,
+  actual: Record<string, number | string>
+): StampVerdict {
+  if (stamp === null) return { state: 'absent' }
+  if (stamp.sourceGeneration > head) {
+    // A stamp AHEAD of the log claims state that never committed — the
+    // projection was stamped against truth that a crash rolled back.
+    return {
+      state: 'incoherent',
+      failures: [`sourceGeneration ${stamp.sourceGeneration} is ahead of the log head ${head}`]
+    }
+  }
+  if (stamp.sourceGeneration < head) {
+    return { state: 'behind', stampSource: stamp.sourceGeneration, head }
+  }
+  const failures: string[] = []
+  if (stamp.members.mode === 'rollup') {
+    for (const [name, expected] of Object.entries(stamp.members.invariants)) {
+      const observed = actual[name]
+      if (observed === undefined) {
+        failures.push(`rollup invariant '${name}' has no observed value`)
+      } else if (observed !== expected) {
+        failures.push(`rollup invariant '${name}': stamped ${expected}, observed ${observed}`)
+      }
+    }
+  } else {
+    for (const member of stamp.members.files) {
+      const observed = actual[member.path]
+      if (observed === undefined) {
+        failures.push(`member '${member.path}' is missing`)
+      } else if (observed !== member.bytes) {
+        failures.push(`member '${member.path}': stamped ${member.bytes} bytes, observed ${observed}`)
+      }
+    }
+  }
+  return failures.length > 0 ? { state: 'incoherent', failures } : { state: 'coherent' }
+}

src/db/generationSegments.ts

@@ -0,0 +1,459 @@
+/**
+ * @module db/generationSegments
+ * @description The generation-segment store — Stage-2 D1+D3+repacking's file
+ * format (co-frozen 2026-07-19; design: the d1-d3-repacking spec).
+ *
+ * Packs CONSECUTIVE cold generations' record-sets (before-images + delta)
+ * into append-once segment files with derived sidecar indexes, so history
+ * scales in SEGMENTS (tens) instead of FILES-PER-GENERATION (hundreds of
+ * thousands), and cold-open reads ONE manifest instead of listing the
+ * backlog. Layout under `_generations/segments/`:
+ *
+ * - `seg-<firstGen, zero-padded 20>.bgs` — magic "BGS1", then one frame per
+ *   generation: `u32 payloadLen | u32 crc32c | msgpack payload`. Payload is
+ *   POSITIONAL: `[generation, timestamp, delta, records[], flags]` with
+ *   records `[kindByte, id, record]`. `flags` reserves encoding evolution
+ *   (bit 0 = compressed payload — v1 always 0; a future writer upgrade,
+ *   never a format break). Sealed segments are IMMUTABLE — the fact log's
+ *   own law, generalized.
+ * - `seg-<firstGen>.idx` — DERIVED sidecar (msgpack): per-generation frame
+ *   offsets (point reads = one ranged read, never a listing) + per-id
+ *   generation postings (per-id chain rebuilds read only what they need).
+ *   Corrupt/missing → rebuilt from its segment in one sequential read,
+ *   loudly.
+ * - `manifest.json` — the segment catalogue + `compactedBelow` (D3's
+ *   horizon marker). Cold-open reads THIS; the packed backlog is never
+ *   listed.
+ *
+ * D3 semantics carried here: bounded-retention reclaim drops WHOLE segments
+ * at boundaries (O(1) per segment, no rewrite); under the archival profile
+ * (`retention: 'all'`) nothing here is ever dropped — folding is the only
+ * transform (re-representation, never deletion).
+ */
+
+import { encode as msgpackEncode, decode as msgpackDecode } from '@msgpack/msgpack'
+import { crc32c } from '../utils/crc32c.js'
+import type { FactLogStorage } from './factLog.js'
+import { prodLog } from '../utils/logger.js'
+
+/** Directory for segment files + manifest, under the generations prefix. */
+export const SEGMENTS_PREFIX = '_generations/segments'
+
+/** Target sealed-segment size (co-freeze proposal; tunable on evidence). */
+export const SEGMENT_TARGET_BYTES = 64 * 1024 * 1024
+
+const MAGIC = new TextEncoder().encode('BGS1')
+const FRAME_PREFIX_BYTES = 8 // u32 payloadLen + u32 crc32c
+const MANIFEST_PATH = `${SEGMENTS_PREFIX}/manifest.json`
+
+/** One generation's fold input — exactly what the live tier holds for it. */
+export interface FoldGeneration {
+  generation: number
+  timestamp: number
+  /** The tx.json delta object, carried verbatim. */
+  delta: unknown
+  /** The before-image record-set (empty for record-less generations). */
+  records: Array<{ kind: 'noun' | 'verb'; id: string; record: unknown }>
+}
+
+/** Manifest entry for one sealed segment. */
+export interface SegmentMeta {
+  file: string
+  firstGeneration: number
+  lastGeneration: number
+  frames: number
+  bytes: number
+  /** crc32c of the full segment byte stream — the digest chain's link. */
+  checksum: number
+}
+
+interface SegmentManifest {
+  version: 1
+  compactedBelow: number
+  segments: SegmentMeta[]
+}
+
+interface SidecarIndex {
+  version: 1
+  /** [generation, frameOffset, frameLen] ascending by generation. */
+  generations: Array<[number, number, number]>
+  /** `${kindByte}:${id}` → ascending generations holding a record for it. */
+  ids: Record<string, number[]>
+}
+
+const segmentFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.bgs`
+const sidecarFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.idx`
+
+/**
+ * The generation-segment store. Owns the packed tier ONLY — the live
+ * per-generation tier and the routing between tiers belong to
+ * `GenerationStore`. All mutating entry points here are called under the
+ * generation store's commit mutex.
+ */
+export class GenerationSegmentStore {
+  private readonly storage: FactLogStorage
+  private manifest: SegmentManifest = { version: 1, compactedBelow: 0, segments: [] }
+  /** Sidecar cache — segments are immutable, so entries never invalidate. */
+  private readonly sidecars = new Map<string, SidecarIndex>()
+
+  constructor(storage: FactLogStorage) {
+    this.storage = storage
+  }
+
+  /** Load the manifest (ONE read — never a directory listing). */
+  async open(): Promise<void> {
+    const raw = (await this.storage.readRawObject(MANIFEST_PATH)) as SegmentManifest | null
+    if (raw) {
+      if (raw.version !== 1) {
+        throw new Error(
+          `[GenerationSegments] manifest version ${String(raw.version)} is newer than this ` +
+            `engine understands — refusing to serve partial history. Upgrade the engine.`
+        )
+      }
+      this.manifest = raw
+    }
+  }
+
+  /** The packed tier's catalogue (ascending, immutable snapshot). */
+  segments(): readonly SegmentMeta[] {
+    return this.manifest.segments
+  }
+
+  /** D3's horizon marker: generations below this were reclaimed (bounded profiles only). */
+  compactedBelow(): number {
+    return this.manifest.compactedBelow
+  }
+
+  /** The covering sealed segment for `gen`, or null if it lives outside the packed tier. */
+  private coveringSegment(gen: number): SegmentMeta | null {
+    // Manifest is ascending and ranges never overlap — binary search.
+    const segs = this.manifest.segments
+    let lo = 0
+    let hi = segs.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      const s = segs[mid]
+      if (gen < s.firstGeneration) hi = mid - 1
+      else if (gen > s.lastGeneration) lo = mid + 1
+      else return s
+    }
+    return null
+  }
+
+  /** True when `gen` is packed (readable from this tier). */
+  hasGeneration(gen: number): boolean {
+    return this.coveringSegment(gen) !== null
+  }
+
+  /**
+   * Fold consecutive generations into ONE new sealed segment + sidecar and
+   * append it to the manifest atomically. Caller guarantees: `gens` is
+   * ascending, contiguous with the packed tier (first = last packed + 1 when
+   * segments exist), and already durable in the live tier. Crash between the
+   * segment write and the caller's live-tier delete leaves a DUPLICATE
+   * representation — resolved live-tier-wins by the reader; never a gap.
+   */
+  async fold(gens: FoldGeneration[]): Promise<SegmentMeta> {
+    if (gens.length === 0) {
+      throw new Error('[GenerationSegments] fold() requires at least one generation')
+    }
+    for (let i = 1; i < gens.length; i++) {
+      if (gens[i].generation <= gens[i - 1].generation) {
+        throw new Error('[GenerationSegments] fold() input must be strictly ascending')
+      }
+    }
+    const last = this.manifest.segments[this.manifest.segments.length - 1]
+    if (last && gens[0].generation <= last.lastGeneration) {
+      throw new Error(
+        `[GenerationSegments] fold() overlaps the packed tier: ${gens[0].generation} ≤ ` +
+          `sealed ${last.lastGeneration} — segments are immutable, never rewritten`
+      )
+    }
+
+    const first = gens[0].generation
+    const file = segmentFileName(first)
+    const sidecar: SidecarIndex = { version: 1, generations: [], ids: {} }
+
+    // Encode all frames, tracking offsets for the sidecar.
+    const parts: Uint8Array[] = [MAGIC]
+    let offset = MAGIC.length
+    for (const g of gens) {
+      const payload = msgpackEncode([
+        g.generation,
+        g.timestamp,
+        g.delta,
+        g.records.map((r) => [r.kind === 'noun' ? 0 : 1, r.id, r.record]),
+        0 // flags: v1 = uncompressed
+      ])
+      const frame = new Uint8Array(FRAME_PREFIX_BYTES + payload.length)
+      const view = new DataView(frame.buffer)
+      view.setUint32(0, payload.length, true)
+      view.setUint32(4, crc32c(payload), true)
+      frame.set(payload, FRAME_PREFIX_BYTES)
+      sidecar.generations.push([g.generation, offset, frame.length])
+      for (const r of g.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(sidecar.ids[key] ??= []).push(g.generation)
+      }
+      parts.push(frame)
+      offset += frame.length
+    }
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const bytes = new Uint8Array(total)
+    let at = 0
+    for (const p of parts) {
+      bytes.set(p, at)
+      at += p.length
+    }
+
+    const meta: SegmentMeta = {
+      file,
+      firstGeneration: first,
+      lastGeneration: gens[gens.length - 1].generation,
+      frames: gens.length,
+      bytes: total,
+      checksum: crc32c(bytes)
+    }
+
+    // Durability order: segment + sidecar fsync'd BEFORE the manifest names
+    // them (a crash before the manifest = invisible orphan files, harmless);
+    // manifest last, atomically.
+    const segPath = `${SEGMENTS_PREFIX}/${file}`
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(first)}`
+    await this.storage.writeRawBytes(segPath, bytes)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(sidecar))
+    await this.storage.syncRawObjects([segPath, idxPath])
+    const next: SegmentManifest = {
+      ...this.manifest,
+      segments: [...this.manifest.segments, meta]
+    }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    this.sidecars.set(file, sidecar)
+    return meta
+  }
+
+  /** Load (or rebuild, loudly) a segment's sidecar. */
+  private async sidecarFor(meta: SegmentMeta): Promise<SidecarIndex> {
+    const cached = this.sidecars.get(meta.file)
+    if (cached) return cached
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(meta.firstGeneration)}`
+    const raw = await this.storage.readRawBytes(idxPath)
+    if (raw) {
+      try {
+        const idx = msgpackDecode(raw) as SidecarIndex
+        if (idx.version === 1) {
+          this.sidecars.set(meta.file, idx)
+          return idx
+        }
+      } catch {
+        // fall through to rebuild
+      }
+    }
+    // Sidecars are DERIVED: rebuild from the segment, loudly — never serve
+    // wrong offsets silently.
+    prodLog.warn(
+      `[GenerationSegments] sidecar for ${meta.file} missing or unreadable — rebuilding from the segment`
+    )
+    const rebuilt = await this.rebuildSidecar(meta)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(rebuilt))
+    this.sidecars.set(meta.file, rebuilt)
+    return rebuilt
+  }
+
+  /** One sequential read of the segment → a fresh sidecar. Verifies every frame CRC. */
+  private async rebuildSidecar(meta: SegmentMeta): Promise<SidecarIndex> {
+    const frames = await this.readAllFrames(meta)
+    const idx: SidecarIndex = { version: 1, generations: [], ids: {} }
+    for (const f of frames) {
+      idx.generations.push([f.generation, f.offset, f.frameLen])
+      for (const r of f.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(idx.ids[key] ??= []).push(f.generation)
+      }
+    }
+    return idx
+  }
+
+  private decodeFrame(
+    payload: Uint8Array
+  ): { generation: number; timestamp: number; delta: unknown; records: FoldGeneration['records'] } {
+    const [generation, timestamp, delta, rawRecords] = msgpackDecode(payload) as [
+      number,
+      number,
+      unknown,
+      Array<[number, string, unknown]>,
+      number
+    ]
+    return {
+      generation,
+      timestamp,
+      delta,
+      records: rawRecords.map(([kindByte, id, record]) => ({
+        kind: kindByte === 0 ? ('noun' as const) : ('verb' as const),
+        id,
+        record
+      }))
+    }
+  }
+
+  private async readAllFrames(meta: SegmentMeta): Promise<
+    Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }>
+  > {
+    const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+    if (!bytes) {
+      throw new Error(
+        `[GenerationSegments] sealed segment ${meta.file} is MISSING — packed history is damaged; ` +
+          `refusing to continue silently`
+      )
+    }
+    const out: Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }> = []
+    let at = MAGIC.length
+    const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength)
+    while (at + FRAME_PREFIX_BYTES <= bytes.length) {
+      const payloadLen = view.getUint32(at, true)
+      const crc = view.getUint32(at + 4, true)
+      const payload = bytes.subarray(at + FRAME_PREFIX_BYTES, at + FRAME_PREFIX_BYTES + payloadLen)
+      if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+        throw new Error(
+          `[GenerationSegments] frame CRC mismatch in ${meta.file} at offset ${at} — ` +
+            `packed history is damaged; refusing to serve it`
+        )
+      }
+      out.push({ ...this.decodeFrame(payload), offset: at, frameLen: FRAME_PREFIX_BYTES + payloadLen })
+      at += FRAME_PREFIX_BYTES + payloadLen
+    }
+    return out
+  }
+
+  /** Read one packed generation's frame via its sidecar offset (one ranged read). */
+  private async readFrame(
+    gen: number
+  ): Promise<ReturnType<GenerationSegmentStore['decodeFrame']> | null> {
+    const meta = this.coveringSegment(gen)
+    if (!meta) return null
+    const idx = await this.sidecarFor(meta)
+    // generations ascending → binary search.
+    const gens = idx.generations
+    let lo = 0
+    let hi = gens.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      if (gens[mid][0] < gen) lo = mid + 1
+      else if (gens[mid][0] > gen) hi = mid - 1
+      else {
+        const [, offset, frameLen] = gens[mid]
+        const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+        if (!bytes) {
+          throw new Error(`[GenerationSegments] sealed segment ${meta.file} is MISSING`)
+        }
+        const frame = bytes.subarray(offset, offset + frameLen)
+        const view = new DataView(frame.buffer, frame.byteOffset, frame.byteLength)
+        const payloadLen = view.getUint32(0, true)
+        const crc = view.getUint32(4, true)
+        const payload = frame.subarray(FRAME_PREFIX_BYTES, FRAME_PREFIX_BYTES + payloadLen)
+        if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+          throw new Error(
+            `[GenerationSegments] frame CRC mismatch for generation ${gen} in ${meta.file} — ` +
+              `packed history is damaged; refusing to serve it`
+          )
+        }
+        return this.decodeFrame(payload)
+      }
+    }
+    // In the covering range but not present: the packed tier is dense by
+    // construction (fold packs every generation it is handed, including
+    // record-less ones) — absence inside a sealed range is damage.
+    throw new Error(
+      `[GenerationSegments] generation ${gen} is inside sealed segment ${meta.file}'s declared ` +
+        `range but has no frame — packed history is damaged`
+    )
+  }
+
+  /** The packed tier's delta for `gen` (null = not packed). */
+  async readDelta(gen: number): Promise<{ delta: unknown; timestamp: number } | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? { delta: frame.delta, timestamp: frame.timestamp } : null
+  }
+
+  /** The packed tier's full record-set for `gen` (null = not packed). */
+  async readRecords(gen: number): Promise<FoldGeneration['records'] | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? frame.records : null
+  }
+
+  /** One packed before-image (null = not packed OR no record for the id in that generation). */
+  async readRecord(gen: number, kind: 'noun' | 'verb', id: string): Promise<unknown | null> {
+    const frame = await this.readFrame(gen)
+    if (!frame) return null
+    const hit = frame.records.find((r) => r.kind === kind && r.id === id)
+    return hit ? hit.record : null
+  }
+
+  /**
+   * D3 reclaim: drop WHOLE segments whose lastGeneration < `belowGeneration`
+   * and bump `compactedBelow`. Partial segments are never dropped — the
+   * boundary waits. NEVER called under the archival profile (the caller
+   * enforces retention semantics; this method only executes boundary drops).
+   */
+  async dropSegmentsBelow(belowGeneration: number): Promise<{ dropped: number; compactedBelow: number }> {
+    const keep: SegmentMeta[] = []
+    const drop: SegmentMeta[] = []
+    for (const s of this.manifest.segments) {
+      ;(s.lastGeneration < belowGeneration ? drop : keep).push(s)
+    }
+    if (drop.length === 0) {
+      return { dropped: 0, compactedBelow: this.manifest.compactedBelow }
+    }
+    const compactedBelow = Math.max(
+      this.manifest.compactedBelow,
+      drop[drop.length - 1].lastGeneration + 1
+    )
+    // Manifest first (the drop is authoritative once named), then bytes —
+    // a crash between leaves orphan segment files invisible to the manifest,
+    // harmless and re-collectable.
+    const next: SegmentManifest = { ...this.manifest, compactedBelow, segments: keep }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    for (const s of drop) {
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${s.file}`)
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${sidecarFileName(s.firstGeneration)}`)
+      this.sidecars.delete(s.file)
+    }
+    return { dropped: drop.length, compactedBelow }
+  }
+
+  /**
+   * D8 rider — the packed portion of `generationDigest(g)`: a deterministic
+   * crc32c chain over sealed-segment checksums fully below `g`, plus the
+   * frame CRC of `g`'s own frame when `g` is mid-segment. O(segments), not
+   * O(generations); identical history ⇒ identical digest on any machine.
+   * The live-tier portion is composed by the caller.
+   */
+  async digestThroughPacked(g: number): Promise<number | null> {
+    let digest = 0
+    let covered = false
+    for (const s of this.manifest.segments) {
+      if (s.lastGeneration <= g) {
+        digest = crc32c(new TextEncoder().encode(`${digest}:${s.checksum}`))
+        if (s.lastGeneration === g) covered = true
+      } else if (s.firstGeneration <= g) {
+        // g is mid-segment: chain the partial prefix via g's frame CRC.
+        const frame = await this.readFrame(g)
+        if (frame === null) return null
+        const idx = await this.sidecarFor(s)
+        const upTo = idx.generations.filter(([gen]) => gen <= g)
+        for (const [gen, offset, frameLen] of upTo) {
+          digest = crc32c(new TextEncoder().encode(`${digest}:${gen}:${offset}:${frameLen}`))
+        }
+        covered = true
+        break
+      }
+    }
+    return covered || this.manifest.segments.length > 0 ? digest : null
+  }
+}

src/db/types.ts

@@ -116,6 +116,16 @@ export interface TransactOptions {
    * record is staged.
    */
   ifAtGeneration?: number
+  /**
+   * Budget (ms) for the atomic apply phase. When omitted, the budget SCALES
+   * with the batch: `max(30 000, opCount × 2 000)` — production imports on
+   * network-attached disks measure ~2 s per operation, so a flat 30 s budget
+   * silently capped honest bulk work at ~15 operations. A tripped budget
+   * rolls the whole batch back and throws a retryable
+   * `TransactionTimeoutError` naming the operation it stopped at, the batch
+   * size, and the elapsed/budget times.
+   */
+  timeoutMs?: number
 }
 
 /**
@@ -166,6 +176,15 @@ export interface CompactHistoryOptions {
    * of each surviving generation's serialized record set (`GenerationDelta.bytes`).
    */
   maxBytes?: number
+  /**
+   * Stop reclaiming after this many milliseconds even if caps are still
+   * exceeded (8.9.0). Compaction is maintenance — a bounded pass keeps
+   * `close()` (and any explicit maintenance window) from stalling on a large
+   * backlog; the next pass resumes where this one stopped (reclamation is
+   * oldest-first, so an early stop is always a consistent prefix). Unset =
+   * run to completion.
+   */
+  timeBudgetMs?: number
 }
 
 /**
@@ -183,6 +202,36 @@ export interface CompactHistoryResult {
   horizon: number
 }
 
+/**
+ * @description Result of `brain.historyStats()` — the read-only generational
+ * history footprint, for fleet audits and ops doors. A pool operator runs this
+ * per brain to size retention exposure (how much MVCC history each brain
+ * carries and under which policy) without touching any data.
+ */
+export interface HistoryStats {
+  /** Committed generation record-sets currently on disk. */
+  generations: number
+  /** Total on-disk history bytes across those record-sets. */
+  bytes: number
+  /** Oldest committed generation still on disk (null when history is empty). */
+  oldestGeneration: number | null
+  /** Newest committed generation (null when history is empty). */
+  newestGeneration: number | null
+  /** Commit timestamp (ms) of the oldest on-disk generation. */
+  oldestTimestamp: number | null
+  /** Commit timestamp (ms) of the newest on-disk generation. */
+  newestTimestamp: number | null
+  /** Compaction horizon — generations below it were reclaimed. */
+  horizon: number
+  /** The effective retention mode this brain runs under. */
+  retentionMode: 'all' | 'adaptive' | 'explicit'
+  /**
+   * The adaptive byte budget in force (coordinator-driven or the local
+   * free-memory probe); null under 'all' or explicit caps.
+   */
+  effectiveBudgetBytes: number | null
+}
+
 // ============================================================================
 // Db surfaces
 // ============================================================================
@@ -301,6 +350,16 @@ export interface GenerationDelta {
   nouns: string[]
   /** Relationship ids touched by this generation. */
   verbs: string[]
+  /**
+   * Content-blob hashes referenced by this generation's before-image records
+   * (a MULTISET — one entry per referencing record occurrence), captured at
+   * persist time so compaction can release the exact history references it
+   * reclaims without re-reading the records. Always present (possibly empty)
+   * on deltas written under the temporal-blob contract; absent only on
+   * pre-contract generations, for which compaction falls back to reading the
+   * record-set itself.
+   */
+  blobHashes?: string[]
   /**
    * `true` for a Model-B single-operation generation persisted by the async
    * group-commit flush (`GenerationStore.flushPendingSingleOps`). It marks the
@@ -381,6 +440,24 @@ export interface GenerationStorage {
   /** Durability barrier: fsync the given object paths (no-op in memory). */
   syncRawObjects(paths: string[]): Promise<void>
 
+  /**
+   * OPTIONAL transaction durability barrier. `commitTransaction` calls
+   * {@link beginWriteBarrier} immediately before running the planned operations
+   * and {@link flushWriteBarrier} immediately after — BEFORE the generation
+   * counter and manifest are advanced. An adapter whose canonical writes are
+   * not synchronously durable (the filesystem adapter's tmp+rename lands in the
+   * page cache) MUST implement these so a transaction reported "committed" is
+   * durable before its generation stamp: otherwise a hard kill can leave the
+   * fsync'd counter ahead of the still-buffered entity bytes (phantom progress
+   * for any generation-based consumer). Adapters whose writes are already
+   * durable per-call (cloud object PUT) may leave these undefined — the store
+   * treats them as no-ops. `beginWriteBarrier` also resets any tracking left by
+   * an aborted prior transaction.
+   */
+  beginWriteBarrier?(): void
+  /** @see beginWriteBarrier — fsync every canonical write since begin. */
+  flushWriteBarrier?(): Promise<void>
+
   /** Read an entity's raw stored metadata+vector objects. */
   readNounRaw(id: string): Promise<{ metadata: any | null; vector: any | null }>
   /** Restore an entity's raw stored objects (`null` part ⇒ delete that file). */
@@ -395,6 +472,44 @@ export interface GenerationStorage {
   /** Read all lines of `_system/tx-log.jsonl` (empty array if absent). */
   readTxLogLines(): Promise<string[]>
 
+  /**
+   * OPTIONAL binary raw-byte primitives — the substrate for the generation
+   * fact log's append-only CRC-framed segments. Feature-detected: a storage
+   * layer that omits them hosts no fact log (dual-write is skipped; readers
+   * fall back to canonical enumeration). Paths are used VERBATIM (no
+   * suffixing). Append durability rides `syncRawObjects` at the commit
+   * barrier, exactly like the staged history files.
+   */
+  appendRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+  /** Read a raw binary file whole; absent → null; a real fault throws. */
+  readRawBytes?(path: string): Promise<Uint8Array | null>
+  /** Replace a raw binary file atomically (tmp → fsync → rename). */
+  writeRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+  /** Byte size of a raw binary file, or null when absent. */
+  rawByteSize?(path: string): Promise<number | null>
+
+  /**
+   * OPTIONAL temporal-blob contract (implemented by blob-aware storage; the
+   * generation store treats the hashes as opaque strings). Extract the
+   * content-blob hashes a record-set references — a pure multiset extraction,
+   * no side effects.
+   */
+  extractBlobHashesFromRecords?(records: GenerationRecord[]): string[]
+  /**
+   * OPTIONAL: record history references for the given hashes (one increment
+   * per occurrence). Called BEFORE the referencing record-set is persisted —
+   * a crash between the two can only over-count (a leak the scrub repairs),
+   * never under-count (which would risk reclaiming bytes history still needs).
+   */
+  recordHistoryBlobReferences?(hashes: string[]): Promise<void>
+  /**
+   * OPTIONAL: release history references for the given hashes (one decrement
+   * per occurrence) and physically reclaim any hash left with zero live AND
+   * zero history references. Called AFTER the referencing record-set is
+   * deleted by compaction — same over-count-only crash ordering.
+   */
+  releaseHistoryBlobReferences?(hashes: string[]): Promise<void>
+
   /**
    * Register the generation-bump hook invoked on every entity-visible
    * single-operation write (see `BaseStorage.setGenerationBumpHook`).

src/errors/brainyError.ts

@@ -14,6 +14,9 @@ export type BrainyErrorType =
   | 'FIELD_NOT_INDEXED'
   | 'GRAPH_INDEX_NOT_READY'
   | 'METADATA_INDEX_NOT_READY'
+  | 'VECTOR_INDEX_NOT_READY'
+  | 'PROTECTED_ARTIFACT'
+  | 'DERIVED_ARTIFACT_MISSING'
   | 'MIGRATION_IN_PROGRESS'
 
 /**
@@ -280,6 +283,86 @@ export class MetadataIndexNotReadyError extends BrainyError {
     }
 }
 
+/**
+ * Thrown when the vector index reports vectors (`size() > 0` or, on a native
+ * provider, `isReady() === false`) but cannot return a KNOWN persisted vector
+ * even after a rebuild — i.e. the semantic serving structure did not load on a
+ * cold open and could not be restored. The vector-search counterpart of
+ * {@link GraphIndexNotReadyError} / {@link MetadataIndexNotReadyError}: it
+ * replaces the silent-empty failure mode (a cold `find({ query })` returning
+ * `[]` indistinguishable from "no similar data") with a loud, catchable error,
+ * so a consumer never renders "nothing found" over data that is simply
+ * not-yet-warm.
+ *
+ * Detected once per brain by a known-vector serving probe on the first
+ * semantic / proximity `find()`; brainy self-heals (rebuilds the index from the
+ * canonical records) first and only raises this if the rebuild still cannot
+ * serve the known vector.
+ */
+export class VectorIndexNotReadyError extends BrainyError {
+    constructor(message: string, originalError?: Error) {
+        super(message, 'VECTOR_INDEX_NOT_READY', false, originalError)
+        this.name = 'VectorIndexNotReadyError'
+        if (Error.captureStackTrace) {
+            Error.captureStackTrace(this, VectorIndexNotReadyError)
+        }
+    }
+}
+
+/**
+ * Thrown when a delete (`deleteBinaryBlob` / `removeRawPrefix`) would remove a
+ * blob that is a declared member of a protected derived-index FAMILY (the
+ * registered-blob contract). Declared derived artifacts are undeletable through
+ * the storage layer — this makes an in-process GC / sweeper INCAPABLE of removing
+ * a load-bearing index file (the lost-`main.dkann` class). Intentional retirement
+ * is the explicit `unregisterDerivedFamily(name)` step, then the delete.
+ */
+export class ProtectedArtifactError extends BrainyError {
+    /** The blob key the delete targeted. */
+    public readonly key: string
+    /** The protected family the key belongs to. */
+    public readonly family: string
+    constructor(key: string, family: string) {
+        super(
+            `Refused to delete '${key}': it is a declared member of the protected ` +
+                `derived-index family '${family}'. Declared derived artifacts are undeletable ` +
+                `through the storage layer (COLD ≠ DEAD) — unregisterDerivedFamily('${family}') ` +
+                `first if retirement is intentional.`,
+            'PROTECTED_ARTIFACT',
+            false
+        )
+        this.name = 'ProtectedArtifactError'
+        this.key = key
+        this.family = family
+    }
+}
+
+/**
+ * Raised (loudly) when a declared derived-index family is missing one or more of
+ * its members on open — i.e. a load-bearing blob was deleted OUTSIDE the write
+ * path (an external sweeper the in-process refusal cannot stop). The index must
+ * be rebuilt from canonical; "healthy-while-broken" is impossible because the
+ * missing member is named, not silently tolerated.
+ */
+export class DerivedArtifactMissingError extends BrainyError {
+    /** The family with missing members. */
+    public readonly family: string
+    /** The member keys that are absent. */
+    public readonly missing: string[]
+    constructor(family: string, missing: string[]) {
+        super(
+            `Derived-index family '${family}' is missing ${missing.length} declared ` +
+                `member(s) on open (${missing.join(', ')}) — deleted outside the write path. ` +
+                `The index must be rebuilt from canonical.`,
+            'DERIVED_ARTIFACT_MISSING',
+            false
+        )
+        this.name = 'DerivedArtifactMissingError'
+        this.family = family
+        this.missing = missing
+    }
+}
+
 /**
  * Thrown when a data-plane read or write is issued against a brain that is
  * running its one-time, automatic 7.x → 8.0 on-disk upgrade — the coordinated

src/events/changeFeed.ts

@@ -0,0 +1,171 @@
+/**
+ * @module events/changeFeed
+ * @description The in-process change feed behind {@link Brainy.onChange} — the
+ * authoritative "something committed" signal for every canonical mutation,
+ * regardless of origin (direct API calls, batches, transactions, imports, the
+ * VFS, or an accelerated native deployment: all of them funnel through the
+ * same generation-store commit points this feed is emitted from).
+ *
+ * Design properties:
+ * - **Post-commit only.** Events are enqueued after the commit succeeds, so an
+ *   aborted commit (a losing `ifRev` CAS, a failed transaction) never emits.
+ * - **Commit-ordered.** Events are enqueued in commit order and dispatched
+ *   FIFO, so a subscriber observes mutations in the order they became durable.
+ * - **Never blocks the write path.** Dispatch happens in a microtask after the
+ *   committing call returns; a slow or throwing listener cannot delay or fail
+ *   a write. Listener errors are isolated per listener and per event.
+ * - **Zero cost when unused.** Callers consult {@link ChangeFeed.hasListeners}
+ *   before constructing event payloads; with no subscribers the write path
+ *   does no event work at all.
+ * - **Fire-and-forget.** No acknowledgement, backpressure, or replay. Events
+ *   carry the committed `generation`, so a consumer that needs catch-up
+ *   semantics can pair the live feed with `transactionLog()` / `asOf()`.
+ */
+
+/** The post-commit view of an entity carried by entity change events. */
+export interface ChangeEventEntity {
+  /** The entity's canonical UUID. */
+  id: string
+  /** The entity's NounType string (e.g. `'person'`, `'document'`). */
+  type: string
+  /** The entity's subtype, when set. */
+  subtype?: string
+  /** The entity's custom (indexed) metadata fields. */
+  metadata: Record<string, unknown>
+  /** The writing service, when set. */
+  service?: string
+}
+
+/** The post-commit view of a relationship carried by relation change events. */
+export interface ChangeEventRelation {
+  /** The relationship's canonical UUID. */
+  id: string
+  /** Source entity UUID. */
+  from: string
+  /** Target entity UUID. */
+  to: string
+  /** The relationship's VerbType string (e.g. `'contains'`). */
+  type: string
+  /** The relationship's custom metadata fields, when present. */
+  metadata?: Record<string, unknown>
+}
+
+/**
+ * One committed mutation, as delivered to {@link Brainy.onChange} listeners.
+ *
+ * Exactly one of `entity` / `relation` is populated for `kind: 'entity'` /
+ * `kind: 'relation'` events. `kind: 'store'` events (`clear` / `restore`)
+ * carry neither — they mean "the whole store changed; refetch what you care
+ * about".
+ *
+ * For `op: 'remove'` / `op: 'unrelate'` the payload is the record's LAST
+ * committed state (sourced from the commit's own before-image), so deletes are
+ * fully described rather than id-only.
+ */
+export interface BrainyChangeEvent {
+  /** What changed: one record, one relationship, or the whole store. */
+  kind: 'entity' | 'relation' | 'store'
+  /** The mutation, in Brainy's own API vocabulary. */
+  op:
+    | 'add'
+    | 'update'
+    | 'remove'
+    | 'relate'
+    | 'unrelate'
+    | 'updateRelation'
+    | 'clear'
+    | 'restore'
+  /** The mutated record's id (absent for store-level events). */
+  id?: string
+  /** Post-commit entity view (entity events only). */
+  entity?: ChangeEventEntity
+  /** Post-commit relation view (relation events only). */
+  relation?: ChangeEventRelation
+  /**
+   * The committed generation this mutation belongs to (Model B: every write
+   * is a generation; a transaction's items share one). Absent for store-level
+   * events and init-time bootstrap writes, which are not generation-stamped.
+   */
+  generation?: number
+  /** Commit timestamp (ms since epoch). */
+  timestamp: number
+}
+
+/** Listener signature for {@link Brainy.onChange}. */
+export type ChangeListener = (event: BrainyChangeEvent) => void
+
+/**
+ * A change event as constructed by a mutation method BEFORE its commit: the
+ * commit seam stamps `generation`/`timestamp` after the write becomes
+ * durable. An entity `remove` descriptor may omit `entity` — the seam fills
+ * it from the commit's own before-image (the record's last committed state).
+ */
+export type PendingChangeEvent = Omit<BrainyChangeEvent, 'generation' | 'timestamp'>
+
+/**
+ * @description Listener registry + commit-ordered async dispatcher for
+ * {@link BrainyChangeEvent}s. One instance per {@link Brainy}.
+ */
+export class ChangeFeed {
+  private listeners = new Set<ChangeListener>()
+  private queue: BrainyChangeEvent[] = []
+  private draining = false
+
+  /** Whether any listener is subscribed — the write path's zero-cost gate. */
+  get hasListeners(): boolean {
+    return this.listeners.size > 0
+  }
+
+  /**
+   * @description Subscribe to committed mutations.
+   * @param listener - Called once per committed mutation, in commit order.
+   * @returns An unsubscribe function.
+   */
+  subscribe(listener: ChangeListener): () => void {
+    this.listeners.add(listener)
+    return () => {
+      this.listeners.delete(listener)
+    }
+  }
+
+  /**
+   * @description Enqueue committed events and schedule dispatch. Call ONLY
+   * after the commit has succeeded — this feed must never announce a write
+   * that did not become durable. Safe to call with an empty array.
+   * @param events - The committed mutations, in commit order.
+   */
+  emit(events: BrainyChangeEvent[]): void {
+    if (events.length === 0 || this.listeners.size === 0) return
+    this.queue.push(...events)
+    if (!this.draining) {
+      this.draining = true
+      queueMicrotask(() => this.drain())
+    }
+  }
+
+  /** Deliver everything queued, FIFO, isolating listener errors. */
+  private drain(): void {
+    try {
+      while (this.queue.length > 0) {
+        const event = this.queue.shift()!
+        for (const listener of this.listeners) {
+          try {
+            listener(event)
+          } catch (err) {
+            // A subscriber's bug must never affect the write path or its
+            // sibling subscribers.
+            console.error('[Brainy] onChange listener threw:', err)
+          }
+        }
+      }
+    } finally {
+      this.draining = false
+    }
+  }
+
+  /** Drop all listeners (brain close/teardown). Queued events are discarded. */
+  close(): void {
+    this.listeners.clear()
+    this.queue.length = 0
+  }
+}

src/graph/graphAdjacencyIndex.ts

@@ -282,6 +282,16 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
       }
 
       hasMore = result.hasMore
+      if (hasMore && (!result.nextCursor || result.nextCursor === cursor)) {
+        // A stalled cursor with hasMore=true would re-read the same page
+        // forever — a silent full-CPU loop at cold open. Abort loudly; a
+        // graph read failing beats a process that spins without a log line.
+        throw new Error(
+          `GraphAdjacencyIndex: verb walk stalled after ${count} verbs — storage returned ` +
+            `hasMore=true with ${result.nextCursor ? 'a non-advancing' : 'no'} cursor. ` +
+            `Aborting the cold-load; run brain.repairIndex() if this persists.`
+        )
+      }
       cursor = result.nextCursor
     }
 

src/graph/graphAudit.ts

@@ -0,0 +1,214 @@
+/**
+ * @module graph/graphAudit
+ * @description Read-only graph-truth audit — the graph sibling of `repairIndex()`'s
+ * diagnosis half. Verifies three layers against each other without mutating anything:
+ *
+ *   1. CANONICAL verb records (the storage walk — the source of truth)
+ *   2. the RELATIONSHIP READ PATH (`related()` with all visibility tiers — exactly
+ *      what application reads like a VFS `readdir` consult)
+ *   3. ENTITY ENDPOINTS (does each verb's source/target still exist?)
+ *
+ * and classifies every discrepancy into the three failure families production
+ * incidents have shown:
+ *
+ *   - `missingFromReads` — a canonical verb record the read path does NOT return
+ *     for its source: PRESENT BUT INVISIBLE (adjacency/membership staleness).
+ *   - `danglingEndpoints` — a canonical verb whose endpoint entity is gone:
+ *     the SCAR class (write-path loss / partial delete).
+ *   - `readOnlyVerbIds` — the read path returns an edge with NO canonical
+ *     record: GHOST edges (stale index entries).
+ *
+ * `visibilityHiddenCount` is reported separately: an internal/system edge that is
+ * indexed and present but hidden from DEFAULT reads is working as designed — the
+ * audit reads with all tiers included so design-hiding is never misclassified as
+ * index loss.
+ *
+ * Full counts are always exact; only the example LISTS are capped (`maxExamples`)
+ * — a capped report says so via `truncatedExamples`, never silently.
+ */
+
+import { prodLog } from '../utils/logger.js'
+
+/** One discrepant relationship, identified fully enough to inspect by hand. */
+export interface GraphAuditDiscrepancy {
+  verbId: string
+  from: string
+  to: string
+  type: string
+}
+
+export interface GraphAuditReport {
+  /** True iff every discrepancy count is zero — `related()` returns canonical truth. */
+  coherent: boolean
+  verbsInCanonical: number
+  entitiesInCanonical: number
+  /** Distinct source entities whose read path was actually consulted (coverage honesty). */
+  sourcesChecked: number
+
+  /** PRESENT BUT INVISIBLE: canonical records the read path omits. */
+  missingFromReadsCount: number
+  missingFromReads: GraphAuditDiscrepancy[]
+
+  /** SCAR CLASS: canonical verbs with a missing endpoint entity. */
+  danglingEndpointsCount: number
+  danglingEndpoints: Array<GraphAuditDiscrepancy & { missingEnd: 'from' | 'to' | 'both' }>
+
+  /** GHOST EDGES: read-path verb ids with no canonical record. */
+  readOnlyCount: number
+  readOnlyVerbIds: string[]
+
+  /** Canonical verbs hidden from DEFAULT reads by design (internal/system visibility). */
+  visibilityHiddenCount: number
+
+  /** Example lists above were capped at maxExamples; counts remain exact. */
+  truncatedExamples: boolean
+  durationMs: number
+}
+
+/** A canonical verb record, as the audit needs it. */
+export interface AuditVerbRecord {
+  id: string
+  type: string
+  sourceId: string
+  targetId: string
+  visibility?: string
+}
+
+/** The seams the audit runs over — injected so the walk is testable in isolation. */
+export interface GraphAuditDeps {
+  /** Stream every canonical entity id (id-only; no per-entity reads needed). */
+  eachNounId(consume: (id: string) => void): Promise<void>
+  /** Stream every canonical verb record. */
+  eachVerb(consume: (verb: AuditVerbRecord) => void): Promise<void>
+  /**
+   * The END-TO-END relationship read for one source, ALL visibility tiers
+   * included — must be the same path application reads consult.
+   */
+  readRelationsFrom(sourceId: string): Promise<Array<{ id: string }>>
+}
+
+export interface GraphAuditOptions {
+  /** Cap on entries per example list (counts stay exact). Default 100. */
+  maxExamples?: number
+}
+
+export async function runGraphAudit(
+  deps: GraphAuditDeps,
+  options: GraphAuditOptions = {}
+): Promise<GraphAuditReport> {
+  const maxExamples = options.maxExamples ?? 100
+  const started = Date.now()
+
+  // 1. Canonical entity ids — endpoint existence oracle.
+  const entityIds = new Set<string>()
+  await deps.eachNounId((id) => entityIds.add(id))
+
+  // 2. Canonical verb walk: group by source, check endpoints, note visibility.
+  const canonicalVerbIds = new Set<string>()
+  const bySource = new Map<string, AuditVerbRecord[]>()
+  let verbsInCanonical = 0
+  let visibilityHiddenCount = 0
+  let danglingEndpointsCount = 0
+  const danglingEndpoints: GraphAuditReport['danglingEndpoints'] = []
+
+  await deps.eachVerb((verb) => {
+    verbsInCanonical++
+    canonicalVerbIds.add(verb.id)
+    const list = bySource.get(verb.sourceId)
+    if (list) list.push(verb)
+    else bySource.set(verb.sourceId, [verb])
+
+    if (verb.visibility === 'internal' || verb.visibility === 'system') {
+      visibilityHiddenCount++
+    }
+
+    const fromMissing = !entityIds.has(verb.sourceId)
+    const toMissing = !entityIds.has(verb.targetId)
+    if (fromMissing || toMissing) {
+      danglingEndpointsCount++
+      if (danglingEndpoints.length < maxExamples) {
+        danglingEndpoints.push({
+          verbId: verb.id,
+          from: verb.sourceId,
+          to: verb.targetId,
+          type: verb.type,
+          missingEnd: fromMissing && toMissing ? 'both' : fromMissing ? 'from' : 'to'
+        })
+      }
+    }
+  })
+
+  // 3. Per-source read-path comparison. A verb must be returned by the read
+  //    path of ITS OWN source — the exact consult a readdir/traversal makes.
+  let missingFromReadsCount = 0
+  const missingFromReads: GraphAuditDiscrepancy[] = []
+  let readOnlyCount = 0
+  const readOnlyVerbIds: string[] = []
+  const readOnlySeen = new Set<string>()
+
+  for (const [sourceId, verbs] of bySource) {
+    const readIds = new Set((await deps.readRelationsFrom(sourceId)).map((r) => r.id))
+
+    for (const verb of verbs) {
+      if (!readIds.has(verb.id)) {
+        missingFromReadsCount++
+        if (missingFromReads.length < maxExamples) {
+          missingFromReads.push({
+            verbId: verb.id,
+            from: verb.sourceId,
+            to: verb.targetId,
+            type: verb.type
+          })
+        }
+      }
+    }
+
+    for (const readId of readIds) {
+      if (!canonicalVerbIds.has(readId) && !readOnlySeen.has(readId)) {
+        readOnlySeen.add(readId)
+        readOnlyCount++
+        if (readOnlyVerbIds.length < maxExamples) {
+          readOnlyVerbIds.push(readId)
+        }
+      }
+    }
+  }
+
+  const coherent =
+    missingFromReadsCount === 0 && danglingEndpointsCount === 0 && readOnlyCount === 0
+
+  const report: GraphAuditReport = {
+    coherent,
+    verbsInCanonical,
+    entitiesInCanonical: entityIds.size,
+    sourcesChecked: bySource.size,
+    missingFromReadsCount,
+    missingFromReads,
+    danglingEndpointsCount,
+    danglingEndpoints,
+    readOnlyCount,
+    readOnlyVerbIds,
+    visibilityHiddenCount,
+    truncatedExamples:
+      missingFromReadsCount > missingFromReads.length ||
+      danglingEndpointsCount > danglingEndpoints.length ||
+      readOnlyCount > readOnlyVerbIds.length,
+    durationMs: Date.now() - started
+  }
+
+  if (coherent) {
+    prodLog.info(
+      `[GraphAudit] coherent: ${verbsInCanonical} verbs across ${bySource.size} sources — ` +
+        `the read path returns canonical truth (${report.durationMs}ms)`
+    )
+  } else {
+    prodLog.warn(
+      `[GraphAudit] INCOHERENT: ${missingFromReadsCount} present-but-invisible, ` +
+        `${danglingEndpointsCount} dangling-endpoint, ${readOnlyCount} ghost ` +
+        `(of ${verbsInCanonical} canonical verbs, ${bySource.size} sources, ` +
+        `${visibilityHiddenCount} visibility-hidden by design) — ${report.durationMs}ms`
+    )
+  }
+
+  return report
+}

src/graph/lsm/LSMTree.ts

@@ -574,6 +574,7 @@ export class LSMTree {
    * Load SSTables from storage based on manifest
    */
   private async loadSSTables(): Promise<void> {
+    const failures: string[] = []
     const loadPromises: Promise<void>[] = []
 
     this.manifest.sstables.forEach((level, sstableId) => {
@@ -598,7 +599,12 @@ export class LSMTree {
             }
           }
         } catch (error) {
+          // A per-SSTable load failure means the persisted adjacency is INCOMPLETE.
+          // Record it and fail the whole load closed (below): a partially-loaded
+          // tree that still publishes its full manifest count via size() would
+          // serve silent-empty traversals as truth (the cold-load swallow class).
           prodLog.warn(`LSMTree: Failed to load SSTable ${sstableId}`, error)
+          failures.push(sstableId)
         }
       })()
 
@@ -606,6 +612,20 @@ export class LSMTree {
     })
 
     await Promise.all(loadPromises)
+
+    if (failures.length > 0) {
+      // Fail closed. loadManifest()'s catch resets sstables/totalRelationships/
+      // sstablesByLevel to honest-empty, so size() reports 0 and the graph
+      // self-heal (_initializeGraphIndex size()===0 → rebuild) restores the index
+      // from the canonical records. Honest-partial is never published.
+      throw new Error(
+        `LSMTree(${this.config.storagePrefix}): ${failures.length} of ` +
+          `${this.manifest.sstables.size} SSTable(s) failed to load ` +
+          `(${failures.join(', ')}) — failing the load closed so size() reports 0 ` +
+          `and the graph self-heal rebuilds from canonical.`
+      )
+    }
+
     prodLog.info(`LSMTree: Loaded ${this.manifest.sstables.size} SSTables`)
   }
 

src/hnsw/hnswIndex.ts

@@ -25,6 +25,29 @@ const DEFAULT_CONFIG: HNSWConfig = {
   ml: 16 // Max level
 }
 
+/**
+ * @description Thrown by {@link JsHnswVectorIndex.flush} when one or more dirty
+ * nodes (or the system record) could not be persisted. The failed nodes remain
+ * in the dirty set for the next flush; this error tells the caller the flush did
+ * NOT achieve durability instead of a node-count that lies. Mandate: loud
+ * errors, never quiet losses.
+ */
+export class HnswFlushError extends Error {
+  constructor(
+    public readonly failedNodeCount: number,
+    public readonly systemFailed: boolean,
+    public override readonly cause?: Error
+  ) {
+    super(
+      `HNSW flush did not achieve durability: ${failedNodeCount} node(s) failed to ` +
+        `persist${systemFailed ? ' and the system record (entryPoint/maxLevel) failed' : ''}. ` +
+        `Failed nodes remain dirty for retry.` +
+        (cause ? ` First error: ${cause.message}` : '')
+    )
+    this.name = 'HnswFlushError'
+  }
+}
+
 /**
  * Implements {@link VectorIndexProvider}: the vector-index surface Brainy calls
  * on whatever the `'vector'` factory returns (its own `JsHnswVectorIndex`, or a native
@@ -150,41 +173,69 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
     const startTime = Date.now()
     const nodeCount = this.dirtyNodes.size
 
-    // Batch persist all dirty nodes concurrently
+    // Batch persist all dirty nodes concurrently. A node whose connections FAIL
+    // to persist must stay dirty (retried on the next flush) — clearing it would
+    // silently drop the write forever. Track failures; only successfully-
+    // persisted (or deleted) nodes leave the dirty set, so nodes added to it
+    // during this flush are preserved.
+    const failedNodes = new Set<string>()
+    let firstError: Error | null = null
+
     if (this.dirtyNodes.size > 0) {
       const batchSize = 50 // Reasonable batch size for cloud storage
       const nodeIds = Array.from(this.dirtyNodes)
 
       for (let i = 0; i < nodeIds.length; i += batchSize) {
         const batch = nodeIds.slice(i, i + batchSize)
-        const promises = batch.map(nodeId => {
+        const promises = batch.map(async nodeId => {
           const noun = this.nouns.get(nodeId)
-          if (!noun) return Promise.resolve() // Node was deleted
-
-          return this.persistNodeConnections(nodeId, noun).catch(error => {
-            console.error(`[HNSW flush] Failed to persist node ${nodeId}:`, error)
-          })
+          if (!noun) return // Node was deleted — drop it from the dirty set.
+          try {
+            await this.persistNodeConnections(nodeId, noun)
+          } catch (error) {
+            failedNodes.add(nodeId)
+            if (firstError === null) firstError = error as Error
+            prodLog.error(`[HNSW flush] Failed to persist node ${nodeId}: ${(error as Error).message}`)
+          }
         })
 
-        await Promise.allSettled(promises)
+        await Promise.all(promises)
       }
 
-      this.dirtyNodes.clear()
+      // Remove only nodes that were persisted (or deleted mid-flush); keep the
+      // failed ones dirty for the next attempt.
+      for (const nodeId of nodeIds) {
+        if (!failedNodes.has(nodeId)) this.dirtyNodes.delete(nodeId)
+      }
     }
 
-    // Persist system data if dirty
+    // Persist system data if dirty — keep it dirty on failure so the next flush
+    // retries rather than losing the entry-point/maxLevel update.
+    let systemFailed = false
     if (this.dirtySystem) {
-      await this.storage.saveHNSWSystem({
-        entryPointId: this.entryPointId,
-        maxLevel: this.maxLevel
-      }).catch(error => {
-        console.error('[HNSW flush] Failed to persist system data:', error)
-      })
-
-      this.dirtySystem = false
+      try {
+        await this.storage.saveHNSWSystem({
+          entryPointId: this.entryPointId,
+          maxLevel: this.maxLevel
+        })
+        this.dirtySystem = false
+      } catch (error) {
+        systemFailed = true
+        if (firstError === null) firstError = error as Error
+        prodLog.error(`[HNSW flush] Failed to persist system data: ${(error as Error).message}`)
+      }
     }
 
     const duration = Date.now() - startTime
+
+    // Loud failure: if ANY node or the system record could not be persisted the
+    // flush did not achieve durability. Throw so callers (close(), explicit
+    // flush(), the flush-request watcher) see the failure instead of a success
+    // count that lies. The failed nodes/system stay dirty for retry.
+    if (failedNodes.size > 0 || systemFailed) {
+      throw new HnswFlushError(failedNodes.size, systemFailed, firstError ?? undefined)
+    }
+
     if (nodeCount > 0) {
       prodLog.info(`[HNSW] Flushed ${nodeCount} dirty nodes in ${duration}ms`)
     }
@@ -396,13 +447,14 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
       this.maxLevel = nounLevel
       this.nouns.set(id, noun)
 
-      // Persist system data for first noun (previously skipped)
+      // Persist system data for first noun (previously skipped). Surface a
+      // persist failure loudly — the entry point is the root of the whole index;
+      // silently dropping it while addItem() returns the id would strand every
+      // future search on a rootless index. Mandate: never a quiet loss.
       if (this.storage && this.persistMode === 'immediate') {
         await this.storage.saveHNSWSystem({
           entryPointId: this.entryPointId,
           maxLevel: this.maxLevel
-        }).catch(error => {
-          console.error('Failed to persist initial HNSW system data:', error)
         })
       } else if (this.persistMode === 'deferred') {
         this.dirtySystem = true

src/import/BackgroundDeduplicator.ts

@@ -41,6 +41,14 @@ export interface DeduplicationStats {
  * - Import-scoped deduplication (no cross-contamination)
  * - 3-tier strategy (ID → Name → Similarity)
  * - Uses existing indexes (EntityIdMapper, MetadataIndexManager, TypeAware HNSW)
+ *
+ * Lifecycle: ONE instance per brain, owned by Brainy (getBackgroundDeduplicator)
+ * so the debounce genuinely spans imports and brain.close() cancels pending
+ * work via cancelPending() — this pass merge-DELETES duplicate entities, so it
+ * must never fire against a closed brain. The enableDeduplication gate lives
+ * at the scheduling call site (ImportCoordinator); scheduleDedup itself is
+ * unconditional. The timer is unref'd — a pending pass never holds the
+ * process open.
  */
 export class BackgroundDeduplicator {
   private brain: Brainy
@@ -67,12 +75,15 @@ export class BackgroundDeduplicator {
       clearTimeout(this.debounceTimer)
     }
 
-    // Schedule for 5 minutes from now
+    // Schedule for 5 minutes from now. unref'd: a pending dedup pass must
+    // never hold the process open (exit-hang class) — if the process exits
+    // first, the pass simply never runs; imports are already durable.
     this.debounceTimer = setTimeout(() => {
       this.runBatchDedup().catch(error => {
         prodLog.error('[BackgroundDedup] Batch dedup failed:', error)
       })
     }, 5 * 60 * 1000)
+    this.debounceTimer.unref?.()
   }
 
   /**

src/import/ImportCoordinator.ts

@@ -13,7 +13,6 @@
 import { Brainy } from '../brainy.js'
 import { FormatDetector, SupportedFormat } from './FormatDetector.js'
 import { ImportHistory, type ImportHistoryEntry } from './ImportHistory.js'
-import { BackgroundDeduplicator } from './BackgroundDeduplicator.js'
 import { SmartExcelImporter } from '../importers/SmartExcelImporter.js'
 import { SmartPDFImporter } from '../importers/SmartPDFImporter.js'
 import { SmartCSVImporter } from '../importers/SmartCSVImporter.js'
@@ -112,7 +111,12 @@ export interface ValidImportOptions {
   /** Confidence threshold for entities */
   confidenceThreshold?: number
 
-  /** Enable entity deduplication across imports */
+  /**
+   * Enable entity deduplication (default: true). Gates BOTH passes: the
+   * inline merge during import AND the debounced background pass that runs
+   * ~5 minutes after the last import (which merge-DELETES duplicate entities).
+   * Set false for deployments that must never auto-remove records.
+   */
   enableDeduplication?: boolean
 
   /** Similarity threshold for deduplication (0-1) */
@@ -286,7 +290,6 @@ export class ImportCoordinator {
   private brain: Brainy
   private detector: FormatDetector
   private history: ImportHistory
-  private backgroundDedup: BackgroundDeduplicator
   private excelImporter: SmartExcelImporter
   private pdfImporter: SmartPDFImporter
   private csvImporter: SmartCSVImporter
@@ -300,7 +303,6 @@ export class ImportCoordinator {
     this.brain = brain
     this.detector = new FormatDetector()
     this.history = new ImportHistory(brain)
-    this.backgroundDedup = new BackgroundDeduplicator(brain)
     this.excelImporter = new SmartExcelImporter(brain)
     this.pdfImporter = new SmartPDFImporter(brain)
     this.csvImporter = new SmartCSVImporter(brain)
@@ -1459,9 +1461,16 @@ export class ImportCoordinator {
       }
     }
 
-    // Schedule background deduplication (debounced 5 minutes)
-    if (trackingContext && trackingContext.importId) {
-      this.backgroundDedup.scheduleDedup(trackingContext.importId)
+    // Schedule background deduplication (debounced 5 minutes, brain-owned so
+    // close() can cancel it). Honors the same enableDeduplication gate as the
+    // inline pass — false means NO dedup, inline or background.
+    if (
+      trackingContext &&
+      trackingContext.importId &&
+      options.enableDeduplication !== false
+    ) {
+      const backgroundDedup = await this.brain.getBackgroundDeduplicator()
+      backgroundDedup.scheduleDedup(trackingContext.importId)
     }
 
     return {

src/index.ts

@@ -15,8 +15,29 @@ import { Brainy } from './brainy.js'
 
 export { Brainy }
 
+// The in-process change feed (brain.onChange) — event + listener types.
+export type {
+  BrainyChangeEvent,
+  ChangeEventEntity,
+  ChangeEventRelation,
+  ChangeListener
+} from './events/changeFeed.js'
+
+// Temporal VFS — a file version entry (vfs.history / readFile({ asOf })).
+export type { FileVersion } from './vfs/types.js'
+
 // Export diagnostics result type
 export type { DiagnosticsResult } from './brainy.js'
+export type {
+  GraphAuditReport,
+  GraphAuditDiscrepancy
+} from './graph/graphAudit.js'
+export {
+  checkOsLimits,
+  NOFILE_POOL_FLOOR,
+  MAX_MAP_COUNT_POOL_FLOOR
+} from './utils/osLimits.js'
+export type { OsLimitsReport } from './utils/osLimits.js'
 
 // Export Brainy configuration and types
 export type {
@@ -139,7 +160,7 @@ export { EntityNotFoundError, RelationNotFoundError } from './errors/notFound.js
 
 // Base error + typed migration-lock error — thrown by any data-plane call while a
 // brain runs its one-time 7.x→8.0 upgrade; catch to answer HTTP 503 + Retry-After.
-export { BrainyError, MigrationInProgressError, GraphIndexNotReadyError, MetadataIndexNotReadyError } from './errors/brainyError.js'
+export { BrainyError, MigrationInProgressError, GraphIndexNotReadyError, MetadataIndexNotReadyError, VectorIndexNotReadyError, ProtectedArtifactError, DerivedArtifactMissingError } from './errors/brainyError.js'
 export type { BrainyErrorType } from './errors/brainyError.js'
 
 // ============= 8.0 Db API — generational MVCC =============
@@ -163,8 +184,11 @@ export type {
 export {
   GenerationConflictError,
   SpeculativeOverlayError,
-  GenerationCompactedError
+  GenerationCompactedError,
+  StoreInconsistentError,
+  PendingFlushDurabilityError
 } from './db/errors.js'
+export type { UnreconciledRecord } from './db/errors.js'
 export type {
   TxOperation,
   TxAddOperation,
@@ -177,14 +201,30 @@ export type {
   TxLogEntry,
   CompactHistoryOptions,
   CompactHistoryResult,
+  HistoryStats,
   ChangedIds,
   DiffResult,
   HistoryVersion,
   EntityHistory
 } from './db/types.js'
+// The generation fact log — sequential after-image scan surface
+// (brain.scanFacts / brain.factSegmentPaths) for index heals and replays.
+export type {
+  CommitFact,
+  FactOp,
+  FactScanBatch,
+  SCANFACTS_FIRST_BATCH_MS,
+  FactScanHandle
+} from './db/factLog.js'
+// The generalized family stamp — which source generation a projection
+// reflects + the surface that verifies it whole; one verifier, both member
+// modes (enumerated byte-exact / rollup invariants).
+export { readFamilyStamp, verifyFamilyStamp, ENTITY_TREE_STAMP_PATH } from './db/familyStamp.js'
+export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.js'
 // Optional provider capability for generation-aware native indexes
 export { isVersionedIndexProvider } from './plugin.js'
 export type { VersionedIndexProvider } from './plugin.js'
+export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
 // Optional native graph-acceleration engine (cor 3.0) — the published provider
 // contract + its columnar wire types. Brainy feature-detects an implementation
 // and falls back to its pure-TS adjacency when absent.
@@ -286,7 +326,8 @@ import type {
   HNSWNoun,
   HNSWVerb,
   HNSWConfig,
-  StorageAdapter
+  StorageAdapter,
+  DerivedFamilyDeclaration
 } from './coreTypes.js'
 
 // Export vector index implementation (the JS HNSW path)
@@ -304,7 +345,8 @@ export type {
   HNSWNoun,
   HNSWVerb,
   HNSWConfig,
-  StorageAdapter
+  StorageAdapter,
+  DerivedFamilyDeclaration
 }
 
 // Export graph types

src/indexes/columnStore/ColumnStore.ts

@@ -69,6 +69,34 @@ interface HeapEntry {
  * await store.flush()
  * const newest = await store.sortTopK('createdAt', 'desc', 10) // bigint[]
  */
+/**
+ * @description Thrown when a segment the manifest lists cannot be loaded —
+ * either it yields no bytes (missing/empty on disk) or its bytes are
+ * undecodable (corruption). Previously such a segment was silently skipped,
+ * dropping ALL of its entities from every `filter`/`rangeQuery`/`sortTopK` with
+ * no error — a manifest↔segments divergence that looked like a short result.
+ * Surfacing it loudly makes the divergence visible and repairable. (A genuine
+ * storage IO fault is a different class — it propagates as the underlying error,
+ * not wrapped in this.)
+ */
+export class ColumnSegmentLoadError extends Error {
+  /** The indexed field whose segment failed to load. */
+  public readonly field: string
+  /** The manifest-listed segment id that could not be loaded. */
+  public readonly segmentId: number | string
+  constructor(field: string, segmentId: number | string, reason: string) {
+    super(
+      `ColumnStore segment ${field}:${segmentId} is listed in the manifest but ` +
+        `could not be loaded (${reason}). The index is inconsistent with its ` +
+        `manifest — rebuild/repair the metadata index rather than trusting a ` +
+        `short query result.`
+    )
+    this.name = 'ColumnSegmentLoadError'
+    this.field = field
+    this.segmentId = segmentId
+  }
+}
+
 export class ColumnStore implements ColumnStoreProvider {
   private storage!: StorageAdapter
   private idMapper!: EntityIdMapper
@@ -594,14 +622,15 @@ export class ColumnStore implements ColumnStoreProvider {
       let cursor = this.segmentCache.get(cacheKey)
 
       if (!cursor) {
-        const loaded = await this.loadSegmentCursor(field, seg)
-        if (loaded) {
-          cursor = loaded
-          this.segmentCache.set(cacheKey, cursor)
-        }
+        // loadSegmentCursor either returns a cursor or THROWS — a corrupt /
+        // missing manifest-listed segment raises ColumnSegmentLoadError and a
+        // real storage fault propagates, so a listed segment is never silently
+        // dropped from the result set.
+        cursor = await this.loadSegmentCursor(field, seg)
+        this.segmentCache.set(cacheKey, cursor)
       }
 
-      if (cursor) cursors.push(cursor)
+      cursors.push(cursor)
     }
 
     return cursors
@@ -615,39 +644,51 @@ export class ColumnStore implements ColumnStoreProvider {
    * `.cidx` object-path so indexes written before the format unification keep
    * loading correctly. Mirror of cortex's 2.3.1 read-side fallback.
    */
-  private async loadSegmentCursor(field: string, seg: SegmentMeta): Promise<ColumnSegmentCursor | null> {
+  private async loadSegmentCursor(field: string, seg: SegmentMeta): Promise<ColumnSegmentCursor> {
     const manifest = this.manifests.get(field)
-    if (!manifest) return null
+    if (!manifest) {
+      // Defensive: getSegmentCursors guards on the manifest before calling.
+      throw new ColumnSegmentLoadError(field, seg.id, 'no manifest for field')
+    }
 
-    try {
-      let buf: Buffer | null = null
+    let buf: Buffer | null = null
 
-      const storage = this.storage as unknown as {
-        loadBinaryBlob?: (key: string) => Promise<Buffer | null>
-        readObjectFromPath: (path: string) => Promise<any>
-      }
+    const storage = this.storage as unknown as {
+      loadBinaryBlob?: (key: string) => Promise<Buffer | null>
+      readObjectFromPath: (path: string) => Promise<any>
+    }
 
-      // Preferred: raw blob at the cor-shared key.
-      if (typeof storage.loadBinaryBlob === 'function') {
-        const key = `${this.basePath}/${field}/L${seg.level}-${String(seg.id).padStart(6, '0')}`
-        const blob = await storage.loadBinaryBlob(key)
-        if (blob && blob.length > 0) buf = blob
-      }
+    // Preferred: raw blob at the cor-shared key. A real IO fault PROPAGATES —
+    // loadBinaryBlob throws on a fault and returns null only for genuine absence
+    // (a present-but-unreadable segment must not read as "missing").
+    if (typeof storage.loadBinaryBlob === 'function') {
+      const key = `${this.basePath}/${field}/L${seg.level}-${String(seg.id).padStart(6, '0')}`
+      const blob = await storage.loadBinaryBlob(key)
+      if (blob && blob.length > 0) buf = blob
+    }
 
-      // Legacy fallback: `{ _binary, base64 }` envelope at the .cidx object-path.
-      if (!buf) {
-        const segPath = manifest.segmentPath(seg.level, seg.id)
-        const stored = await storage.readObjectFromPath(segPath)
-        if (!stored) return null
+    // Legacy fallback: `{ _binary, base64 }` envelope at the .cidx object-path.
+    if (!buf) {
+      const segPath = manifest.segmentPath(seg.level, seg.id)
+      const stored = await storage.readObjectFromPath(segPath)
+      if (stored) {
         if (stored._binary && stored.data) {
           buf = Buffer.from(stored.data, 'base64')
         } else if (Buffer.isBuffer(stored)) {
           buf = stored
-        } else {
-          return null
         }
       }
+    }
 
+    // A manifest-listed segment that yields NO loadable bytes is corruption, not
+    // benign absence: swallowing it silently dropped all of the segment's
+    // entities from every filter/rangeQuery/sortTopK with no error. Surface it
+    // loudly so the manifest↔segments divergence is visible (and repairable).
+    if (!buf) {
+      throw new ColumnSegmentLoadError(field, seg.id, 'manifest-listed segment has no loadable bytes')
+    }
+
+    try {
       const parsed = readSegmentFromBuffer(buf)
       return new ColumnSegmentCursor(
         parsed.header,
@@ -655,8 +696,13 @@ export class ColumnStore implements ColumnStoreProvider {
         parsed.entityIds,
         parsed.tombstones
       )
-    } catch {
-      return null
+    } catch (err) {
+      // Undecodable bytes for a listed segment — corruption, not a short result.
+      throw new ColumnSegmentLoadError(
+        field,
+        seg.id,
+        `segment decode failed: ${(err as Error).message}`
+      )
     }
   }
 

src/internals.ts

@@ -19,3 +19,29 @@ export type { MemoryInfo, CacheAllocationStrategy } from './utils/memoryDetectio
 // HNSWNounWithMetadata. First-party plugins (Cor) use this to stay in
 // lockstep with the entity shape contract.
 export { resolveEntityField, STANDARD_ENTITY_FIELDS } from './coreTypes.js'
+
+// The generalized family stamp — ONE verifier, both member modes, shared
+// verbatim with native providers so stamp verification is literally one
+// function, never two synchronized copies. Providers write the same shape
+// (their set-swap rewrites stamps, so adoption is migration-free).
+export {
+  readFamilyStamp,
+  writeFamilyStamp,
+  verifyFamilyStamp,
+  FAMILY_STAMPS_PREFIX,
+  ENTITY_TREE_STAMP_PATH
+} from './db/familyStamp.js'
+export type {
+  FamilyStamp,
+  StampMembers,
+  StampVerdict,
+  EnumeratedMember,
+  StampStorage
+} from './db/familyStamp.js'
+
+// Generation fact-log types — the scan surface a provider reaches through the
+// storage capability (`storage.scanFacts` / `storage.factLogHeadGeneration` /
+// `storage.factSegmentPaths`, wired by the host brain at init). Providers
+// never construct a FactLog themselves: open() is writer-side (it reconciles
+// by truncating/rewriting) and there is exactly one writer.
+export type { CommitFact, FactOp, FactScanBatch, FactScanHandle } from './db/factLog.js'

src/plugin.ts

@@ -115,6 +115,62 @@ export interface BrainyPluginContext {
 // implementation then fails to compile until it provides the member.
 // ===========================================================================
 
+/**
+ * @description How a failed provider invariant should be remediated:
+ * - `'none'`    — informational; the invariant held or nothing to do.
+ * - `'repair'`  — a targeted, cheap fix exists (e.g. re-derive a count/manifest field).
+ * - `'rebuild'` — the derived state must be rebuilt from canonical (`provider.rebuild()`).
+ */
+export type InvariantHeal = 'none' | 'repair' | 'rebuild'
+
+/**
+ * @description The result of ONE provider invariant check. A failure
+ * (`holds === false`) NAMES what diverged, with numbers, so it is diagnosable
+ * from the report alone — never a bare boolean. `name` is a stable kebab-case id
+ * for telemetry / remediation routing.
+ */
+export interface InvariantResult {
+  /** Stable kebab-case id, e.g. `'manifest-residency'` / `'posted-count-floor'`. */
+  name: string
+  /** `true` when the invariant holds. */
+  holds: boolean
+  /** Human-readable detail; on failure, names the divergence WITH numbers. */
+  detail: string
+  /** The value the invariant expected (optional, for diagnosis). */
+  expected?: unknown
+  /** The value actually observed (optional, for diagnosis). */
+  actual?: unknown
+  /** How a failure should be remediated. Ignored when `holds === true`. */
+  heal: InvariantHeal
+}
+
+/**
+ * @description A provider's self-report of its own cross-layer invariants
+ * (the `validateInvariants()` hook). Contract:
+ * - It NEVER throws — a failure is DATA (`healthy: false` + a failing invariant),
+ *   not an exception.
+ * - It is BOUNDED (<50ms): residency checks + O(1) counts only, NO canonical
+ *   walks — safe to call on a live brain, repeatedly.
+ * - `serving` = can the provider answer queries right now (the `isReady()` truth);
+ *   `healthy` = do ALL invariants hold. A provider can be `serving` while an
+ *   invariant flags a latent divergence, or `healthy` but not-yet-`serving` on a
+ *   cold open.
+ */
+export interface ProviderInvariantReport {
+  /** Which provider produced this report, e.g. `'vector'` / `'graph'` / `'metadata'` / `'column'`. */
+  provider: string
+  /** `true` iff every invariant in {@link invariants} holds. */
+  healthy: boolean
+  /** `true` iff the provider can serve queries now (the `isReady()` truth). */
+  serving: boolean
+  /** Each checked invariant and its verdict. */
+  invariants: InvariantResult[]
+  /** Epoch millis when the check ran. */
+  checkedAt: number
+  /** How long the check took (must stay well under 50ms). */
+  durationMs: number
+}
+
 /**
  * The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`.
  * Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
@@ -139,6 +195,18 @@ export interface MetadataIndexProvider {
    */
   isReady?(): boolean
 
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL. A native provider returns true from the moment its
    * `init()` detects a large epoch-drift until its background
@@ -289,6 +357,18 @@ export interface GraphIndexProvider {
    */
   isReady?(): boolean
 
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL eager cold-load. Called once during brain init — AFTER
    * the metadata provider's `init()` (so the id-mapper is hydrated; a native int
@@ -943,6 +1023,18 @@ export interface VectorIndexProvider {
    */
   isReady?(): boolean
 
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL. A native provider returns true from the moment its
    * `init()` detects a large epoch-drift until its background

src/storage/adapters/memoryStorage.ts

@@ -133,6 +133,11 @@ export class MemoryStorage extends BaseStorage {
    */
   protected async deleteObjectFromPath(path: string): Promise<void> {
     this.objectStore.delete(path)
+    // Filesystem parity: on disk, objects and raw BYTE files are both just
+    // files — unlink removes whichever exists. Without this, deleteRawObject
+    // on a raw-bytes path (fact-log/generation segments) silently no-ops on
+    // memory storage: the delete "succeeds" and the bytes remain.
+    this.rawBytesStore.delete(path)
   }
 
   /**
@@ -183,6 +188,9 @@ export class MemoryStorage extends BaseStorage {
    * @param key - The blob key.
    */
   public async deleteBinaryBlob(key: string): Promise<void> {
+    // Registered-blob contract — parity with the filesystem adapter:
+    // a declared family member is undeletable (throws ProtectedArtifactError).
+    await this.assertBlobKeyDeletable(key)
     this.blobStore.delete(key)
   }
 
@@ -219,6 +227,45 @@ export class MemoryStorage extends BaseStorage {
     return [...this.txLogLines]
   }
 
+  // ===========================================================================
+  // Binary raw-byte primitives — in-memory mirror of the filesystem adapter's
+  // append-only substrate (the generation fact log's segments), so memory
+  // brains dual-write facts too and the compat suite runs on both adapters.
+  // ===========================================================================
+
+  /** Raw binary files, keyed by verbatim path. */
+  private rawBytesStore: Map<string, Uint8Array> = new Map()
+
+  /** Append bytes to a raw binary file, creating it when absent. */
+  public async appendRawBytes(rawPath: string, bytes: Uint8Array): Promise<void> {
+    const existing = this.rawBytesStore.get(rawPath)
+    if (!existing) {
+      this.rawBytesStore.set(rawPath, bytes.slice())
+      return
+    }
+    const merged = new Uint8Array(existing.length + bytes.length)
+    merged.set(existing, 0)
+    merged.set(bytes, existing.length)
+    this.rawBytesStore.set(rawPath, merged)
+  }
+
+  /** Read a raw binary file whole (a copy); absent → null. */
+  public async readRawBytes(rawPath: string): Promise<Uint8Array | null> {
+    const bytes = this.rawBytesStore.get(rawPath)
+    return bytes ? bytes.slice() : null
+  }
+
+  /** Replace a raw binary file (atomic by construction in memory). */
+  public async writeRawBytes(rawPath: string, bytes: Uint8Array): Promise<void> {
+    this.rawBytesStore.set(rawPath, bytes.slice())
+  }
+
+  /** Byte size of a raw binary file, or null when absent. */
+  public async rawByteSize(rawPath: string): Promise<number | null> {
+    const bytes = this.rawBytesStore.get(rawPath)
+    return bytes ? bytes.length : null
+  }
+
   /**
    * Serialize the entire in-memory store to a directory in the exact layout
    * the filesystem adapter uses (uncompressed JSON objects, `_blobs/*.bin`
@@ -351,6 +398,7 @@ export class MemoryStorage extends BaseStorage {
   public async clear(): Promise<void> {
     this.objectStore.clear()
     this.blobStore.clear()
+    this.rawBytesStore.clear()
     this.txLogLines = []
     this.statistics = null
 

src/storage/blobStorage.ts

@@ -15,6 +15,7 @@
 
 import { createHash } from 'crypto'
 import { unwrapBinaryData } from './binaryDataCodec.js'
+import { InMemoryMutex } from '../utils/mutex.js'
 
 /**
  * @description Key-value bridge the blob store persists through. Implemented
@@ -47,8 +48,19 @@ export interface BlobMetadata {
   compression: 'none' | 'zstd'
   /** Creation timestamp (epoch ms). */
   createdAt: number
-  /** Number of logical references to this blob (deduplicated writes). */
+  /** Number of LIVE logical references to this blob (deduplicated writes). */
   refCount: number
+  /**
+   * Number of persisted generation record-sets (Model-B before-images) that
+   * reference this hash — the blob's membership in the temporal history.
+   * Bytes are physically reclaimed only when BOTH counts are zero, and only
+   * by history compaction: live references protect the present, history
+   * references protect every `asOf` read inside the retention window (pins
+   * ride generation pinning, which compaction already respects). Absent on
+   * metas written before the temporal contract existed (treated as 0; the
+   * one-time open-time backfill makes legacy stores exact).
+   */
+  historyRefCount?: number
 }
 
 /**
@@ -147,7 +159,9 @@ interface CacheEntry {
  * @example
  * const hash = await blobStorage.write(buffer, { mimeType: 'image/png' })
  * const bytes = await blobStorage.read(hash) // verified against the hash
- * await blobStorage.delete(hash)             // decrements refCount first
+ * await blobStorage.release(hash)            // drop one LIVE reference
+ * // bytes are physically reclaimed only by history compaction, once no
+ * // live reference AND no in-window generation references the hash
  */
 export class BlobStorage {
   private adapter: BlobStoreAdapter
@@ -164,6 +178,19 @@ export class BlobStorage {
   private readonly CACHE_MAX_SIZE = 100 * 1024 * 1024 // 100MB default
   private readonly COMPRESSION_THRESHOLD = 1024 // 1KB - don't compress smaller
 
+  /**
+   * Per-hash write serialization. Every reference-count-bearing mutation
+   * (`write`'s dedup check-then-act, `delete`'s decrement-then-maybe-remove)
+   * is a read-modify-write over `blob-meta:<hash>` — unserialized, two
+   * concurrent writes of identical content both saw "absent" and both wrote
+   * `refCount: 1` (one reference lost → a later delete removed bytes another
+   * file still referenced), and concurrent increments/decrements could drop
+   * counts. Keyed by hash, so distinct content never contends; the process
+   * is the whole concurrency domain (storage enforces single-writer per
+   * directory).
+   */
+  private readonly hashLocks = new InMemoryMutex()
+
   /**
    * @param adapter - Key-value bridge to persist through.
    * @param options - `cacheMaxSize` bounds the LRU read cache (bytes,
@@ -222,45 +249,53 @@ export class BlobStorage {
   async write(data: Buffer, options: BlobWriteOptions = {}): Promise<string> {
     const hash = BlobStorage.hash(data)
 
-    // Deduplication: identical content already stored — just add a reference.
-    if (await this.has(hash)) {
-      await this.incrementRefCount(hash)
+    // The dedup decision (exists → add a reference; absent → create with
+    // refCount 1) is check-then-act over the same metadata a concurrent
+    // same-content write mutates — serialized per hash so N concurrent
+    // writes of identical content yield exactly N references, never a lost
+    // count (a lost reference turns a later delete into premature removal
+    // of bytes another file still needs).
+    return this.hashLocks.runExclusive(hash, async () => {
+      // Deduplication: identical content already stored — just add a reference.
+      if (await this.has(hash)) {
+        await this.incrementRefCount(hash)
+        return hash
+      }
+
+      await this.ensureCompressionReady()
+
+      // Determine compression strategy
+      const compression = this.selectCompression(data, options)
+
+      // Compress if needed
+      let finalData = data
+      let compressedSize = data.length
+
+      if (compression === 'zstd' && this.zstdCompress) {
+        finalData = await this.zstdCompress(data)
+        compressedSize = finalData.length
+      }
+
+      // Record the ACTUAL compression state, not the intended one — prevents
+      // corruption if compression failed to initialize.
+      const actualCompression = finalData === data ? 'none' : compression
+      const metadata: BlobMetadata = {
+        hash,
+        size: data.length,
+        compressedSize,
+        compression: actualCompression,
+        createdAt: Date.now(),
+        refCount: 1
+      }
+
+      await this.adapter.put(`blob:${hash}`, finalData)
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+
+      // Write-through cache (caches the ORIGINAL bytes, not the compressed form)
+      this.addToCache(hash, data, metadata)
+
       return hash
-    }
-
-    await this.ensureCompressionReady()
-
-    // Determine compression strategy
-    const compression = this.selectCompression(data, options)
-
-    // Compress if needed
-    let finalData = data
-    let compressedSize = data.length
-
-    if (compression === 'zstd' && this.zstdCompress) {
-      finalData = await this.zstdCompress(data)
-      compressedSize = finalData.length
-    }
-
-    // Record the ACTUAL compression state, not the intended one — prevents
-    // corruption if compression failed to initialize.
-    const actualCompression = finalData === data ? 'none' : compression
-    const metadata: BlobMetadata = {
-      hash,
-      size: data.length,
-      compressedSize,
-      compression: actualCompression,
-      createdAt: Date.now(),
-      refCount: 1
-    }
-
-    await this.adapter.put(`blob:${hash}`, finalData)
-    await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
-
-    // Write-through cache (caches the ORIGINAL bytes, not the compressed form)
-    this.addToCache(hash, data, metadata)
-
-    return hash
+    })
   }
 
   /**
@@ -334,23 +369,105 @@ export class BlobStorage {
   }
 
   /**
-   * @description Drop one reference to the blob. The stored bytes and
-   * metadata are physically deleted only when the reference count reaches
-   * zero — deduplicated content shared by other writers survives.
+   * @description Drop one LIVE reference to the blob. Never deletes bytes —
+   * blob content is immutable under the temporal model, exactly like every
+   * other record: a past generation's `asOf` read may still need these bytes
+   * even when no live file references them. Physical reclamation happens in
+   * ONE place only — history compaction via {@link reclaimIfUnreferenced},
+   * once no live reference AND no retained generation references the hash.
    *
    * @param hash - The blob's SHA-256 hash.
    */
-  async delete(hash: string): Promise<void> {
-    const refCount = await this.decrementRefCount(hash)
+  async release(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      await this.decrementRefCount(hash)
+    })
+  }
 
-    // Only delete if no references remain
-    if (refCount > 0) {
-      return
-    }
+  /**
+   * @description Record that one persisted generation record-set references
+   * this hash (called by the commit path BEFORE the record-set is written —
+   * a crash between the two can only over-count, which leaks until the scrub
+   * recounts; it can never under-count, which would risk premature deletion).
+   * A missing meta (bytes never stored or already gone) is skipped with a
+   * warning — counting it could not make its bytes readable.
+   * @param hash - The blob's SHA-256 hash.
+   */
+  async recordHistoryReference(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) {
+        console.warn(
+          `[BlobStorage] history reference recorded for absent blob ${hash} — skipped`
+        )
+        return
+      }
+      metadata.historyRefCount = (metadata.historyRefCount ?? 0) + 1
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
 
-    await this.adapter.delete(`blob:${hash}`)
-    await this.adapter.delete(`blob-meta:${hash}`)
-    this.removeFromCache(hash)
+  /**
+   * @description Drop one history reference (called by compaction AFTER the
+   * referencing generation record-set is deleted — the safe ordering: a crash
+   * between the two over-counts, never under-counts). Floored at zero.
+   * @param hash - The blob's SHA-256 hash.
+   */
+  async releaseHistoryReference(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return
+      metadata.historyRefCount = Math.max(0, (metadata.historyRefCount ?? 0) - 1)
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
+
+  /**
+   * @description Physically delete the blob's bytes + metadata IFF nothing
+   * references it: zero live references AND zero history references. The one
+   * reclamation point in the system, invoked by history compaction after it
+   * releases the reclaimed generations' references. Atomic per hash.
+   * @param hash - The blob's SHA-256 hash.
+   * @returns `true` when the bytes were reclaimed.
+   */
+  async reclaimIfUnreferenced(hash: string): Promise<boolean> {
+    return this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return false
+      if ((metadata.refCount ?? 0) > 0 || (metadata.historyRefCount ?? 0) > 0) {
+        return false
+      }
+      await this.adapter.delete(`blob:${hash}`)
+      await this.adapter.delete(`blob-meta:${hash}`)
+      this.removeFromCache(hash)
+      return true
+    })
+  }
+
+  /**
+   * @description Set the history reference count to an absolute value — the
+   * backfill/scrub primitive (recounts derived from the actual generation
+   * records replace whatever the incremental counters hold). Idempotent.
+   * @param hash - The blob's SHA-256 hash.
+   * @param count - The exact history reference count.
+   */
+  async setHistoryRefCount(hash: string, count: number): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return
+      metadata.historyRefCount = Math.max(0, count)
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
+
+  /**
+   * @description Enumerate every stored blob hash (from the metadata keys) —
+   * the backfill/scrub walk. O(stored blobs).
+   * @returns All hashes with a stored metadata record.
+   */
+  async listHashes(): Promise<string[]> {
+    const keys = await this.adapter.list('blob-meta:')
+    return keys.map((k) => k.slice('blob-meta:'.length))
   }
 
   /**
@@ -403,6 +520,8 @@ export class BlobStorage {
 
   /**
    * Increment the reference count for an existing blob.
+   * Caller MUST hold the per-hash lock ({@link hashLocks}) — this is a raw
+   * read-modify-write with no serialization of its own.
    */
   private async incrementRefCount(hash: string): Promise<number> {
     const metadata = await this.getMetadata(hash)
@@ -417,6 +536,8 @@ export class BlobStorage {
 
   /**
    * Decrement the reference count for a blob (floored at zero).
+   * Caller MUST hold the per-hash lock ({@link hashLocks}) — this is a raw
+   * read-modify-write with no serialization of its own.
    */
   private async decrementRefCount(hash: string): Promise<number> {
     const metadata = await this.getMetadata(hash)

src/transaction/Transaction.ts

@@ -37,6 +37,24 @@ const DEFAULT_OPTIONS: Required<TransactionOptions> = {
   maxRollbackRetries: 3
 }
 
+/**
+ * The apply-phase budget for a batch of `opCount` operations.
+ *
+ * An explicit override wins untouched. Otherwise the budget SCALES with the
+ * batch: `max(30 000 ms, opCount × 2 000 ms)`. The per-op term is calibrated
+ * from field data — bulk imports on network-attached disks measure ~2 s per
+ * operation (each op pays canonical writes + fsync + index maintenance) — so
+ * a flat 30 s budget silently capped honest work at ~15 operations while
+ * looking generous for small batches. Scaling keeps small transacts
+ * fast-failing and gives bulk ones a budget proportional to the work they
+ * actually asked for; a trip still rolls back atomically and throws a
+ * retryable, fully-labeled TransactionTimeoutError.
+ */
+export function transactTimeoutBudget(opCount: number, override?: number): number {
+  if (override !== undefined) return override
+  return Math.max(30_000, opCount * 2_000)
+}
+
 /**
  * Transaction class
  */
@@ -98,49 +116,64 @@ export class Transaction implements TransactionContext {
     }
 
     try {
-      // Execute each operation in order
+      // Execute each operation in order. This loop is the sole rollback-guarded
+      // region: ANY error that escapes it — an operation failure OR a
+      // mid-flight timeout — is caught below and rolls back every operation
+      // applied so far, in reverse order. That single guarantee is the
+      // transaction's atomicity contract, and the generation-store commit path
+      // depends on it: its abort cleanup assumes a throw from here already
+      // restored the applied operations byte-identically (it only discards the
+      // uncommitted staging directory). A rollback per exit path — the previous
+      // design — let the timeout throw slip past rollback and strand the
+      // already-applied writes as torn, generation-less state in canonical
+      // storage.
       for (let i = 0; i < this.operations.length; i++) {
-        // Check timeout
+        // Budget check BEFORE starting the next operation. A trip here throws
+        // into the catch below and rolls back like any other failure — it must
+        // never bypass rollback.
         if (Date.now() - this.startTime > this.options.timeout) {
-          throw new TransactionTimeoutError(this.options.timeout, i)
+          throw new TransactionTimeoutError(this.options.timeout, i, {
+            elapsedMs: Date.now() - this.startTime,
+            totalOperations: this.operations.length,
+            operationName: this.operations[i]?.name
+          })
         }
 
         const operation = this.operations[i]
 
+        if (this.options.logging) {
+          prodLog.info(`[Transaction] Executing operation ${i}: ${operation.name || 'unnamed'}`)
+        }
+
+        let rollbackAction: RollbackAction | undefined
         try {
-          if (this.options.logging) {
-            prodLog.info(`[Transaction] Executing operation ${i}: ${operation.name || 'unnamed'}`)
-          }
-
-          // Execute operation
-          const rollbackAction = await operation.execute()
-
-          // Record rollback action (if provided)
-          if (rollbackAction) {
-            this.rollbackActions.push(rollbackAction)
-          }
-
+          rollbackAction = await operation.execute()
         } catch (error) {
-          // Operation failed - rollback and re-throw
-          const executionError = new TransactionExecutionError(
+          // Normalize an operation failure to a TransactionExecutionError; the
+          // outer catch performs the (single) rollback and surfaces it.
+          throw new TransactionExecutionError(
             `Operation ${i} failed: ${(error as Error).message}`,
             i,
             operation.name,
             error as Error
           )
+        }
 
-          await this.rollback(executionError)
-          throw executionError
+        // Record rollback action (if the operation provided one)
+        if (rollbackAction) {
+          this.rollbackActions.push(rollbackAction)
         }
       }
-
-      // All operations succeeded - commit
-      this.commit()
-
     } catch (error) {
-      // Error already handled in rollback
+      // Single rollback point: undo everything applied so far, in reverse
+      // order, then surface the original error. A rollback failure supersedes
+      // it (rollback() throws TransactionRollbackError wrapping this error).
+      await this.rollback(error as Error)
       throw error
     }
+
+    // All operations succeeded — commit.
+    this.commit()
   }
 
   /**
@@ -208,15 +241,22 @@ export class Transaction implements TransactionContext {
       }
     }
 
-    this.state = 'rolled_back'
+    // Honest terminal state: only claim 'rolled_back' when EVERY undo applied.
+    // If any undo failed, the store is not cleanly rolled back — mark
+    // 'inconsistent' so the commit orchestration reconciles rather than trusts
+    // a false 'rolled_back'. (Fixes the state half of the post-commit response
+    // lie: the record whose undo failed is still durable.)
+    this.state = rollbackErrors.length > 0 ? 'inconsistent' : 'rolled_back'
     this.endTime = Date.now()
 
     if (this.options.logging) {
       const duration = this.endTime - (this.startTime || this.endTime)
-      prodLog.info(`[Transaction] Rolled back in ${duration}ms`)
+      prodLog.info(`[Transaction] ${this.state === 'inconsistent' ? 'Rollback INCOMPLETE' : 'Rolled back'} in ${duration}ms`)
     }
 
-    // If rollback encountered errors, wrap them with original error
+    // If rollback encountered errors, wrap them with original error. The
+    // orchestration keys on `instanceof TransactionRollbackError` to know the
+    // undo did not fully apply and reconciliation is required.
     if (rollbackErrors.length > 0) {
       throw new TransactionRollbackError(
         `Transaction rollback encountered ${rollbackErrors.length} errors during cleanup`,

src/transaction/errors.ts

@@ -77,11 +77,24 @@ export class InvalidTransactionStateError extends TransactionError {
 export class TransactionTimeoutError extends TransactionError {
   constructor(
     timeoutMs: number,
-    operationIndex: number
+    operationIndex: number,
+    telemetry?: {
+      elapsedMs?: number
+      totalOperations?: number
+      operationName?: string
+    }
   ) {
+    const progress =
+      telemetry?.totalOperations !== undefined
+        ? `${operationIndex}/${telemetry.totalOperations}`
+        : String(operationIndex)
+    const name = telemetry?.operationName ? ` ('${telemetry.operationName}')` : ''
+    const elapsed =
+      telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
     super(
-      `Transaction timed out after ${timeoutMs}ms at operation ${operationIndex}`,
-      { timeoutMs, operationIndex }
+      `Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
+        `The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`,
+      { timeoutMs, operationIndex, ...telemetry }
     )
     this.name = 'TransactionTimeoutError'
   }

src/transaction/operations/IndexOperations.ts

@@ -176,14 +176,36 @@ export class RemoveFromMetadataIndexOperation implements Operation {
  * generation is reused for the rollback removal, so an add and its undo
  * reference one watermark in a provider's per-generation edge chain.
  */
+/**
+ * A verb's interned endpoint ints — eager (already resolved), or a thunk
+ * evaluated when the operation EXECUTES. The lazy form exists for
+ * `transact()` forward references: a relate whose endpoint is added in the
+ * SAME batch cannot resolve ints at plan time (the entity does not exist yet
+ * — a strict native id mapper rightly refuses to assign, and even a permissive
+ * one would leak the assignment if the batch is rejected at precommit).
+ * Deferring to execute time resolves after the batch's add operations have
+ * applied, mirroring how `generationFn` is already evaluated lazily.
+ */
+export type VerbEndpointInts =
+  | { sourceInt: bigint; targetInt: bigint }
+  | (() => { sourceInt: bigint; targetInt: bigint })
+
+/** Resolve a {@link VerbEndpointInts} at execution time. */
+function resolveEndpointInts(
+  endpoints: VerbEndpointInts
+): { sourceInt: bigint; targetInt: bigint } {
+  return typeof endpoints === 'function' ? endpoints() : endpoints
+}
+
 export class AddToGraphIndexOperation implements Operation {
   readonly name = 'AddToGraphIndex'
 
   /**
    * @param index - The graph-index provider (JS baseline or native).
    * @param verb - The verb to index (`sourceInt`/`targetInt` mirrored on it).
-   * @param sourceInt - The source entity's interned int.
-   * @param targetInt - The target entity's interned int.
+   * @param endpointInts - The endpoints' interned ints — eager, or a thunk
+   *   evaluated at execute time (REQUIRED for transact forward references;
+   *   see {@link VerbEndpointInts}).
    * @param generationFn - Resolves the commit generation to stamp this edge at,
    *   evaluated when the operation executes (see class note).
    * @param onVerbInt - Optional hook invoked with the interned verb int
@@ -192,17 +214,18 @@ export class AddToGraphIndexOperation implements Operation {
   constructor(
     private readonly index: GraphIndexProvider,
     private readonly verb: GraphVerb,
-    private readonly sourceInt: bigint,
-    private readonly targetInt: bigint,
+    private readonly endpointInts: VerbEndpointInts,
     private readonly generationFn: () => bigint,
     private readonly onVerbInt?: (verbInt: bigint) => void
   ) {}
 
   async execute(): Promise<RollbackAction> {
     // Stamp this edge at the in-flight commit generation; reuse it for the
-    // rollback so add + undo reference the same watermark.
+    // rollback so add + undo reference the same watermark. Endpoint ints
+    // resolve HERE — after any same-batch adds have applied.
     const generation = this.generationFn()
-    const verbInt = await this.index.addVerb(this.verb, this.sourceInt, this.targetInt, generation)
+    const { sourceInt, targetInt } = resolveEndpointInts(this.endpointInts)
+    const verbInt = await this.index.addVerb(this.verb, sourceInt, targetInt, generation)
     this.onVerbInt?.(verbInt)
 
     // Return rollback action
@@ -231,28 +254,32 @@ export class RemoveFromGraphIndexOperation implements Operation {
   /**
    * @param index - The graph-index provider (JS baseline or native).
    * @param verb - The verb being removed (required for rollback re-add).
-   * @param sourceInt - The source entity's interned int (rollback re-add).
-   * @param targetInt - The target entity's interned int (rollback re-add).
+   * @param endpointInts - The endpoints' interned ints for the rollback
+   *   re-add — eager, or a thunk evaluated at execute time (required when the
+   *   verb or its endpoints were created in the SAME transact batch; see
+   *   {@link VerbEndpointInts}).
    * @param generationFn - Resolves the commit generation for this removal,
    *   evaluated when the operation executes.
    */
   constructor(
     private readonly index: GraphIndexProvider,
     private readonly verb: GraphVerb, // Required for rollback
-    private readonly sourceInt: bigint,
-    private readonly targetInt: bigint,
+    private readonly endpointInts: VerbEndpointInts,
     private readonly generationFn: () => bigint
   ) {}
 
   async execute(): Promise<RollbackAction> {
     // Resolve the removal generation once; reuse it for the rollback re-add.
+    // Endpoint ints resolve HERE (after any same-batch adds applied) and are
+    // captured for the rollback, whose re-add must use the same mappings.
     const generation = this.generationFn()
+    const { sourceInt, targetInt } = resolveEndpointInts(this.endpointInts)
     await this.index.removeVerb(this.verb.id, generation)
 
     // Return rollback action
     return async () => {
       // Re-add verb with original data
-      await this.index.addVerb(this.verb, this.sourceInt, this.targetInt, generation)
+      await this.index.addVerb(this.verb, sourceInt, targetInt, generation)
     }
   }
 }

src/transaction/operations/StorageOperations.ts

@@ -104,35 +104,72 @@ export class SaveNounOperation implements Operation {
 }
 
 /**
- * Delete noun metadata with rollback support
+ * Delete a noun — FULL canonical removal, with rollback support.
+ *
+ * Despite the historical name, this removes the WHOLE entity: both canonical
+ * legs (metadata + vector) AND the entity's container. Previously it deleted
+ * only the metadata leg via `deleteNounMetadata`, leaving the canonical
+ * `vectors.json` leg and the `<id>/` directory orphaned on disk — a "ghost"
+ * that reads as absent (getNoun needs both legs) yet inflates the enumerated
+ * count and can never be told apart from a damage scar. Routing through
+ * `storage.deleteNoun` removes both legs + the container in one place.
+ *
+ * Immutability is preserved: this cleans only the live-HEAD projection; the
+ * generation store retains the before-image so `asOf()` still reconstructs the
+ * deleted entity until retention expires.
  *
  * Rollback strategy:
- * - Restore deleted metadata
+ * - Restore BOTH legs from the before-image (vector leg raw, metadata leg via
+ *   the count-aware save so deleteNoun()'s decrement is reversed).
  */
 export class DeleteNounMetadataOperation implements Operation {
-  readonly name = 'DeleteNounMetadata'
+  readonly name = 'DeleteNoun'
 
   constructor(
     private readonly storage: StorageAdapter,
-    private readonly id: string
+    private readonly id: string,
+    /**
+     * OPTIONAL already-known metadata of the entity being removed (the caller's
+     * pre-delete read). Removal must never REQUIRE re-reading the thing being
+     * removed: if the reads here return null (replace race, or a ghost left by
+     * an earlier version), the count decrement downstream falls back to this
+     * record instead of being silently skipped — the skip minted permanent
+     * counter inflation (adds counted, paired removals not decremented).
+     */
+    private readonly priorMetadata?: NounMetadata | null
   ) {}
 
   async execute(): Promise<RollbackAction> {
-    // Get metadata before deletion (for rollback)
-    const previousMetadata = await this.storage.getNounMetadata(this.id)
+    // Capture the FULL before-image (both legs) so the undo restores the whole
+    // entity — a metadata-only rollback would leave the vector leg unrestored.
+    // A null metadata read falls back to the caller's pre-delete read.
+    const previousNoun = await this.storage.getNoun(this.id)
+    const previousMetadata = (await this.storage.getNounMetadata(this.id)) ?? this.priorMetadata ?? null
 
-    if (!previousMetadata) {
+    if (!previousNoun && !previousMetadata) {
       // Nothing to delete - no rollback needed
       return async () => {}
     }
 
-    // Delete metadata
-    await this.storage.deleteNounMetadata(this.id)
+    // Full removal: both canonical legs + the entity container + count decrement
+    // (the prior record keeps the decrement honest on a null canonical read).
+    await this.storage.deleteNoun(this.id, previousMetadata)
 
     // Return rollback action
     return async () => {
-      // Restore deleted metadata
-      await this.storage.saveNounMetadata(this.id, previousMetadata)
+      // Restore the vector leg, then the metadata leg through the count-aware
+      // save so deleteNoun()'s decrement is reversed.
+      if (previousNoun) {
+        await this.storage.saveNoun({
+          id: previousNoun.id,
+          vector: previousNoun.vector,
+          connections: previousNoun.connections || new Map(),
+          level: previousNoun.level || 0
+        })
+      }
+      if (previousMetadata) {
+        await this.storage.saveNounMetadata(this.id, previousMetadata)
+      }
     }
   }
 }
@@ -242,9 +279,9 @@ export class DeleteVerbMetadataOperation implements Operation {
       return async () => {}
     }
 
-    // Delete verb (metadata + vector)
-    // Note: StorageAdapter has deleteVerb but not deleteVerbMetadata
-    await this.storage.deleteVerb(this.id)
+    // Delete verb (metadata + vector). The pre-read rides along so the count
+    // decrement never depends on re-reading the record being removed.
+    await this.storage.deleteVerb(this.id, previousMetadata)
 
     // Return rollback action
     return async () => {

src/transaction/types.ts

@@ -14,6 +14,10 @@ export type TransactionState =
   | 'committed'    // Successfully committed
   | 'rolling_back' // Rolling back due to failure
   | 'rolled_back'  // Successfully rolled back
+  | 'inconsistent' // Rollback ran but ≥1 undo could not be applied — the store
+                   // is NOT cleanly rolled back; the commit orchestration must
+                   // reconcile (adopt-forward or fail-loud). Never claim
+                   // 'rolled_back' when an undo failed.
 
 /**
  * Rollback action - undoes an operation

src/types/brainy.types.ts

@@ -1737,7 +1737,10 @@ export interface BrainyConfig {
    * Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
    * `remove`/`relate`) produces an immutable generation record-set serving
    * historical reads (`asOf()`, pinned `Db` values). Without compaction those
-   * accumulate, so Brainy **auto-compacts on every `flush()` and `close()`**.
+   * accumulate, so Brainy **auto-compacts at `close()`** (time-bounded per
+   * pass; 8.9.0 removed compaction from `flush()` — flush is durability work
+   * and never pays maintenance costs). A long-lived writer that never closes
+   * accumulates history until its next explicit `compactHistory()` call.
    * Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
    *
    * Modes:
@@ -1754,7 +1757,7 @@ export interface BrainyConfig {
    *   the oldest unpinned generations while ANY supplied cap is exceeded
    *   (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
    *
-   * `autoCompact: false` disables the automatic flush/close compaction (manage
+   * `autoCompact: false` disables the automatic close() compaction (manage
    * manually via `brain.compactHistory()`). `budgetBytes` is the settable
    * adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
    * Long-term archives belong in `db.persist(path)` snapshots, which compaction
@@ -1772,7 +1775,7 @@ export interface BrainyConfig {
         maxBytes?: number
         /** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
         budgetBytes?: number
-        /** Run compaction automatically on flush()/close() (default: true). */
+        /** Run compaction automatically at close() (default: true; 8.9.0 — flush() never compacts). */
         autoCompact?: boolean
       }
 

src/utils/crc32c.ts

@@ -0,0 +1,43 @@
+/**
+ * @module utils/crc32c
+ * @description CRC-32C (Castagnoli, polynomial 0x1EDC6F41, reflected 0x82F63B78)
+ * — the storage-industry frame checksum (ext4, iSCSI, SCTP, LSM segment files).
+ * Used to frame generation-fact segments: every appended record carries the
+ * CRC-32C of its payload, so a torn tail (crash mid-append) or bit rot is
+ * DETECTED at scan time and never silently read as data.
+ *
+ * Table-driven, dependency-free reference implementation. Native providers may
+ * substitute a hardware-accelerated (SSE4.2 / ARMv8 CRC) implementation — the
+ * polynomial is the contract, byte-identical results required.
+ */
+
+/** The 256-entry lookup table for the reflected CRC-32C polynomial. */
+const TABLE: Uint32Array = (() => {
+  const table = new Uint32Array(256)
+  for (let n = 0; n < 256; n++) {
+    let c = n
+    for (let k = 0; k < 8; k++) {
+      c = c & 1 ? 0x82f63b78 ^ (c >>> 1) : c >>> 1
+    }
+    table[n] = c >>> 0
+  }
+  return table
+})()
+
+/**
+ * Compute the CRC-32C checksum of a byte buffer.
+ *
+ * Known-answer vectors (RFC 3720 appendix / the standard test suite):
+ * - ASCII "123456789" → 0xE3069283
+ * - 32 zero bytes → 0x8A9136AA
+ *
+ * @param bytes - The payload to checksum.
+ * @returns The CRC-32C as an unsigned 32-bit integer.
+ */
+export function crc32c(bytes: Uint8Array): number {
+  let crc = 0xffffffff
+  for (let i = 0; i < bytes.length; i++) {
+    crc = TABLE[(crc ^ bytes[i]) & 0xff] ^ (crc >>> 8)
+  }
+  return (crc ^ 0xffffffff) >>> 0
+}

src/utils/errorClassification.ts

@@ -0,0 +1,37 @@
+/**
+ * @module utils/errorClassification
+ * @description Shared classification of caught errors into "genuine absence" vs
+ * "real fault" — the antidote to blind `catch { return null }` handlers that
+ * cannot tell ENOENT (the object is legitimately not on disk) from EIO / EACCES
+ * / EMFILE / … (a transient or permission fault on data that IS on disk).
+ * Masking a fault as absence yields wrong results (a present record read as
+ * "not found") or a needless rebuild. Mandate: loud errors, never quiet losses.
+ */
+
+/**
+ * The error `code`s that denote GENUINE absence of a file/object. Only `ENOENT`
+ * ("no such file or directory") qualifies — on every platform Node maps a
+ * missing file/directory to ENOENT, and no other errno means "simply not
+ * there". Every other errno (EIO, EACCES, EPERM, EMFILE, ENFILE, EBUSY,
+ * ENOTDIR, EISDIR, ELOOP) is a real fault and must propagate, as must any error
+ * without an errno `code` (parse/decompress failures, generic Errors).
+ * `ENOTDIR`/`EISDIR` are deliberately faults: a path component of the wrong
+ * type is corruption, not benign absence. Named constant so a future
+ * genuine-absence code can be added in one reviewed place.
+ */
+const ABSENCE_CODES: ReadonlySet<string> = new Set(['ENOENT'])
+
+/**
+ * @description True IFF `e` represents genuine absence (an ENOENT-class errno),
+ * for which returning `null`/`[]`/`undefined` is the correct answer. Returns
+ * `false` for every real fault, so the canonical call site is:
+ * `catch (e) { if (isAbsentError(e)) return null; throw e }`.
+ *
+ * @param e - The caught value (typed `unknown`; non-objects are never absence).
+ * @returns Whether the error means "the thing is simply not there".
+ */
+export function isAbsentError(e: unknown): boolean {
+  if (e === null || typeof e !== 'object') return false
+  const code = (e as { code?: unknown }).code
+  return typeof code === 'string' && ABSENCE_CODES.has(code)
+}

src/utils/indexReadiness.ts

@@ -0,0 +1,38 @@
+/**
+ * @module indexReadiness
+ * @description The single honest-readiness classifier shared by the vector,
+ * graph and metadata index sites. It exists to kill "Pattern A" — the dishonest
+ * readiness proxy where `size() > 0` / `isInitialized` is treated as "this index
+ * actually serves queries." A cold native index that loaded its COUNT but not its
+ * SERVING structure passes those proxies and silently returns `[]`.
+ *
+ * This classifier reads ONLY the provider's OPTIONAL, honest `isReady()` signal
+ * (see {@link import('../plugin.js').VectorIndexProvider.isReady},
+ * {@link import('../plugin.js').GraphIndexProvider.isReady},
+ * {@link import('../plugin.js').MetadataIndexProvider.isReady}). It NEVER inspects
+ * `size()` or `isInitialized`. When `isReady()` is absent, callers must fall back
+ * to a KNOWN-ITEM PROBE (a real search/lookup that must return a known-present
+ * datum) before trusting an empty result — never a `size()` proxy.
+ */
+
+/** A provider that MAY expose the honest cold-load readiness signal. */
+export interface MaybeReadyProvider {
+  isReady?: () => boolean
+}
+
+/** Three-valued honest-readiness verdict. */
+export type IndexReadiness = 'ready' | 'not-ready' | 'unknown'
+
+/**
+ * @description Classify an index provider's honest readiness.
+ * @param provider - Any index provider (vector / graph / metadata) or `null`.
+ * @returns
+ *  - `'ready'`     when `isReady() === true`  (serving structure loaded — trust it);
+ *  - `'not-ready'` when `isReady() === false` (count/manifest loaded, NOT serving — rebuild);
+ *  - `'unknown'`   when the provider exposes no `isReady()` (caller must probe / keep the JS heuristic).
+ */
+export function assessIndexReadiness(provider: unknown): IndexReadiness {
+  const p = provider as MaybeReadyProvider | null | undefined
+  if (p == null || typeof p.isReady !== 'function') return 'unknown'
+  return p.isReady() ? 'ready' : 'not-ready'
+}

src/utils/metadataIndex.ts

@@ -1867,9 +1867,26 @@ export class MetadataIndexManager implements MetadataIndexProvider {
     // not once per AND-clause inside it.
     const unindexedFields: string[] = []
 
-    for (const [field, condition] of Object.entries(filter)) {
+    for (const [rawField, condition] of Object.entries(filter)) {
       // Skip logical operators
-      if (field === 'allOf' || field === 'anyOf' || field === 'not') continue
+      if (rawField === 'allOf' || rawField === 'anyOf' || rawField === 'not') continue
+
+      // Metadata is FLATTENED at index time (metadata.entry.title indexes as
+      // entry.title), so a `metadata.`-prefixed where key is almost always
+      // the caller spelling the STORAGE shape rather than the index shape.
+      // Accept both spellings: when the key as spelled is unindexed but its
+      // stripped spelling is, query the stripped one. A literal nested
+      // custom key named `metadata` still wins when indexed as spelled
+      // (checked first), so that rare shape keeps working.
+      let field = rawField
+      if (
+        rawField.startsWith('metadata.') &&
+        this.columnStore &&
+        !this.columnStore.hasField(rawField) &&
+        this.columnStore.hasField(rawField.slice('metadata.'.length))
+      ) {
+        field = rawField.slice('metadata.'.length)
+      }
 
       let fieldResults: string[] = []
 

src/utils/osLimits.ts

@@ -0,0 +1,141 @@
+/**
+ * @module utils/osLimits
+ * @description Detect-and-warn for OS resource limits that bite at POOL scale.
+ *
+ * A single brain rarely notices them, but a pool of brains — especially with a
+ * native accelerator memory-mapping many index files per brain — consumes file
+ * descriptors and memory mappings multiplicatively. On stock Linux defaults
+ * (RLIMIT_NOFILE soft 1024, vm.max_map_count 65530) the failure arrives as
+ * EMFILE or a failed mmap deep inside an index open, long after the real cause
+ * (the limit) stopped being visible. This module reads the limits at open and
+ * WARNS ONCE per process with the exact raise commands, so the operator learns
+ * the fix before the incident instead of from it.
+ *
+ * Read-only and Linux-only by construction: both sources are `/proc` files.
+ * On platforms where they are absent the check reports nulls and stays silent —
+ * no limit read means no claim made, never a guessed warning.
+ */
+
+import * as fs from 'node:fs'
+import { prodLog } from './logger.js'
+
+/** Soft-NOFILE floor below which pool-scale use is at EMFILE risk. */
+export const NOFILE_POOL_FLOOR = 65536
+
+/** vm.max_map_count floor below which mmap-heavy native indexes are at risk. */
+export const MAX_MAP_COUNT_POOL_FLOOR = 262144
+
+export interface OsLimitsReport {
+  /** RLIMIT_NOFILE soft limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileSoft: number | null
+  /** RLIMIT_NOFILE hard limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileHard: number | null
+  /** vm.max_map_count (null when unreadable). */
+  maxMapCount: number | null
+  /** Human-actionable warnings for limits below the pool floors. Empty = fine. */
+  warnings: string[]
+}
+
+/**
+ * Parse the `Max open files` row of a `/proc/<pid>/limits` document into
+ * soft/hard values. Returns nulls when the row is absent or malformed.
+ */
+export function parseProcLimits(content: string): { soft: number | null; hard: number | null } {
+  const line = content.split('\n').find((l) => l.startsWith('Max open files'))
+  if (!line) return { soft: null, hard: null }
+  const m = line.match(/^Max open files\s+(\S+)\s+(\S+)/)
+  if (!m) return { soft: null, hard: null }
+  const parse = (v: string): number | null => {
+    if (v === 'unlimited') return Infinity
+    const n = Number.parseInt(v, 10)
+    return Number.isNaN(n) ? null : n
+  }
+  return { soft: parse(m[1]), hard: parse(m[2]) }
+}
+
+/**
+ * Assess readable limits against the pool floors. Pure — feed it any values.
+ * A null (unreadable) limit produces NO warning: no measurement, no claim.
+ */
+export function assessOsLimits(limits: {
+  nofileSoft: number | null
+  nofileHard: number | null
+  maxMapCount: number | null
+}): string[] {
+  const warnings: string[] = []
+
+  if (limits.nofileSoft !== null && limits.nofileSoft < NOFILE_POOL_FLOOR) {
+    const hardNote =
+      limits.nofileHard !== null && limits.nofileHard >= NOFILE_POOL_FLOOR
+        ? ` (the hard limit ${limits.nofileHard === Infinity ? 'unlimited' : limits.nofileHard} already allows it — raise the soft limit only)`
+        : ''
+    warnings.push(
+      `RLIMIT_NOFILE soft limit is ${limits.nofileSoft} — below the ${NOFILE_POOL_FLOOR} recommended ` +
+        `for pool-scale use (a pool of brains with a native accelerator opens many index files per brain; ` +
+        `the failure mode is EMFILE deep inside an index open). Raise with \`ulimit -n ${NOFILE_POOL_FLOOR}\` ` +
+        `or LimitNOFILE=${NOFILE_POOL_FLOOR} in the service unit${hardNote}.`
+    )
+  }
+
+  if (limits.maxMapCount !== null && limits.maxMapCount < MAX_MAP_COUNT_POOL_FLOOR) {
+    warnings.push(
+      `vm.max_map_count is ${limits.maxMapCount} — below the ${MAX_MAP_COUNT_POOL_FLOOR} recommended ` +
+        `for mmap-heavy native indexes at pool scale (each mapped index segment consumes map entries; ` +
+        `the failure mode is a failed mmap mid-heal). Raise with ` +
+        `\`sysctl -w vm.max_map_count=${MAX_MAP_COUNT_POOL_FLOOR}\` (persist in /etc/sysctl.d/).`
+    )
+  }
+
+  return warnings
+}
+
+/**
+ * Read the limits from /proc and assess them. `readFile` is injectable for
+ * tests; absent/unreadable sources yield nulls (and therefore no warnings).
+ */
+export async function checkOsLimits(
+  readFile: (path: string) => Promise<string> = async (p) => fs.promises.readFile(p, 'utf-8')
+): Promise<OsLimitsReport> {
+  let nofileSoft: number | null = null
+  let nofileHard: number | null = null
+  let maxMapCount: number | null = null
+
+  try {
+    const parsed = parseProcLimits(await readFile('/proc/self/limits'))
+    nofileSoft = parsed.soft
+    nofileHard = parsed.hard
+  } catch {
+    // Not Linux (or /proc unavailable) — no measurement, no claim.
+  }
+
+  try {
+    const raw = (await readFile('/proc/sys/vm/max_map_count')).trim()
+    const n = Number.parseInt(raw, 10)
+    maxMapCount = Number.isNaN(n) ? null : n
+  } catch {
+    // Not Linux — same rule.
+  }
+
+  const warnings = assessOsLimits({ nofileSoft, nofileHard, maxMapCount })
+  return { nofileSoft, nofileHard, maxMapCount, warnings }
+}
+
+/** Once-per-process latch so a brain pool warns once, not once per brain. */
+let osLimitsWarned = false
+
+/**
+ * Run the check and warn (once per process) about limits below the pool
+ * floors. Called from brain open; safe everywhere (silent off-Linux).
+ */
+export async function warnOnLowOsLimits(): Promise<void> {
+  if (osLimitsWarned) return
+  osLimitsWarned = true
+  try {
+    const report = await checkOsLimits()
+    for (const warning of report.warnings) {
+      prodLog.warn(`[Brainy] OS limit check: ${warning}`)
+    }
+  } catch {
+    // The check must never affect open — measurement-only.
+  }
+}

src/utils/paramValidation.ts

@@ -209,8 +209,10 @@ export interface ValidationConfigOptions {
 }
 
 /**
- * Auto-configured limits based on system resources
- * These adapt to available memory and observed performance
+ * Auto-configured limits based on system resources.
+ * Derived from memory (explicit overrides > reserved memory > container limit >
+ * free memory). Query timing is recorded for diagnostics only — it never
+ * changes the cap (see `recordQuery`).
  */
 export class ValidationConfig {
   private static instance: ValidationConfig | null = null
@@ -323,24 +325,23 @@ export class ValidationConfig {
   }
 
   /**
-   * Learn from actual usage to adjust limits
+   * Record query timing for diagnostics. Telemetry ONLY — never mutates the cap.
+   *
+   * `maxLimit` is a MEMORY-protection bound; query duration says nothing about
+   * memory-per-result, so duration must never drive it. An earlier version
+   * "learned" here: while the lifetime-average query time exceeded 1s it shrank
+   * `maxLimit` by 20% per recorded query down to a floor of 1000 — below the
+   * documented `MIN_AUTO_QUERY_LIMIT` (10 000) and with no recovery once slow
+   * samples poisoned the cumulative average. On a production host a burst of
+   * slow aggregate queries silently strangled every consumer's `find()` to
+   * 1000 while the error message blamed "available free memory" — exactly the
+   * silent throttling this module's own contract forbids. The cap now comes
+   * from its construction-time basis (or explicit overrides) alone.
    */
   recordQuery(duration: number, resultCount: number) {
+    void resultCount
     this.queryCount++
     this.avgQueryTime = (this.avgQueryTime * (this.queryCount - 1) + duration) / this.queryCount
-
-    // Only auto-adjust if not using explicit overrides
-    if (this.limitBasis !== 'override') {
-      // If queries are consistently fast with large results, increase limits
-      if (this.avgQueryTime < 100 && resultCount > this.maxLimit * 0.8) {
-        this.maxLimit = Math.min(this.maxLimit * 1.5, 100000)
-      }
-
-      // If queries are slow, reduce limits
-      if (this.avgQueryTime > 1000) {
-        this.maxLimit = Math.max(this.maxLimit * 0.8, 1000)
-      }
-    }
   }
 }
 

src/vfs/VirtualFileSystem.ts

@@ -36,6 +36,7 @@ import {
   VFSErrorCode,
   WriteOptions,
   ReadOptions,
+  FileVersion,
   MkdirOptions,
   ReaddirOptions,
   CopyOptions,
@@ -406,6 +407,14 @@ export class VirtualFileSystem implements IVirtualFileSystem {
   async readFile(path: string, options?: ReadOptions): Promise<Buffer> {
     await this.ensureInitialized()
 
+    // Temporal read: the file's exact bytes as of a past generation or
+    // instant. Materializes the entity from the generation history — content
+    // blobs referenced by any in-window generation are retention-protected,
+    // so the bytes are guaranteed present. Bypasses the content cache.
+    if (options?.asOf !== undefined) {
+      return this.readFileAt(path, options.asOf, options)
+    }
+
     // Check cache first
     if (options?.cache !== false && this.contentCache.has(path)) {
       const cached = this.contentCache.get(path)!
@@ -472,6 +481,122 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     }
   }
 
+  /**
+   * @description The temporal read behind `readFile(path, { asOf })`: resolve
+   * the path's CURRENT entity, materialize its state at the target
+   * generation/instant from the Model-B history, and read that version's
+   * content blob (retention-protected, so present for every in-window
+   * generation). The pinned view is always released so compaction is never
+   * blocked by a read.
+   * @param path - The file path (resolved against the live tree).
+   * @param asOf - A generation number or wall-clock `Date`.
+   * @param options - `encoding` applies; caching does not (never cached).
+   * @returns The exact bytes the file held at that generation.
+   * @throws VFSError ENOENT when the file did not exist at that generation;
+   *   the generation store's compacted-generation error when `asOf` is past
+   *   the retention window's horizon.
+   */
+  private async readFileAt(
+    path: string,
+    asOf: number | Date,
+    options?: ReadOptions
+  ): Promise<Buffer> {
+    const entityId = await this.pathResolver.resolve(path)
+    const db = await this.brain.asOf(asOf)
+    try {
+      const entity = await db.get(entityId)
+      if (!entity) {
+        throw new VFSError(
+          VFSErrorCode.ENOENT,
+          `File did not exist as of ${String(asOf)}: ${path}`,
+          path,
+          'readFile'
+        )
+      }
+      const storage = (entity.metadata as Record<string, any> | undefined)?.storage
+      if (storage?.type !== 'blob' || typeof storage.hash !== 'string') {
+        throw new VFSError(
+          VFSErrorCode.EIO,
+          `File had no blob storage as of ${String(asOf)}: ${path}`,
+          path,
+          'readFile'
+        )
+      }
+      const content = await this.blobStorage.read(storage.hash)
+      if (options?.encoding) {
+        return Buffer.from(content.toString(options.encoding))
+      }
+      return content
+    } finally {
+      await db.release()
+    }
+  }
+
+  /**
+   * @description A file's version history within the retention window: one
+   * entry per generation that wrote the file, ascending — the newest entry is
+   * the current state. Pair with `readFile(path, { asOf: entry.generation })`
+   * to fetch any version's exact bytes, and restore by writing those bytes
+   * back (a NEW write — history is never rewritten). Bounded by the retention
+   * window: versions whose generations were compacted are not listed.
+   * @param path - The file path (resolved against the live tree).
+   * @returns The versions, oldest first.
+   * @example
+   * const versions = await brain.vfs.history('/pages/home.json')
+   * const before = await brain.vfs.readFile('/pages/home.json', {
+   *   asOf: versions[versions.length - 2].generation
+   * })
+   */
+  async history(path: string): Promise<FileVersion[]> {
+    await this.ensureInitialized()
+    const entityId = await this.pathResolver.resolve(path)
+
+    // Boundary note: reaches Brainy's internal generation store (the same
+    // bracket-access precedent as the blobStorage getter) — the per-id
+    // generation chain and horizon are not on the public Brainy surface.
+    const generationStore = (this.brain as unknown as {
+      generationStore: {
+        horizon(): number
+        generation(): number
+        generationsTouching(
+          kind: 'noun' | 'verb',
+          id: string,
+          fromGen: number,
+          toGen: number
+        ): Promise<number[]>
+      }
+    }).generationStore
+
+    const gens = await generationStore.generationsTouching(
+      'noun',
+      entityId,
+      generationStore.horizon(),
+      generationStore.generation()
+    )
+
+    const versions: FileVersion[] = []
+    for (const gen of gens) {
+      const db = await this.brain.asOf(gen)
+      try {
+        const entity = await db.get(entityId)
+        const meta = entity?.metadata as Record<string, any> | undefined
+        const storage = meta?.storage
+        if (storage?.type === 'blob' && typeof storage.hash === 'string') {
+          versions.push({
+            generation: gen,
+            timestamp: db.timestamp,
+            hash: storage.hash,
+            size: typeof meta?.size === 'number' ? meta.size : (storage.size ?? 0),
+            ...(typeof meta?.mimeType === 'string' && { mimeType: meta.mimeType })
+          })
+        }
+      } finally {
+        await db.release()
+      }
+    }
+    return versions
+  }
+
   /**
    * Write a file
    */
@@ -493,14 +618,18 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     // Ensure parent directory exists
     const parentId = await this.ensureDirectory(parentPath)
 
-    // Check if file already exists
+    // Check if file already exists (and capture its current content hash so
+    // the overwrite can release the superseded live reference afterwards).
     let existingId: string | null = null
+    let previousHash: string | undefined
     try {
       existingId = await this.pathResolver.resolve(path, { cache: false })
       // Verify the entity still exists in the brain
       const existing = await this.brain.get(existingId)
       if (!existing) {
         existingId = null  // Entity was deleted but cache wasn't cleared
+      } else if (existing.metadata?.storage?.type === 'blob') {
+        previousHash = existing.metadata.storage.hash as string | undefined
       }
     } catch (err) {
       // File doesn't exist, which is fine
@@ -551,14 +680,32 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       Object.assign(metadata, await this.extractMetadata(buffer, mimeType))
     }
 
+    // For embedding: use text content, for storage: use raw data. Computed
+    // for BOTH branches — an overwrite must refresh the entity's `data` (and
+    // therefore its embedding), or semantic search and any `data` read keep
+    // serving the FIRST version's text forever.
+    const embeddingData = mimeDetector.isTextFile(mimeType)
+      ? buffer.toString('utf-8')
+      : `File: ${name} (${mimeType}, ${buffer.length} bytes)`
+
     if (existingId) {
-      // Update existing file
-      // No entity.data - content is in BlobStorage
+      // Update existing file — content bytes live in BlobStorage; `data`
+      // carries the embedding text and MUST track the new content.
       await this.brain.update({
         id: existingId,
+        data: embeddingData,
         metadata
       })
 
+      // The update committed: drop the superseded live reference. When the
+      // content is unchanged this cancels the dedup increment the write()
+      // above just made (net: one live reference per referencing file); when
+      // it changed, the old bytes stay retention-protected for asOf reads
+      // via the update's own generation record.
+      if (previousHash) {
+        await this.blobStorage.release(previousHash)
+      }
+
       // Ensure Contains relationship exists (fix for missing relationships)
       const existingRelations = await this.brain.related({
         from: parentId,
@@ -578,9 +725,6 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       }
     } else {
       // Create new file entity
-      // For embedding: use text content, for storage: use raw data
-      const embeddingData = mimeDetector.isTextFile(mimeType) ? buffer.toString('utf-8') : `File: ${name} (${mimeType}, ${buffer.length} bytes)`
-
       const entity = await this.brain.add({
         data: embeddingData,  // Always provide string for embeddings
         type: this.getFileNounType(mimeType),
@@ -649,14 +793,21 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       throw new VFSError(VFSErrorCode.EISDIR, `Is a directory: ${path}`, path, 'unlink')
     }
 
-    // Delete blob from BlobStorage (decrements ref count)
-    if (entity.metadata.storage?.type === 'blob') {
-      await this.blobStorage.delete(entity.metadata.storage.hash)
-    }
-
-    // Delete the entity
+    // Delete the entity FIRST, then drop its live blob reference — never
+    // release a reference while the referencing entity might survive (a
+    // failed remove must not leave a live file whose bytes compaction could
+    // later reclaim).
     await this.brain.remove(entityId)
 
+    // Drop the file's LIVE reference to its content blob. The bytes are NOT
+    // deleted — the remove's own generation record references this hash, so
+    // `readFile(path, { asOf })` keeps serving it inside the retention
+    // window; history compaction reclaims the bytes when the last referencing
+    // generation is reclaimed.
+    if (entity.metadata.storage?.type === 'blob') {
+      await this.blobStorage.release(entity.metadata.storage.hash)
+    }
+
     // Invalidate caches
     this.pathResolver.invalidatePath(path)
     this.invalidateCaches(path)
@@ -1032,23 +1183,26 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       // Phase 1: Gather all descendants in ONE batch fetch
       const descendants = await this.gatherDescendants(entityId, Infinity)
 
-      // Phase 2: Parallel blob cleanup (chunked to avoid overwhelming storage)
-      // Blob deletion is reference-counted, so safe to call for all files
+      // Phase 2: Batch delete all entities (including root directory) FIRST —
+      // never release a blob reference while its referencing entity might
+      // survive a failed delete (see unlink's ordering note).
+      const allIds = [...descendants.map(d => d.id), entityId]
+      await this.brain.removeMany({ ids: allIds, continueOnError: false })
+
+      // Phase 3: Drop the deleted files' LIVE blob references (chunked).
+      // Live-reference drops only — the bytes stay for in-window asOf reads
+      // and are reclaimed by history compaction (see unlink's note).
       const blobFiles = descendants.filter(d =>
         d.metadata.vfsType === 'file' && d.metadata.storage?.type === 'blob'
       )
 
-      const BLOB_CHUNK_SIZE = 20  // Parallel delete 20 blobs at a time
+      const BLOB_CHUNK_SIZE = 20  // Parallel release 20 blob references at a time
       for (let i = 0; i < blobFiles.length; i += BLOB_CHUNK_SIZE) {
         const chunk = blobFiles.slice(i, i + BLOB_CHUNK_SIZE)
         await Promise.all(chunk.map(f =>
-          this.blobStorage.delete(f.metadata.storage!.hash)
+          this.blobStorage.release(f.metadata.storage!.hash)
         ))
       }
-
-      // Phase 3: Batch delete all entities (including root directory)
-      const allIds = [...descendants.map(d => d.id), entityId]
-      await this.brain.removeMany({ ids: allIds, continueOnError: false })
     } else {
       // No children or not recursive - just delete the directory entity
       await this.brain.remove(entityId)
@@ -1800,15 +1954,27 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     const newParentPath = this.getParentPath(newPath)
 
     if (oldParentPath !== newParentPath) {
-      // Remove from old parent
+      // Remove the OLD parent's containment edge(s) — by edge id, resolved from
+      // the graph's own adjacency (the removal law: a removal never requires
+      // reading the thing being removed). This step used to be skipped as "not
+      // critical", which left the moved entity a child of BOTH directories:
+      // readdir(oldDir) kept listing it, re-creating the old path showed the
+      // name twice, and tree-walking consumers saw the file in two places.
       if (oldParentPath) {
         const oldParentId = await this.pathResolver.resolve(oldParentPath)
-        // unrelate takes the relation ID, not params - need to find and remove relation
-        // For now, skip unrelate as it's not critical for rename
+        const staleEdges = await this.brain.related({
+          from: oldParentId,
+          to: entityId,
+          type: VerbType.Contains
+        })
+        for (const edge of staleEdges) {
+          await this.brain.unrelate(edge.id)
+        }
       }
 
-      // Add to new parent
-      if (newParentPath && newParentPath !== '/') {
+      // Add to the new parent. The root ('/') is a REAL parent — skipping it
+      // orphaned a move-to-root out of readdir('/') entirely.
+      if (newParentPath) {
         const newParentId = await this.pathResolver.resolve(newParentPath)
         await this.brain.relate({
           from: newParentId,
@@ -1849,6 +2015,95 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     this.triggerWatchers(newPath, 'rename')
   }
 
+  /**
+   * Reconcile every VFS entity's containment edges against its canonical
+   * `metadata.path` — the path is the truth (maintained by write/rename); the
+   * `Contains` edges are a projection of it. Heals the "cosmetic ghost" class
+   * left by pre-fix renames that added the new parent's edge without removing
+   * the old one (an entity listed in TWO directories; a re-created old path
+   * showing its name twice), plus duplicate edges from the same parent left by
+   * concurrent writers.
+   *
+   * CONSERVATIVE by design: only VFS containment edges (subtype
+   * `'vfs-contains'` or `metadata.isVFS`) are ever touched — a user's own
+   * knowledge-graph `Contains` edge between the same entities is never
+   * removed. An entity whose expected parent path has no entity is logged
+   * loudly and left alone (never orphaned further). Operator-invoked via
+   * `brain.repairIndex()`.
+   *
+   * The canonical pagination walk (not an index query) is deliberate: this is
+   * a repair op — the projections are the thing under suspicion, so the walk
+   * reads the source of truth.
+   *
+   * @returns Counts of stale edges removed and missing expected edges restored.
+   */
+  async repairContainment(): Promise<{ removed: number; restored: number }> {
+    await this.ensureInitialized()
+
+    // Pass 1: canonical walk → every VFS entity's id + path.
+    const idByPath = new Map<string, string>()
+    const vfsEntities: Array<{ id: string; path: string }> = []
+    let cursor: string | undefined
+    for (;;) {
+      const page = await (this.brain as any).storage.getNounsWithPagination({ limit: 500, cursor })
+      for (const noun of page.items) {
+        const meta = (noun as any).metadata ?? noun
+        const p = meta?.path
+        if (meta?.vfsType && typeof p === 'string') {
+          idByPath.set(p, noun.id)
+          vfsEntities.push({ id: noun.id, path: p })
+        }
+      }
+      if (!page.hasMore) break
+      cursor = page.nextCursor
+    }
+
+    let removed = 0
+    let restored = 0
+    for (const { id, path } of vfsEntities) {
+      if (path === '/') continue // the root has no parent
+      const expectedParentId = idByPath.get(this.getParentPath(path))
+      if (!expectedParentId) {
+        console.warn(
+          `[VFS] repairContainment: no entity found for parent of ${path} — leaving its edges untouched.`
+        )
+        continue
+      }
+
+      const incoming = await this.brain.related({ to: id, type: VerbType.Contains })
+      let expectedSeen = false
+      for (const edge of incoming) {
+        const isVfsEdge = edge.subtype === 'vfs-contains' || (edge.metadata as any)?.isVFS === true
+        if (!isVfsEdge) continue // never touch user knowledge edges
+        if (edge.from === expectedParentId && !expectedSeen) {
+          expectedSeen = true // keep exactly one correct edge
+          continue
+        }
+        // Stale parent (a pre-fix rename ghost) or a duplicate of the correct
+        // edge (concurrent-writer artifact) — remove it, loudly.
+        await this.brain.unrelate(edge.id)
+        removed++
+        console.warn(
+          `[VFS] repairContainment: removed ${edge.from === expectedParentId ? 'duplicate' : 'stale'} ` +
+            `containment edge ${edge.from} -> ${id} (${path})`
+        )
+      }
+      if (!expectedSeen) {
+        await this.brain.relate({
+          from: expectedParentId,
+          to: id,
+          type: VerbType.Contains,
+          subtype: 'vfs-contains',
+          metadata: { isVFS: true }
+        })
+        restored++
+        console.warn(`[VFS] repairContainment: restored missing containment edge for ${path}`)
+      }
+    }
+
+    return { removed, restored }
+  }
+
   /**
    * Copy a file or directory to a new path.
    *

src/vfs/types.ts

@@ -195,6 +195,36 @@ export interface ReadOptions {
   // VFS-specific options
   cache?: boolean                // Use cache if available (default: true)
   decompress?: boolean           // Auto-decompress (default: true)
+
+  /**
+   * Read the file's content as it stood at a past generation (a number) or
+   * wall-clock instant (a Date) — the temporal read. Serves the exact bytes
+   * the file held then: the historical entity is materialized from the
+   * generation history, and its content blob is retention-protected (bytes
+   * referenced by any in-window generation are never reclaimed). Bounded by
+   * the retention window: reading past the compaction horizon throws.
+   * Bypasses the content cache.
+   */
+  asOf?: number | Date
+}
+
+/**
+ * One entry of a file's version history (see `vfs.history(path)`): the state
+ * the file held immediately after `generation` committed.
+ * `readFile(path, { asOf: generation })` returns these exact bytes while the
+ * generation remains inside the retention window.
+ */
+export interface FileVersion {
+  /** The generation whose write produced this version. */
+  generation: number
+  /** Commit timestamp of that generation (ms since epoch). */
+  timestamp: number
+  /** Content hash of this version's bytes in the content-addressed store. */
+  hash: string
+  /** Content size in bytes. */
+  size: number
+  /** Detected MIME type at that version. */
+  mimeType?: string
 }
 
 export interface MkdirOptions {

tests/integration/aggregate-reserved-fields.test.ts

@@ -0,0 +1,169 @@
+/**
+ * @module tests/integration/aggregate-reserved-fields
+ * @description One field-resolution law across the whole aggregation + query
+ * surface (SELF-AGGREGATE-DELETE-DRIFT). Laws:
+ * (1) aggregates grouped by a RESERVED field (subtype) decrement on delete —
+ *     the delete-side entity view carries every reserved field, so the
+ *     decrement finds its group (counts must never drift from ground truth);
+ * (2) same for update: moving an entity between reserved-field groups
+ *     decrements the old group and increments the new one (no double-count);
+ * (3) aggregation source.where on a reserved field (subtype) FILTERS instead
+ *     of silently matching nothing;
+ * (4) removeMany refuses empty/invalid selectors loudly (bare array, empty
+ *     object, ids: []) instead of resolving as a silent no-op;
+ * (5) find() accepts both where spellings: flattened (entry.title) and
+ *     storage-shaped (metadata.entry.title) resolve to the same rows.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+
+const stubEmbedding = async (text: string): Promise<number[]> => {
+  const hash = text.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0)
+  return new Array(384).fill(0).map((_, i) => Math.sin(hash + i))
+}
+
+describe('aggregation + query field-resolution law', () => {
+  let brain: Brainy
+
+  beforeEach(async () => {
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' as const },
+      embeddingFunction: stubEmbedding
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+  })
+
+  it('reserved-field groupBy decrements on delete (the drift bug)', async () => {
+    brain.defineAggregate({
+      name: 'by_subtype',
+      source: { type: NounType.Document },
+      groupBy: ['subtype'],
+      metrics: { count: { op: 'count' } }
+    })
+
+    const ids: string[] = []
+    for (let i = 0; i < 5; i++) {
+      ids.push(
+        await brain.add({
+          data: `doc-${i}`,
+          type: NounType.Document,
+          subtype: 'note',
+          metadata: { team: 'alpha' }
+        })
+      )
+    }
+    let groups = await brain.queryAggregate('by_subtype')
+    expect(groups).toHaveLength(1)
+    expect(groups[0].groupKey).toEqual({ subtype: 'note' })
+    expect(groups[0].metrics.count).toBe(5)
+
+    await brain.remove(ids[0])
+    await brain.flush()
+
+    groups = await brain.queryAggregate('by_subtype')
+    expect(groups[0].metrics.count).toBe(4)
+    const live = await brain.find({ type: NounType.Document, limit: 100 })
+    expect(groups[0].metrics.count).toBe(live.length)
+  })
+
+  it('reserved-field groupBy moves between groups on update (no double-count)', async () => {
+    brain.defineAggregate({
+      name: 'by_subtype',
+      source: { type: NounType.Document },
+      groupBy: ['subtype'],
+      metrics: { count: { op: 'count' } }
+    })
+    const id = await brain.add({
+      data: 'doc-move',
+      type: NounType.Document,
+      subtype: 'draft'
+    })
+    await brain.update({ id, subtype: 'published' })
+
+    const groups = await brain.queryAggregate('by_subtype')
+    const byKey = Object.fromEntries(
+      groups.map((g) => [String(g.groupKey.subtype), g.metrics.count])
+    )
+    expect(byKey['published']).toBe(1)
+    // The old group must be gone or zero — never still counting the entity.
+    expect(byKey['draft'] ?? 0).toBe(0)
+  })
+
+  it('source.where on a reserved field filters instead of matching nothing', async () => {
+    brain.defineAggregate({
+      name: 'notes_only',
+      source: { type: NounType.Document, where: { subtype: 'note' } },
+      groupBy: ['team'],
+      metrics: { count: { op: 'count' } }
+    })
+    await brain.add({
+      data: 'n1',
+      type: NounType.Document,
+      subtype: 'note',
+      metadata: { team: 'alpha' }
+    })
+    await brain.add({
+      data: 'd1',
+      type: NounType.Document,
+      subtype: 'draft',
+      metadata: { team: 'alpha' }
+    })
+
+    const groups = await brain.queryAggregate('notes_only')
+    expect(groups).toHaveLength(1)
+    expect(groups[0].metrics.count).toBe(1) // the note, never the draft
+  })
+
+  it('removeMany refuses empty/invalid selectors loudly', async () => {
+    const id = await brain.add({ data: 'keep-me', type: NounType.Document })
+
+    // Bare array passed positionally — the classic silent no-op.
+    await expect(
+      brain.removeMany([id] as unknown as Parameters<typeof brain.removeMany>[0])
+    ).rejects.toThrow(/bare array/)
+    // Empty selector object.
+    await expect(
+      brain.removeMany({} as Parameters<typeof brain.removeMany>[0])
+    ).rejects.toThrow(/requires a selector/)
+    // Explicit empty id list.
+    await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
+
+    // Nothing was deleted by any of the refused calls.
+    expect(await brain.get(id)).toBeTruthy()
+  })
+
+  it('find() accepts both flattened and metadata.-prefixed where spellings', async () => {
+    await brain.add({
+      data: 'nested-doc',
+      type: NounType.Document,
+      metadata: { entry: { title: 'T1' }, classifier: { contextHints: { vfsPath: '/n/a.md' } } }
+    })
+    await brain.flush()
+
+    const flat = await brain.find({
+      type: NounType.Document,
+      where: { 'entry.title': 'T1' },
+      limit: 10
+    })
+    const prefixed = await brain.find({
+      type: NounType.Document,
+      where: { 'metadata.entry.title': 'T1' },
+      limit: 10
+    })
+    const deepPrefixed = await brain.find({
+      type: NounType.Document,
+      where: { 'metadata.classifier.contextHints.vfsPath': '/n/a.md' },
+      limit: 10
+    })
+    expect(flat).toHaveLength(1)
+    expect(prefixed).toHaveLength(1)
+    expect(prefixed[0].id).toBe(flat[0].id)
+    expect(deepPrefixed).toHaveLength(1)
+  })
+})

tests/integration/aggregation-state-persistence.test.ts

@@ -0,0 +1,311 @@
+/**
+ * @module tests/integration/aggregation-state-persistence
+ * @description The boot-order contract for the aggregation engine. Five laws:
+ * (1) STATE ADOPTION — a reopen with an unchanged defineAggregate() adopts the
+ *     persisted state and performs NO store walk. (The pre-fix behavior: the
+ *     synchronous define always beat the async init, flagged a backfill, and
+ *     the first query wiped the just-loaded state and re-walked the whole
+ *     store — every restart, forever.)
+ * (2) SINGLE-FLIGHT + BATCH — concurrent cold queries across multiple pending
+ *     aggregates share exactly ONE store walk; a query never wipes another's
+ *     partial progress and M pending aggregates cost one enumeration, not M.
+ * (3) CHANGED DEFINITION — a real definition change still backfills, exactly.
+ * (4) PERSISTED-ONLY DEFINITIONS — an app that does not re-define at boot can
+ *     query a persisted aggregate without racing a spurious "not defined".
+ * (5) QUIET KEYS — the engine's persistence keys (__aggregation_*) are
+ *     recognized system keys: no "Unknown key format" warning at boot.
+ */
+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+import { NounType } from '../../src/types/graphTypes.js'
+import type { AggregateDefinition } from '../../src/types/brainy.types.js'
+import { prodLog } from '../../src/utils/logger.js'
+
+const SPENDING: AggregateDefinition = {
+  name: 'spending',
+  source: { type: NounType.Event, where: { domain: 'financial' } },
+  groupBy: ['category'],
+  metrics: {
+    total: { op: 'sum', field: 'amount' },
+    count: { op: 'count' }
+  }
+}
+
+/** Same name, different metrics — a REAL definition change (hash differs). */
+const SPENDING_CHANGED: AggregateDefinition = {
+  ...SPENDING,
+  metrics: {
+    total: { op: 'sum', field: 'amount' },
+    count: { op: 'count' },
+    average: { op: 'avg', field: 'amount' }
+  }
+}
+
+describe('aggregation state persistence — boot-order contract', () => {
+  let dir: string
+
+  beforeEach(() => {
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-agg-persist-'))
+  })
+
+  afterEach(() => {
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  const open = async (): Promise<any> => {
+    const b: any = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true
+    })
+    await b.init()
+    return b
+  }
+
+  /** Count store walks by intercepting the storage adapter's getNouns. */
+  const countWalks = (brain: any): { count: () => number } => {
+    const storage = brain.storage
+    const orig = storage.getNouns.bind(storage)
+    let calls = 0
+    storage.getNouns = async (opts: unknown) => {
+      calls++
+      return orig(opts)
+    }
+    return { count: () => calls }
+  }
+
+  const seed = async (brain: any): Promise<void> => {
+    for (let i = 0; i < 12; i++) {
+      await brain.add({
+        data: `tx ${i}`,
+        type: NounType.Event,
+        metadata: {
+          domain: 'financial',
+          category: i % 2 === 0 ? 'food' : 'transport',
+          amount: 10 + i
+        }
+      })
+    }
+  }
+
+  it('adopts persisted state on reopen with an unchanged definition — zero walks', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    const before = await brain1.queryAggregate('spending')
+    expect(before.length).toBe(2)
+    await brain1.close()
+
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING) // the standard declarative boot pattern
+    await brain2.getNounCount() // settle init paths before counting walks
+    const walks = countWalks(brain2)
+
+    const after = await brain2.queryAggregate('spending')
+
+    expect(walks.count()).toBe(0)
+    const key = (r: any) => r.groupKey.category
+    expect(
+      after.map((r: any) => [key(r), r.metrics.total, r.metrics.count]).sort()
+    ).toEqual(
+      before.map((r: any) => [key(r), r.metrics.total, r.metrics.count]).sort()
+    )
+    await brain2.close()
+  })
+
+  it('adopted state keeps accumulating: post-reopen writes land on top of it', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING)
+    // A write BEFORE the first query: if it lands before state adoption the
+    // engine must choose an exact rescan over adoption — either way the
+    // result must include all 13 entities.
+    await brain2.add({
+      data: 'late tx',
+      type: NounType.Event,
+      metadata: { domain: 'financial', category: 'food', amount: 100 }
+    })
+    const rows = await brain2.queryAggregate('spending')
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(7) // 6 seeded + 1 late
+    await brain2.close()
+  })
+
+  it('a changed definition still backfills — exactly once, with correct results', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING_CHANGED)
+    await brain2.getNounCount()
+    const walks = countWalks(brain2)
+
+    const rows = await brain2.queryAggregate('spending')
+
+    expect(walks.count()).toBe(1) // 12 entities = one page = one getNouns call
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(6)
+    expect(food.metrics.average).toBeCloseTo(food.metrics.total / 6)
+    await brain2.close()
+  })
+
+  it('concurrent cold queries across two aggregates share exactly ONE walk', async () => {
+    const brain = await open()
+    brain.defineAggregate(SPENDING)
+    brain.defineAggregate({
+      ...SPENDING,
+      name: 'by_category_count',
+      metrics: { count: { op: 'count' } }
+    })
+    await seed(brain)
+    await brain.getNounCount()
+    const walks = countWalks(brain)
+
+    const results = await Promise.all([
+      brain.queryAggregate('spending'),
+      brain.queryAggregate('by_category_count'),
+      brain.queryAggregate('spending'),
+      brain.queryAggregate('by_category_count'),
+      brain.queryAggregate('spending'),
+      brain.queryAggregate('by_category_count')
+    ])
+
+    expect(walks.count()).toBe(1) // 12 entities = one page; one walk fills both
+    for (const rows of results) {
+      const total = rows.reduce((s: number, r: any) => s + r.metrics.count, 0)
+      expect(total).toBe(12)
+    }
+
+    // Warm re-query: converged, no further walks.
+    await brain.queryAggregate('spending')
+    expect(walks.count()).toBe(1)
+    await brain.close()
+  })
+
+  it('persisted-only definitions are queryable without re-defining at boot', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    const brain2 = await open()
+    // NO defineAggregate — the app relies on the persisted definition.
+    const walks = countWalks(brain2)
+    const rows = await brain2.queryAggregate('spending')
+    expect(walks.count()).toBe(0) // persisted state adopted here too
+    expect(rows.length).toBe(2)
+    await brain2.close()
+  })
+
+  it('generation-mismatched persisted state is rescanned once, loudly — never adopted', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    // Simulate the copied-store incident class: a fact-log truncation (or an
+    // unclean shutdown) leaves the committed watermark different from the
+    // generation the flushed state was stamped with.
+    const tamper: any = await open()
+    const key = '__aggregation_state_spending__'
+    const stored = await tamper.storage.getMetadata(key)
+    expect(typeof stored.sourceGeneration).toBe('number') // the stamp is really persisted
+    await tamper.storage.saveMetadata(key, {
+      ...stored,
+      sourceGeneration: stored.sourceGeneration + 5
+    })
+    await tamper.close()
+
+    const warnSpy = vi.spyOn(prodLog, 'warn')
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING)
+    await brain2.getNounCount()
+    const walks = countWalks(brain2)
+
+    const rows = await brain2.queryAggregate('spending')
+
+    expect(walks.count()).toBe(1) // exactly ONE rescan — no silent adopt, no spin
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(6) // rescan produced exact results
+    expect(
+      warnSpy.mock.calls.some(args => String(args[0]).includes('rescanning instead of adopting'))
+    ).toBe(true) // and it said so out loud
+    warnSpy.mockRestore()
+    await brain2.close()
+  })
+
+  it('a failing walk is loud, non-destructive, and latched — never a silent retry loop', async () => {
+    // Fresh define + seeded writes: the write hooks have populated LIVE state,
+    // and the first-query rescan is still pending. The incident shape
+    // (wipe-before-scan + no try/catch + per-query re-walk) would have wiped
+    // that live state and silently re-walked on every query.
+    const brain: any = await open()
+    brain.defineAggregate(SPENDING)
+    await seed(brain)
+    expect(brain._aggregationIndex.queryAggregate({ name: 'spending' }).length).toBe(2)
+
+    const storage = brain.storage
+    const origGetNouns = storage.getNouns.bind(storage)
+    let walkAttempts = 0
+    storage.getNouns = async () => {
+      walkAttempts++
+      throw new Error('injected storage failure')
+    }
+
+    // First query: the walk fails LOUDLY with the storage error.
+    await expect(brain.queryAggregate('spending')).rejects.toThrow('injected storage failure')
+    expect(walkAttempts).toBe(1)
+
+    // Live state was NOT destroyed by the failed walk (staging was dropped).
+    expect(brain._aggregationIndex.queryAggregate({ name: 'spending' }).length).toBe(2)
+
+    // Second query inside the cooldown: instant loud failure, NO new walk.
+    await expect(brain.queryAggregate('spending')).rejects.toThrow('failure cooldown')
+    expect(walkAttempts).toBe(1)
+
+    // Heal the storage + expire the cooldown: one fresh walk succeeds exactly.
+    storage.getNouns = origGetNouns
+    brain._aggregationBackfillFailure.at = Date.now() - 60_000
+    const rows = await brain.queryAggregate('spending')
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(6)
+    await brain.close()
+  })
+
+  it('aggregation persistence keys never log "Unknown key format"', async () => {
+    const warnSpy = vi.spyOn(prodLog, 'warn')
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING)
+    await brain2.queryAggregate('spending')
+    await brain2.close()
+
+    const offenders = warnSpy.mock.calls
+      .map(args => String(args[0]))
+      .filter(
+        msg =>
+          msg.includes('Unknown key format') &&
+          (msg.includes('__aggregation_') || msg.includes('brainy:entityIdMapper'))
+      )
+    expect(offenders).toEqual([])
+    warnSpy.mockRestore()
+  })
+})

tests/integration/background-dedup-lifecycle.test.ts

@@ -0,0 +1,85 @@
+/**
+ * @module tests/integration/background-dedup-lifecycle
+ * @description The post-import background deduplication pass (a merge-DELETE
+ * writer) obeys the same contract as the inline pass. Laws:
+ * (1) enableDeduplication:false schedules NO background pass — the brain-owned
+ *     deduplicator is never even constructed;
+ * (2) by default the pass IS scheduled, brain-owned, with an unref'd timer
+ *     (a pending pass never holds the process open);
+ * (3) repeated imports debounce into ONE pending batch on ONE instance
+ *     (per-coordinator instances used to arm one timer per import);
+ * (4) close() cancels pending work — no delete pass can fire after close.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import { Brainy } from '../../src/brainy.js'
+
+const ROWS = [
+  { name: 'Alice Zephyr', role: 'engineer' },
+  { name: 'Bob Quill', role: 'writer' }
+]
+
+// Keep imports fast and deterministic — dedup scheduling is what's under test.
+const FAST = {
+  enableNeuralExtraction: false,
+  enableRelationshipInference: false,
+  enableConceptExtraction: false
+} as const
+
+// Deterministic stub embedder (hnsw-rebuild.test.ts pattern) — dedup
+// scheduling never inspects vector CONTENT, so skip the WASM model load.
+const stubEmbedding = async (text: string): Promise<number[]> => {
+  const hash = text.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0)
+  const vector = new Array(384).fill(0).map((_, i) => Math.sin(hash + i))
+  return vector
+}
+
+describe('background dedup lifecycle', () => {
+  let brain: Brainy
+
+  beforeEach(async () => {
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' as const },
+      embeddingFunction: stubEmbedding
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+  })
+
+  it('enableDeduplication:false schedules no background pass at all', async () => {
+    await brain.import(ROWS, { ...FAST, enableDeduplication: false })
+    expect((brain as any)._backgroundDedup).toBeUndefined()
+  })
+
+  it('default schedules a brain-owned pass with an unref-ed timer', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const dedup = (brain as any)._backgroundDedup
+    expect(dedup).toBeDefined()
+    expect(dedup.pendingImports.size).toBe(1)
+    const timer = dedup.debounceTimer
+    expect(timer).toBeDefined()
+    // Node timers expose hasRef(); an unref'd timer must not hold the process.
+    expect(typeof timer.hasRef).toBe('function')
+    expect(timer.hasRef()).toBe(false)
+  })
+
+  it('imports debounce into one pending batch on one brain-owned instance', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const first = (brain as any)._backgroundDedup
+    await brain.import([{ name: 'Cara Vex', role: 'analyst' }], { ...FAST })
+    expect((brain as any)._backgroundDedup).toBe(first)
+    expect(first.pendingImports.size).toBe(2)
+  })
+
+  it('close() cancels pending background dedup', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const dedup = (brain as any)._backgroundDedup
+    expect(dedup.debounceTimer).toBeDefined()
+    await brain.close()
+    expect(dedup.debounceTimer).toBeUndefined()
+    expect(dedup.pendingImports.size).toBe(0)
+  })
+})

tests/integration/count-invariant.test.ts

@@ -0,0 +1,109 @@
+/**
+ * @module tests/integration/count-invariant
+ * @description Pattern-C acceptance: the user-facing scalar total and the
+ * per-type array are ONE consistent projection — `getNounCount() === Σ
+ * nounCountsByType` (and the verb mirror) after any interleaving of add /
+ * update-visibility-flip / delete, AND across a reopen.
+ *
+ * The bug (finding 5): delete decremented the per-type array but never the
+ * scalar for nouns, and neither for verbs — so `getNounCount()`/`getVerbCount()`
+ * inflated permanently (the stale scalar wins pagination via
+ * `Math.max(total, collected.length)` and is persisted). The fix restores the
+ * symmetric decrement the code always intended (its own comments say "symmetric
+ * with the increments"). This test would report an inflated count before the
+ * fix and the exact count after.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+
+const sum = (a: Uint32Array): number => a.reduce((s, c) => s + c, 0)
+
+describe('count invariant — scalar total === Σ per-type, across delete + reopen (finding 5)', () => {
+  let dir: string
+  let brain: any
+
+  const open = async (d: string) => {
+    const b = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: d },
+      dimensions: 384,
+      silent: true
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-count-inv-'))
+    brain = await open(dir)
+  })
+
+  afterEach(async () => {
+    try { await brain.close() } catch { /* already closed */ }
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('noun delete decrements the scalar total, not just the per-type bucket', async () => {
+    const things: string[] = []
+    for (let i = 0; i < 5; i++) things.push(await brain.add({ data: `thing ${i}`, type: NounType.Thing }))
+    for (let i = 0; i < 3; i++) await brain.add({ data: `concept ${i}`, type: NounType.Concept })
+
+    const storage = brain['storage']
+    expect(await storage.getNounCount()).toBe(8)
+    expect(sum(storage.getNounCountsByType())).toBe(8)
+
+    // Delete 2 Things — the scalar must drop with the bucket (pre-fix it stayed 8).
+    await brain.remove(things[0])
+    await brain.remove(things[1])
+
+    expect(await storage.getNounCount()).toBe(6)
+    expect(sum(storage.getNounCountsByType())).toBe(6)
+    // The invariant: scalar === Σ per-type.
+    expect(await storage.getNounCount()).toBe(sum(storage.getNounCountsByType()))
+  })
+
+  it('verb delete decrements BOTH the scalar total AND the per-type bucket', async () => {
+    const ids: string[] = []
+    for (let i = 0; i < 4; i++) ids.push(await brain.add({ data: `node ${i}`, type: NounType.Thing }))
+    const edges: string[] = []
+    edges.push(await brain.relate({ from: ids[0], to: ids[1], type: VerbType.RelatedTo }))
+    edges.push(await brain.relate({ from: ids[1], to: ids[2], type: VerbType.RelatedTo }))
+    edges.push(await brain.relate({ from: ids[2], to: ids[3], type: VerbType.Contains }))
+    edges.push(await brain.relate({ from: ids[0], to: ids[3], type: VerbType.Contains }))
+
+    const storage = brain['storage']
+    expect(await storage.getVerbCount()).toBe(4)
+    expect(sum(storage.getVerbCountsByType())).toBe(4)
+
+    // Unrelate one edge — pre-fix, verb delete touched NEITHER counter (both stayed 4).
+    await brain.unrelate(edges[0])
+
+    expect(await storage.getVerbCount()).toBe(3)
+    expect(sum(storage.getVerbCountsByType())).toBe(3)
+    expect(await storage.getVerbCount()).toBe(sum(storage.getVerbCountsByType()))
+  })
+
+  it('the corrected counts persist and survive a reopen', async () => {
+    const things: string[] = []
+    for (let i = 0; i < 6; i++) things.push(await brain.add({ data: `t${i}`, type: NounType.Thing }))
+    const a = await brain.relate({ from: things[0], to: things[1], type: VerbType.RelatedTo })
+    await brain.relate({ from: things[1], to: things[2], type: VerbType.RelatedTo })
+    await brain.remove(things[5])
+    await brain.unrelate(a)
+    await brain.flush()
+    await brain.close()
+
+    // Reopen from disk — counts.json must carry the corrected totals.
+    brain = await open(dir)
+    const storage = brain['storage']
+    expect(await storage.getNounCount()).toBe(5)
+    expect(await storage.getVerbCount()).toBe(1)
+    expect(await storage.getNounCount()).toBe(sum(storage.getNounCountsByType()))
+    expect(await storage.getVerbCount()).toBe(sum(storage.getVerbCountsByType()))
+  })
+})

tests/integration/counter-recount.test.ts

@@ -0,0 +1,122 @@
+/**
+ * @module tests/integration/counter-recount
+ * @description Counter honesty. Two laws under test:
+ * (1) REMOVAL NEVER REQUIRES RE-READING THE REMOVED RECORD — the count
+ *     decrement falls back to the caller's pre-delete read when the canonical
+ *     metadata re-read returns null (replace race / ghost), instead of being
+ *     silently skipped. The skip minted permanent inflation: adds counted,
+ *     paired removals not decremented, and Math.max(totalNounCount, scanned)
+ *     pinned the inflated scalar forever.
+ * (2) THE SANCTIONED RECOUNT — repairIndex() unconditionally recomputes and
+ *     PERSISTS every counter rollup (scalar totals + per-type maps +
+ *     type-statistics) from one canonical walk, so an already-inflated brain
+ *     is permanently corrected (survives reopen).
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+
+/** Absolute path of one entity's `<id>` directory (or null if not present). */
+function entityDir(root: string, id: string): string | null {
+  const base = path.join(root, 'entities', 'nouns')
+  if (!fs.existsSync(base)) return null
+  for (const shard of fs.readdirSync(base)) {
+    const candidate = path.join(base, shard, id)
+    if (fs.existsSync(candidate)) return candidate
+  }
+  return null
+}
+
+describe('counter honesty — removal without re-reading + the sanctioned recount', () => {
+  let dir: string
+  let brain: any
+
+  const open = async () => {
+    const b: any = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true,
+      dimensions: 384
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-recount-'))
+    brain = await open()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('the counter returns to baseline across add→remove→re-create cycles (no drift)', async () => {
+    const baseline = await brain.storage.getNounCount()
+    for (let i = 0; i < 4; i++) {
+      const id = await brain.add({ data: `cycle ${i}`, type: 'document', metadata: { i } })
+      expect(await brain.storage.getNounCount()).toBe(baseline + 1)
+      await brain.remove(id)
+      expect(await brain.storage.getNounCount()).toBe(baseline)
+    }
+  })
+
+  it('deleteNoun decrements from the provided prior record when the canonical re-read is null', async () => {
+    const id = await brain.add({ data: 'to be ghosted', type: 'document', metadata: { g: 1 } })
+    await brain.flush()
+    const baseline = await brain.storage.getNounCount()
+
+    // Capture the pre-delete read (what remove() holds), then simulate the
+    // replace-race / ghost shape: the metadata leg vanishes before the delete's
+    // internal re-read.
+    const prior = await brain.storage.getNounMetadata(id)
+    expect(prior).not.toBeNull()
+    const eDir = entityDir(dir, id)!
+    for (const f of fs.readdirSync(eDir)) {
+      if (f.startsWith('metadata.json')) fs.rmSync(path.join(eDir, f))
+    }
+
+    // Without the prior record this decrement used to be silently skipped.
+    await brain.storage.deleteNoun(id, prior)
+    expect(await brain.storage.getNounCount()).toBe(baseline - 1)
+    // Full removal still holds: nothing left on disk.
+    expect(entityDir(dir, id)).toBeNull()
+  })
+
+  it('repairIndex() recounts an inflated persisted scalar over CLEAN shelves — and it survives reopen', async () => {
+    for (let i = 0; i < 3; i++) {
+      await brain.add({ data: `real ${i}`, type: 'document', metadata: { i } })
+    }
+    await brain.flush()
+    const honest = await brain.storage.getNounCount()
+    // Pagination's totalCount has its own baseline: it enumerates EVERYTHING
+    // (including the internal VFS root), while the user-facing scalar counts
+    // only public entities — so the two legitimately differ by the internals.
+    const pageHonest = (await brain.storage.getNounsWithPagination({ limit: 1000, offset: 0 })).totalCount
+
+    // Simulate the historical drift: an inflated persisted scalar (deletes whose
+    // decrement was skipped). Persist it so a reopen rehydrates the lie.
+    ;(brain.storage as any).totalNounCount = honest + 52
+    await (brain.storage as any).persistCounts()
+    await brain.close()
+    brain = await open()
+    expect(await brain.storage.getNounCount()).toBe(honest + 52) // the lie survived reopen
+
+    // The sanctioned recount — unconditional in repairIndex (no orphans needed).
+    await brain.repairIndex()
+    expect(await brain.storage.getNounCount()).toBe(honest)
+
+    // Permanently: the corrected counter survives another reopen.
+    await brain.close()
+    brain = await open()
+    expect(await brain.storage.getNounCount()).toBe(honest)
+
+    // And the paginated totalCount (the Math.max consumer) is honest too —
+    // back to ITS baseline, no longer pinned high by the inflated scalar.
+    const page = await brain.storage.getNounsWithPagination({ limit: 1000, offset: 0 })
+    expect(page.totalCount).toBe(pageHonest)
+  })
+})

tests/integration/db-mvcc.test.ts

@@ -515,7 +515,15 @@ describe('8.0 Db API — generational MVCC', () => {
       await brain.transact([{ op: 'update', id: uid('compact-e'), metadata: { v: 4 } }])
     ).release()
 
-    const recordsBefore = (await storage.listRawObjects('_generations')).length
+    // History record-sets only — the generation FACT LOG also lives under
+    // `_generations/` (at `facts/`) and is deliberately NOT reclaimed by
+    // history compaction (facts are the future canonical, not undo history).
+    const historyRecords = async (): Promise<number> =>
+      (await storage.listRawObjects('_generations')).filter(
+        (p: string) => !p.startsWith('_generations/facts/')
+      ).length
+
+    const recordsBefore = await historyRecords()
     expect(recordsBefore).toBeGreaterThan(0)
 
     // Compact while pinned: record-sets above the pin survive, pinned reads stay correct.
@@ -528,7 +536,7 @@ describe('8.0 Db API — generational MVCC', () => {
     const second = await brain.compactHistory()
     expect(first.removedGenerations + second.removedGenerations).toBeGreaterThan(0)
 
-    const recordsAfter = (await storage.listRawObjects('_generations')).length
+    const recordsAfter = await historyRecords()
     expect(recordsAfter).toBeLessThan(recordsBefore)
     expect(recordsAfter).toBe(0)
 
@@ -1272,24 +1280,32 @@ describe('8.0 Db API — generational MVCC', () => {
     await expect(reopened.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
   })
 
-  it('Model-B retention — setRetentionBudget drives adaptive reclaim on flush; live data intact', async () => {
-    // Default brain → ADAPTIVE retention. A coordinator (e.g. cor's ResourceManager)
-    // pushes a byte budget via setRetentionBudget(); auto-compaction on flush() reclaims
-    // oldest history down toward it. Each update's before-image carries the full prior
-    // 384-dim vector (~KBs), so ~13 generations far exceed a few-KB budget.
-    const { brain } = await openFsBrain()
+  it('Model-B retention — flush() NEVER compacts (8.9.0); adaptive reclaim runs at close()', async () => {
+    // Default brain → ADAPTIVE retention with a driven byte budget far below
+    // the accumulated history (~13 generations of full-vector before-images).
+    // The 8.9.0 law: flush() is durability-only — it must not reclaim even
+    // when the budget is exceeded (reclaim-on-flush blocked production writes
+    // for 25-191s). Maintenance runs at close(), time-bounded.
+    const { brain, dir } = await openFsBrain()
     const a = uid('ret-budget')
     await brain.add({ id: a, type: NounType.Document, data: 'v0', vector: vec(1), metadata: { v: 0 } })
     for (let v = 1; v <= 12; v++) await brain.update({ id: a, metadata: { v } })
 
     brain.setRetentionBudget(6000) // ~6 KB — well below the accumulated history
-    await brain.flush() // group-commit + adaptive auto-compaction under the budget
+    await brain.flush()
 
-    // History was reclaimed (the horizon advanced past the oldest generations)…
-    expect(generationStoreOf(brain).horizon()).toBeGreaterThan(0)
-    await expect(brain.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
-    // …but the budget reclaims HISTORY only — the live record is untouched.
-    expect((await brain.get(a))?.metadata?.v).toBe(12)
+    // flush() paid durability only: nothing reclaimed, all history readable.
+    expect(generationStoreOf(brain).horizon()).toBe(0)
+    const probe = await brain.asOf(1) // readable proves nothing was reclaimed…
+    await probe.release() // …and MUST be released: a held pin would (correctly)
+    // protect every newer generation through the close() compaction below.
+    await brain.close() // ← THE auto-compaction site now
+
+    // close() reclaimed under the budget; live record intact; horizon durable.
+    const { brain: reopened } = await openFsBrain(dir)
+    expect(generationStoreOf(reopened).horizon()).toBeGreaterThan(0)
+    await expect(reopened.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    expect((await reopened.get(a))?.metadata?.v).toBe(12)
   })
 
   // ==========================================================================

tests/integration/delete-full-removal.test.ts

@@ -0,0 +1,161 @@
+/**
+ * @module tests/integration/delete-full-removal
+ * @description Canonical noun/verb deletes are FULL removals: both legs
+ * (metadata + vectors) AND the entity's `<id>/` container are removed, so a
+ * delete leaves NOTHING behind. Regression for the ghost/scar defect where
+ * remove() deleted the vector INDEX entry + the canonical metadata leg but never
+ * the canonical vectors.json leg or the directory — leaving an orphan that reads
+ * as absent (getNoun needs both legs) yet inflated the enumerated count and
+ * confused locator resolution. Also covers the operator repair sweep
+ * (brain.repairIndex()) that prunes orphans left by the pre-fix behavior.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+
+/** All entity-directory names (the `<id>` dirs) under entities/<kind>. */
+function entityDirs(root: string, kind: 'nouns' | 'verbs'): string[] {
+  const base = path.join(root, 'entities', kind)
+  if (!fs.existsSync(base)) return []
+  const ids: string[] = []
+  for (const shard of fs.readdirSync(base)) {
+    const shardDir = path.join(base, shard)
+    if (!fs.statSync(shardDir).isDirectory()) continue
+    for (const id of fs.readdirSync(shardDir)) {
+      if (fs.statSync(path.join(shardDir, id)).isDirectory()) ids.push(id)
+    }
+  }
+  return ids
+}
+
+/** Absolute path of one entity's `<id>` directory (or null if not present). */
+function entityDir(root: string, kind: 'nouns' | 'verbs', id: string): string | null {
+  const base = path.join(root, 'entities', kind)
+  if (!fs.existsSync(base)) return null
+  for (const shard of fs.readdirSync(base)) {
+    const candidate = path.join(base, shard, id)
+    if (fs.existsSync(candidate)) return candidate
+  }
+  return null
+}
+
+describe('canonical delete is a full removal (no ghost/scar directory)', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-del-'))
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir }, silent: true, dimensions: 384 })
+    await brain.init()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('remove() deletes BOTH legs and the container — nothing left on disk', async () => {
+    const ids: string[] = []
+    for (let i = 0; i < 3; i++) ids.push(await brain.add({ data: `doc ${i}`, type: 'document', metadata: { i } }))
+    await brain.flush()
+
+    // (A filesystem brain also has its VFS-root noun, so don't assume an exact set.)
+    const before = entityDirs(dir, 'nouns')
+    expect(before).toEqual(expect.arrayContaining(ids))
+
+    await brain.remove(ids[1])
+    await brain.flush()
+
+    // getNoun is null AND the on-disk container is fully gone (no orphan), and
+    // EXACTLY one directory disappeared (the removed entity's).
+    expect(await brain.get(ids[1])).toBeNull()
+    expect(entityDir(dir, 'nouns', ids[1])).toBeNull()
+    const after = entityDirs(dir, 'nouns')
+    expect(after).toHaveLength(before.length - 1)
+    expect(after).toEqual(expect.arrayContaining([ids[0], ids[2]]))
+    expect(after).not.toContain(ids[1])
+  })
+
+  it('the enumerated count is honest after a delete (no monotonic inflation)', async () => {
+    const ids: string[] = []
+    for (let i = 0; i < 4; i++) ids.push(await brain.add({ data: `n${i}`, type: 'document', metadata: { i } }))
+    await brain.flush()
+
+    const before = await brain.storage.getNounsWithPagination({ limit: 100, offset: 0 })
+
+    await brain.remove(ids[0])
+    await brain.remove(ids[1])
+    await brain.flush()
+
+    // The count drops by EXACTLY the two removed entities — no ghost lingers to
+    // hold the total up (the pre-fix defect left it monotonic).
+    const after = await brain.storage.getNounsWithPagination({ limit: 100, offset: 0 })
+    expect(after.totalCount).toBe(before.totalCount - 2)
+    const remaining = after.items.map((n: any) => n.id)
+    expect(remaining).toEqual(expect.arrayContaining([ids[2], ids[3]]))
+    expect(remaining).not.toContain(ids[0])
+    expect(remaining).not.toContain(ids[1])
+  })
+})
+
+describe('repairIndex() prunes orphan containers left by the pre-fix partial delete', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-orphan-'))
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir }, silent: true, dimensions: 384 })
+    await brain.init()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('a vector-only "ghost" (metadata leg deleted out-of-band) is pruned', async () => {
+    const ids: string[] = []
+    for (let i = 0; i < 3; i++) ids.push(await brain.add({ data: `g${i}`, type: 'document', metadata: { i } }))
+    await brain.flush()
+
+    // Simulate the pre-fix defect on ids[0]: remove ONLY its metadata leg,
+    // leaving vectors.json + the directory (the exact ghost shape).
+    const ghostDir = entityDir(dir, 'nouns', ids[0])!
+    for (const f of fs.readdirSync(ghostDir)) {
+      if (f.startsWith('metadata.json')) fs.rmSync(path.join(ghostDir, f))
+    }
+    // The ghost dir (vectors.json, no metadata content leg) still sits on disk.
+    expect(entityDir(dir, 'nouns', ids[0])).not.toBeNull()
+
+    await brain.repairIndex()
+
+    // The sweep removes the ghost container; the two healthy entities survive.
+    expect(entityDir(dir, 'nouns', ids[0])).toBeNull()
+    expect(entityDir(dir, 'nouns', ids[1])).not.toBeNull()
+    expect(entityDir(dir, 'nouns', ids[2])).not.toBeNull()
+  })
+
+  it('an empty "scar" directory is pruned', async () => {
+    const id = await brain.add({ data: 'lonely', type: 'document', metadata: {} })
+    await brain.flush()
+    const scarDir = entityDir(dir, 'nouns', id)!
+    for (const f of fs.readdirSync(scarDir)) fs.rmSync(path.join(scarDir, f)) // empty the dir, keep it
+    expect(fs.existsSync(scarDir)).toBe(true)
+
+    await brain.repairIndex()
+
+    expect(fs.existsSync(scarDir)).toBe(false)
+  })
+
+  it('a healthy entity (both legs present) is NEVER pruned', async () => {
+    const id = await brain.add({ data: 'keep me', type: 'document', metadata: { keep: true } })
+    await brain.flush()
+
+    await brain.repairIndex()
+
+    expect(entityDir(dir, 'nouns', id)).not.toBeNull()
+    expect(await brain.get(id)).not.toBeNull()
+  })
+})

tests/integration/entity-tree-stamp.test.ts

@@ -0,0 +1,163 @@
+/**
+ * @module tests/integration/entity-tree-stamp
+ * @description The entity tree's FAMILY STAMP: written at flush/close with
+ * `sourceGeneration` (the committed generation the canonical tree reflects)
+ * plus rollup invariants (entity/relationship counts); verified at open by
+ * comparison — coherent on a clean close, LOUD on genuine incoherence
+ * (tampered counters), healed by repairIndex()'s recount + re-stamp. One
+ * verifier reads both member modes.
+ */
+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import {
+  Brainy,
+  readFamilyStamp,
+  verifyFamilyStamp,
+  ENTITY_TREE_STAMP_PATH,
+  type FamilyStamp
+} from '../../src/index.js'
+import { prodLog } from '../../src/utils/logger.js'
+
+describe('entity-tree family stamp', () => {
+  let dir: string
+  let brain: any
+
+  const open = async () => {
+    const b: any = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true,
+      dimensions: 384
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-stamp-'))
+    brain = await open()
+  })
+  afterEach(async () => {
+    vi.restoreAllMocks()
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('flush() writes the stamp: rollup mode, counts + sourceGeneration match live state', async () => {
+    for (let i = 0; i < 3; i++) await brain.add({ data: `s${i}`, type: 'document', metadata: { i } })
+    await brain.flush()
+
+    const stamp = (await readFamilyStamp(brain.storage, ENTITY_TREE_STAMP_PATH)) as FamilyStamp
+    expect(stamp).not.toBeNull()
+    expect(stamp.family).toBe('entity-tree')
+    expect(stamp.members.mode).toBe('rollup')
+    const invariants = (stamp.members as any).invariants
+    expect(invariants.nounCount).toBe(await brain.storage.getNounCount())
+    expect(invariants.verbCount).toBe(await brain.storage.getVerbCount())
+    expect(stamp.sourceGeneration).toBe(brain.generation())
+    expect(stamp.generation).toBeGreaterThanOrEqual(1)
+  })
+
+  it('a cleanly-closed store reopens COHERENT (no incoherence warning)', async () => {
+    await brain.add({ data: 'clean', type: 'document', metadata: {} })
+    await brain.close()
+
+    const warn = vi.spyOn(prodLog, 'warn')
+    brain = await open()
+    const stampWarnings = warn.mock.calls.filter((c) => String(c[0]).includes('entity-tree stamp'))
+    expect(stampWarnings).toEqual([])
+  })
+
+  it('tampered counters surface as INCOHERENT at open; repairIndex() heals + re-stamps', async () => {
+    for (let i = 0; i < 3; i++) await brain.add({ data: `t${i}`, type: 'document', metadata: { i } })
+    await brain.close()
+
+    // Simulate counter drift AFTER the stamp was written: inflate the
+    // persisted scalar the way the historical decrement-skip did.
+    brain = await open()
+    ;(brain.storage as any).totalNounCount += 52
+    await (brain.storage as any).persistCounts()
+    await brain.close()
+    // The close boundary re-stamps with the inflated counter — so tamper the
+    // STAMP instead for a deterministic mismatch: stamped counts differ from
+    // the (inflated) live ones at the NEXT open only if the stamp is older.
+    // Rewrite the stamp with the honest counts + current sourceGeneration.
+    const raw = JSON.parse(
+      require('node:zlib')
+        .gunzipSync(fs.readFileSync(path.join(dir, `${ENTITY_TREE_STAMP_PATH}.gz`)))
+        .toString('utf-8')
+    ) as FamilyStamp
+    const honest = { ...raw }
+    ;(honest.members as any).invariants.nounCount -= 52
+    fs.writeFileSync(
+      path.join(dir, `${ENTITY_TREE_STAMP_PATH}.gz`),
+      require('node:zlib').gzipSync(JSON.stringify(honest))
+    )
+
+    const warn = vi.spyOn(prodLog, 'warn')
+    brain = await open()
+    const incoherent = warn.mock.calls.filter((c) => String(c[0]).includes('INCOHERENT'))
+    expect(incoherent.length).toBeGreaterThanOrEqual(1)
+    expect(String(incoherent[0][0])).toMatch(/nounCount/)
+
+    // The heal: recount from canonical + re-stamp → next open is quiet.
+    await brain.repairIndex()
+    await brain.close()
+    const warn2 = vi.spyOn(prodLog, 'warn')
+    brain = await open()
+    const stillIncoherent = warn2.mock.calls.filter((c) => String(c[0]).includes('INCOHERENT'))
+    expect(stillIncoherent).toEqual([])
+  })
+
+  it('the one verifier handles both member modes', () => {
+    const rollup: FamilyStamp = {
+      family: 'x',
+      generation: 1,
+      committedAt: new Date().toISOString(),
+      sourceGeneration: 5,
+      members: { mode: 'rollup', invariants: { nounCount: 10 } }
+    }
+    expect(verifyFamilyStamp(rollup, 5, { nounCount: 10 })).toEqual({ state: 'coherent' })
+    expect(verifyFamilyStamp(rollup, 5, { nounCount: 11 }).state).toBe('incoherent')
+    expect(verifyFamilyStamp(rollup, 9, { nounCount: 10 })).toEqual({
+      state: 'behind',
+      stampSource: 5,
+      head: 9
+    })
+    expect(verifyFamilyStamp(rollup, 3, { nounCount: 10 }).state).toBe('incoherent') // ahead of head
+    expect(verifyFamilyStamp(null, 5, {})).toEqual({ state: 'absent' })
+
+    const enumerated: FamilyStamp = {
+      family: 'y',
+      generation: 1,
+      committedAt: new Date().toISOString(),
+      sourceGeneration: 2,
+      members: { mode: 'enumerated', files: [{ path: 'a.bin', bytes: 128 }] }
+    }
+    expect(verifyFamilyStamp(enumerated, 2, { 'a.bin': 128 })).toEqual({ state: 'coherent' })
+    expect(verifyFamilyStamp(enumerated, 2, { 'a.bin': 64 }).state).toBe('incoherent')
+    expect(verifyFamilyStamp(enumerated, 2, {}).state).toBe('incoherent') // missing member
+
+    // Rollup invariants may be STRING fingerprints (e.g. a per-tree SHA-256):
+    // strict equality either way; a type mismatch reads as incoherence.
+    const fingerprinted: FamilyStamp = {
+      family: 'z',
+      generation: 1,
+      committedAt: new Date().toISOString(),
+      sourceGeneration: 3,
+      members: { mode: 'rollup', invariants: { treeDigest: 'abc123', rows: 42 } }
+    }
+    expect(verifyFamilyStamp(fingerprinted, 3, { treeDigest: 'abc123', rows: 42 })).toEqual({
+      state: 'coherent'
+    })
+    expect(verifyFamilyStamp(fingerprinted, 3, { treeDigest: 'deadbeef', rows: 42 }).state).toBe(
+      'incoherent'
+    )
+    expect(verifyFamilyStamp(fingerprinted, 3, { treeDigest: 'abc123', rows: '42' }).state).toBe(
+      'incoherent' // type mismatch never passes
+    )
+  })
+})

tests/integration/fact-log-contracts.test.ts

@@ -0,0 +1,125 @@
+/**
+ * @module tests/integration/fact-log-contracts
+ * @description Pinned durability + stability contracts for the fact log.
+ *
+ * (1) FSYNC-BEFORE-ACK: an acknowledged write's fact survives an abrupt
+ *     process end (no flush, no close — reopen from disk).
+ *     - transact(): HOLDS TODAY — the fact is fsync'd before transact returns.
+ *     - single-op: PINNED AS `it.fails` — today's group-commit batches
+ *       DURABILITY (ack precedes the group fsync; a hard kill loses the fact
+ *       AND the generation together, coherently — the documented Model-B
+ *       contract, fine while the tree is authoritative). The destination
+ *       (ack-at-log) requires group commit to become LATENCY batching: the
+ *       ack waits for the shared fsync. When that lands, this pin flips red —
+ *       remove `.fails` and the contract is permanent. No cliff to discover.
+ *
+ * (2) SCAN STABILITY UNDER ROTATION: a scan handle opened before segment
+ *     rotation yields exactly its snapshot — byte-identical facts, no gaps,
+ *     no duplicates, and no bleed-in of facts appended after the snapshot.
+ *     (The reclaim-during-scan variant lands with fact-log compaction, which
+ *     does not exist yet — segments only rotate today, never reclaim.)
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy, type CommitFact } from '../../src/index.js'
+import { MemoryStorage } from '../../src/storage/adapters/memoryStorage.js'
+import { FactLog, type FactLogStorage } from '../../src/db/factLog.js'
+
+describe('fsync-before-ack contract (fact durability at the ack boundary)', () => {
+  let dir: string
+  let brain: any
+
+  const open = async () => {
+    const b: any = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true,
+      dimensions: 384
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-factack-'))
+    brain = await open()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('transact(): the fact is durable the moment the ack returns (kill-after-ack safe)', async () => {
+    const receipt = await brain.transact([
+      { op: 'add', type: 'document', metadata: { durable: 1 }, data: 'ack-at-commit' }
+    ])
+    // Abrupt end: no flush(), no close() — a new instance reads only disk.
+    brain = await open()
+    const facts: CommitFact[] = []
+    for await (const b of brain.scanFacts()!.batches()) facts.push(...b.facts)
+    expect(facts.some((f) => f.generation === receipt.generation)).toBe(true)
+  })
+
+  // PINNED (flips red when group commit becomes latency batching — then
+  // remove `.fails` and the ack-at-log contract is permanent on every path).
+  it.fails('single-op: the fact is durable the moment the ack returns (the ack-at-log target)', async () => {
+    await brain.add({ data: 'acked single-op', type: 'document', metadata: { n: 1 } })
+    const ackedHead = brain.scanFacts()!.headGeneration
+    // Abrupt end immediately after the ack — before any flush window.
+    brain = await open()
+    const facts: CommitFact[] = []
+    for await (const b of brain.scanFacts()!.batches()) facts.push(...b.facts)
+    expect(facts.some((f) => f.generation === ackedHead)).toBe(true)
+  })
+})
+
+describe('scan stability under rotation (the snapshot contract)', () => {
+  const UUID = (n: number): string => `00000000-0000-4000-8000-${String(n).padStart(12, '0')}`
+  const fact = (generation: number): CommitFact => ({
+    generation,
+    timestamp: 1_700_000_000_000 + generation,
+    ops: [
+      {
+        kind: 'noun',
+        id: UUID(generation),
+        // Padding makes each frame ~1KB so a small rotateBytes forces rotations.
+        record: { metadata: { noun: 'document', pad: 'x'.repeat(900), g: generation }, vector: null }
+      }
+    ]
+  })
+
+  it('a scan opened before rotations yields its exact snapshot — no gaps, dups, or bleed-in', async () => {
+    const mem: any = new MemoryStorage()
+    await mem.init()
+    const log = new FactLog(mem as FactLogStorage, { rotateBytes: 4096 }) // ~4 facts per segment
+    await log.open(0)
+    for (let g = 1; g <= 10; g++) await log.append(fact(g))
+    await log.sync()
+
+    // Open the snapshot, THEN keep appending — forcing further rotations.
+    const scan = log.scanFacts()
+    expect(scan.headGeneration).toBe(10)
+    for (let g = 11; g <= 25; g++) await log.append(fact(g))
+    await log.sync()
+    expect(log.headGeneration()).toBe(25)
+
+    const seen: number[] = []
+    for await (const batch of scan.batches()) {
+      for (const f of batch.facts) seen.push(f.generation)
+    }
+    // Exactly the snapshot: 1..10 in order, nothing appended-after bleeds in.
+    expect(seen).toEqual([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
+    expect(scan.summary().factsYielded).toBe(10)
+
+    // And a fresh scan sees everything, across all rotated segments.
+    const all: number[] = []
+    for await (const batch of log.scanFacts().batches()) {
+      for (const f of batch.facts) all.push(f.generation)
+    }
+    expect(all).toEqual(Array.from({ length: 25 }, (_, i) => i + 1))
+    expect(log.segmentPaths().length).toBeGreaterThanOrEqual(2) // rotations actually happened
+  })
+})

tests/integration/fact-log-dual-write.test.ts

@@ -0,0 +1,219 @@
+/**
+ * @module tests/integration/fact-log-dual-write
+ * @description The generation fact log end-to-end through real commits: every
+ * committed generation (single-op AND transact) appends its AFTER-IMAGE fact
+ * at the commit point; removals append body-less tombstones; an aborted
+ * transaction leaves no fact; facts survive reopen and continue monotonically;
+ * the scan surface (brain.scanFacts) carries the frozen telemetry shape; and
+ * the fact-log namespace is protected against prefix-nuking.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy, ProtectedArtifactError, type CommitFact } from '../../src/index.js'
+
+async function allFacts(brain: any): Promise<CommitFact[]> {
+  const scan = brain.scanFacts()
+  expect(scan).not.toBeNull()
+  const facts: CommitFact[] = []
+  for await (const batch of scan!.batches()) facts.push(...batch.facts)
+  return facts
+}
+
+describe('fact log dual-write (memory adapter)', () => {
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, silent: true, dimensions: 384 })
+    await brain.init()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+  })
+
+  it('every single-op write appends its after-image fact; a remove appends a tombstone', async () => {
+    const id = await brain.add({ data: 'first', type: 'document', metadata: { rev: 1 } })
+    await brain.update({ id, metadata: { rev: 2 } })
+    await brain.remove(id)
+
+    const facts = await allFacts(brain)
+    // add + update + remove each committed a generation (the remove may span
+    // cascade ops but is ONE generation). Facts are monotonic.
+    const gens = facts.map((f) => f.generation)
+    expect([...gens].sort((a, b) => a - b)).toEqual(gens)
+    expect(facts.length).toBeGreaterThanOrEqual(3)
+
+    // The add fact carries the after-image of the new entity.
+    const addFact = facts.find((f) => f.ops.some((op) => op.id === id && op.record !== null))
+    expect(addFact).toBeDefined()
+
+    // The remove fact carries a body-less tombstone for the id.
+    const removeFact = facts[facts.length - 1]
+    const tombstone = removeFact.ops.find((op) => op.id === id)
+    expect(tombstone).toBeDefined()
+    expect(tombstone!.record).toBeNull()
+    expect(tombstone!.kind).toBe('noun')
+  })
+
+  it('the update fact holds the NEW state (after-image, not before)', async () => {
+    const id = await brain.add({ data: 'versioned', type: 'document', metadata: { v: 'old' } })
+    await brain.update({ id, metadata: { v: 'new' } })
+
+    const facts = await allFacts(brain)
+    const updateFact = facts[facts.length - 1]
+    const op = updateFact.ops.find((o) => o.id === id)!
+    expect(op.record).not.toBeNull()
+    expect((op.record!.metadata as any).v).toBe('new')
+  })
+
+  it('a transact commits ONE fact carrying all its ops, with meta', async () => {
+    const receipt = await brain.transact(
+      [
+        { op: 'add', type: 'document', metadata: { part: 1 }, data: 'a' },
+        { op: 'add', type: 'document', metadata: { part: 2 }, data: 'b' }
+      ],
+      { meta: { source: 'batch-import' } }
+    )
+
+    const facts = await allFacts(brain)
+    const txFact = facts.find((f) => f.generation === receipt.generation)
+    expect(txFact).toBeDefined()
+    expect(txFact!.ops.filter((op) => op.kind === 'noun').length).toBeGreaterThanOrEqual(2)
+    expect(txFact!.meta).toEqual({ source: 'batch-import' })
+  })
+
+  it('an aborted transact leaves NO fact (absent = never committed)', async () => {
+    const id = await brain.add({ data: 'cas target', type: 'document', metadata: { n: 1 } })
+    const before = (await allFacts(brain)).length
+
+    await expect(
+      brain.transact([{ op: 'update', id, ifRev: 999, metadata: { n: 2 } }])
+    ).rejects.toThrow()
+
+    const after = await allFacts(brain)
+    expect(after.length).toBe(before)
+  })
+
+  it('fact generations line up with the transaction log', async () => {
+    await brain.add({ data: 'x', type: 'document', metadata: {} })
+    await brain.add({ data: 'y', type: 'document', metadata: {} })
+    await brain.flush()
+
+    const facts = await allFacts(brain)
+    const logGens = new Set((await brain.transactionLog()).map((e: any) => e.generation))
+    for (const f of facts) {
+      expect(logGens.has(f.generation)).toBe(true)
+    }
+  })
+
+  it('the storage fact-scan capability serves a provider holding only `storage`', async () => {
+    // An index provider receives `storage` — never the brain — and reaches the
+    // fact log through the host-wired capability (it must never construct its
+    // own fact-log reader: the log's open path is writer-side).
+    const id = await brain.add({ data: 'via storage', type: 'document', metadata: { s: 1 } })
+    await brain.remove(id)
+
+    const storage = brain.storage
+    expect(typeof storage.scanFacts).toBe('function')
+    expect(storage.factLogHeadGeneration()).toBe(brain.scanFacts()!.headGeneration)
+
+    const viaStorage: CommitFact[] = []
+    for await (const b of storage.scanFacts()!.batches()) viaStorage.push(...b.facts)
+    const viaBrain: CommitFact[] = []
+    for await (const b of brain.scanFacts()!.batches()) viaBrain.push(...b.facts)
+    expect(viaStorage.map((f) => f.generation)).toEqual(viaBrain.map((f) => f.generation))
+    expect(storage.factSegmentPaths()).toEqual(brain.factSegmentPaths())
+  })
+
+  it('scan telemetry carries the frozen shape end-to-end', async () => {
+    for (let i = 0; i < 5; i++) await brain.add({ data: `t${i}`, type: 'document', metadata: { i } })
+
+    const scan = brain.scanFacts({ batchSize: 2 })!
+    expect(scan.headGeneration).toBeGreaterThanOrEqual(5)
+    expect(scan.approxFactCount).toBeGreaterThanOrEqual(5)
+    let batches = 0
+    for await (const b of scan.batches()) {
+      batches++
+      expect(b.factCount).toBe(b.facts.length)
+      expect(b.firstGeneration).toBe(b.facts[0].generation)
+      expect(b.lastGeneration).toBe(b.facts[b.facts.length - 1].generation)
+      expect(b.byteSize).toBeGreaterThan(0)
+      expect(typeof b.segmentId).toBe('string')
+    }
+    expect(batches).toBeGreaterThan(1)
+    expect(scan.summary().factsYielded).toBe(scan.approxFactCount)
+  })
+})
+
+describe('fact log dual-write (filesystem adapter — durability + protection)', () => {
+  let dir: string
+  let brain: any
+
+  const open = async () => {
+    const b: any = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true,
+      dimensions: 384
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-factlog-'))
+    brain = await open()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('facts survive close + reopen and appends continue monotonically', async () => {
+    const id = await brain.add({ data: 'persist me', type: 'document', metadata: { k: 1 } })
+    await brain.remove(id)
+    await brain.close()
+
+    brain = await open()
+    const facts = await allFacts(brain)
+    expect(facts.length).toBeGreaterThanOrEqual(2)
+    const headBefore = facts[facts.length - 1].generation
+
+    await brain.add({ data: 'after reopen', type: 'document', metadata: { k: 2 } })
+    const facts2 = await allFacts(brain)
+    expect(facts2[facts2.length - 1].generation).toBeGreaterThan(headBefore)
+  })
+
+  it('the fact segments exist on disk under _generations/facts/ with zero-padded names', async () => {
+    await brain.add({ data: 'on disk', type: 'document', metadata: {} })
+    await brain.flush()
+    const factsDir = path.join(dir, '_generations', 'facts')
+    const files = fs.readdirSync(factsDir)
+    // The manifest rides the store's JSON object discipline (gzip on disk).
+    expect(files.some((f) => f.startsWith('manifest.json'))).toBe(true)
+    const segs = files.filter((f) => /^seg-\d{20}\.bfl$/.test(f))
+    expect(segs.length).toBeGreaterThanOrEqual(1)
+  })
+
+  it('the fact-log namespace is PROTECTED: a prefix-nuke is refused', async () => {
+    await brain.add({ data: 'protected', type: 'document', metadata: {} })
+    await expect(brain.storage.removeRawPrefix('_generations/facts')).rejects.toBeInstanceOf(
+      ProtectedArtifactError
+    )
+    // Per-generation history cleanup remains unaffected (no false intersect).
+    await expect(brain.storage.removeRawPrefix('_generations/999999')).resolves.toBeUndefined()
+  })
+
+  it('transact facts are durable-on-return (no flush needed before reopen)', async () => {
+    const receipt = await brain.transact([
+      { op: 'add', type: 'document', metadata: { durable: true }, data: 'tx' }
+    ])
+    // Simulate an abrupt end: no flush(), no close() — reopen from disk.
+    brain = await open()
+    const facts = await allFacts(brain)
+    expect(facts.some((f) => f.generation === receipt.generation)).toBe(true)
+  })
+})

tests/integration/graph-audit.test.ts

@@ -0,0 +1,164 @@
+/**
+ * @module tests/integration/graph-audit
+ * @description brain.auditGraph() — the read-only graph-truth instrument.
+ * Laws under test:
+ * (1) a healthy brain audits COHERENT: every canonical verb is returned by the
+ *     read path of its source, endpoints exist, no ghosts;
+ * (2) design-hidden edges (internal/system visibility) are counted separately
+ *     and never misclassified as index loss;
+ * (3) a verb whose endpoint entity was destroyed at the storage layer (the
+ *     scar class) is flagged as a dangling endpoint, loudly;
+ * (4) the classification core flags present-but-invisible and ghost edges
+ *     exactly (exercised via injected seams — manufacturing a genuinely stale
+ *     adjacency index end-to-end would require corrupting internals the
+ *     public API rightly refuses to corrupt).
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+import { runGraphAudit, type AuditVerbRecord } from '../../src/graph/graphAudit.js'
+
+describe('brain.auditGraph() — graph-truth audit', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-graph-audit-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  async function seedGraph(): Promise<{ ids: string[]; verbIds: string[] }> {
+    const ids: string[] = []
+    for (let i = 0; i < 5; i++) {
+      ids.push(
+        await brain.add({
+          data: `entity ${i}`,
+          type: NounType.Concept,
+          metadata: { n: i }
+        })
+      )
+    }
+    const verbIds: string[] = []
+    verbIds.push(await brain.relate({ from: ids[0], to: ids[1], type: VerbType.RelatedTo }))
+    verbIds.push(await brain.relate({ from: ids[0], to: ids[2], type: VerbType.Contains }))
+    verbIds.push(await brain.relate({ from: ids[1], to: ids[3], type: VerbType.DependsOn }))
+    verbIds.push(await brain.relate({ from: ids[3], to: ids[4], type: VerbType.RelatedTo }))
+    return { ids, verbIds }
+  }
+
+  it('audits a healthy brain as coherent, with exact counts', async () => {
+    const { ids } = await seedGraph()
+    const report = await brain.auditGraph()
+
+    expect(report.coherent).toBe(true)
+    expect(report.verbsInCanonical).toBe(4)
+    expect(report.entitiesInCanonical).toBeGreaterThanOrEqual(ids.length) // VFS root etc. may add system nouns
+    expect(report.sourcesChecked).toBe(3) // ids[0], ids[1], ids[3]
+    expect(report.missingFromReadsCount).toBe(0)
+    expect(report.danglingEndpointsCount).toBe(0)
+    expect(report.readOnlyCount).toBe(0)
+    expect(report.truncatedExamples).toBe(false)
+  })
+
+  it('counts design-hidden edges separately and stays coherent', async () => {
+    const { ids } = await seedGraph()
+    await brain.relate({
+      from: ids[2],
+      to: ids[4],
+      type: VerbType.RelatedTo,
+      visibility: 'internal'
+    })
+
+    const report = await brain.auditGraph()
+    expect(report.coherent).toBe(true) // hidden-by-design is NOT a discrepancy
+    expect(report.verbsInCanonical).toBe(5)
+    expect(report.visibilityHiddenCount).toBeGreaterThanOrEqual(1)
+  })
+
+  it('flags a destroyed endpoint as a dangling verb (the scar class)', async () => {
+    const { ids } = await seedGraph()
+    // Destroy ids[4] at the STORAGE layer (bypassing remove(), which would
+    // also delete its verbs) — the historical partial-delete scar shape.
+    await brain.storage.deleteNoun(ids[4])
+
+    const report = await brain.auditGraph()
+    expect(report.coherent).toBe(false)
+    expect(report.danglingEndpointsCount).toBe(1)
+    expect(report.danglingEndpoints[0].to).toBe(ids[4])
+    expect(report.danglingEndpoints[0].missingEnd).toBe('to')
+  })
+})
+
+describe('runGraphAudit classification core (injected seams)', () => {
+  const verb = (id: string, from: string, to: string): AuditVerbRecord => ({
+    id,
+    type: 'relatedTo',
+    sourceId: from,
+    targetId: to
+  })
+
+  const deps = (opts: {
+    nouns: string[]
+    verbs: AuditVerbRecord[]
+    reads: Record<string, string[]> // sourceId -> verb ids the read path returns
+  }) => ({
+    eachNounId: async (consume: (id: string) => void) => {
+      for (const id of opts.nouns) consume(id)
+    },
+    eachVerb: async (consume: (v: AuditVerbRecord) => void) => {
+      for (const v of opts.verbs) consume(v)
+    },
+    readRelationsFrom: async (sourceId: string) =>
+      (opts.reads[sourceId] ?? []).map((id) => ({ id }))
+  })
+
+  it('flags a canonical verb the read path omits — present but invisible', async () => {
+    const report = await runGraphAudit(
+      deps({
+        nouns: ['A', 'B', 'C'],
+        verbs: [verb('v1', 'A', 'B'), verb('v2', 'A', 'C')],
+        reads: { A: ['v1'] } // v2 exists canonically but reads miss it
+      })
+    )
+    expect(report.coherent).toBe(false)
+    expect(report.missingFromReadsCount).toBe(1)
+    expect(report.missingFromReads[0].verbId).toBe('v2')
+  })
+
+  it('flags a read-path edge with no canonical record — a ghost', async () => {
+    const report = await runGraphAudit(
+      deps({
+        nouns: ['A', 'B'],
+        verbs: [verb('v1', 'A', 'B')],
+        reads: { A: ['v1', 'ghost-9'] }
+      })
+    )
+    expect(report.coherent).toBe(false)
+    expect(report.readOnlyCount).toBe(1)
+    expect(report.readOnlyVerbIds).toEqual(['ghost-9'])
+  })
+
+  it('caps example lists but keeps counts exact, and says so', async () => {
+    const verbs = Array.from({ length: 10 }, (_, i) => verb(`v${i}`, 'A', 'B'))
+    const report = await runGraphAudit(
+      deps({ nouns: ['A', 'B'], verbs, reads: { A: [] } }),
+      { maxExamples: 3 }
+    )
+    expect(report.missingFromReadsCount).toBe(10)
+    expect(report.missingFromReads.length).toBe(3)
+    expect(report.truncatedExamples).toBe(true)
+  })
+})

tests/integration/history-repacking.test.ts

@@ -0,0 +1,186 @@
+/**
+ * @module tests/integration/history-repacking
+ * @description The D1+D3 two-tier history lifecycle end-to-end on a real
+ * brain. Laws: (1) repacking is RE-REPRESENTATION — after folding, every
+ * asOf() read below the fold boundary answers exactly as before, across a
+ * cold reopen; (2) folded per-generation directories are physically gone
+ * (the file-count cure is real, not cosmetic); (3) repack + reclaim compose:
+ * bounded retention after repacking drops whole segments and asOf below the
+ * horizon throws GenerationCompactedError; (4) repackHistory is explicit
+ * API and time-bounded (spent budget = consistent no-op).
+ *
+ * Uses a tiny REPACK_LIVE_WINDOW override so a small history has a cold
+ * tier at all (the production window is 1024).
+ */
+import { describe, it, expect, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as path from 'node:path'
+import * as os from 'node:os'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+import { GenerationStore } from '../../src/db/generationStore.js'
+import { GenerationCompactedError } from '../../src/db/errors.js'
+import { SEGMENTS_PREFIX } from '../../src/db/generationSegments.js'
+
+const stub = async (text: string): Promise<number[]> => {
+  const h = text.split('').reduce((a, c) => a + c.charCodeAt(0), 0)
+  return new Array(384).fill(0).map((_, i) => Math.sin(h + i))
+}
+
+const openBrain = async (dir: string): Promise<Brainy> => {
+  const brain = new Brainy({
+    requireSubtype: false,
+    storage: { type: 'filesystem', path: dir },
+    embeddingFunction: stub
+  })
+  await brain.init()
+  return brain
+}
+
+describe('history repacking — the two-tier lifecycle', () => {
+  const dirs: string[] = []
+  const tempDir = (): string => {
+    const d = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-repack-'))
+    dirs.push(d)
+    return d
+  }
+  const originalWindow = GenerationStore.REPACK_LIVE_WINDOW
+
+  afterEach(() => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = originalWindow
+    for (const d of dirs.splice(0)) {
+      try {
+        fs.rmSync(d, { recursive: true, force: true })
+      } catch {
+        /* best effort */
+      }
+    }
+  })
+
+  it('repack preserves every historical read across cold reopen; folded dirs are gone', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 3
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+
+    const id = await brain.add({
+      data: 'versioned-entity',
+      type: NounType.Document,
+      metadata: { v: 0 }
+    })
+    for (let v = 1; v <= 10; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+
+    // Ground truth BEFORE repacking: capture asOf views for early generations.
+    const before: Record<number, number> = {}
+    for (const g of [2, 4, 6]) {
+      const db = await brain.asOf(g)
+      before[g] = (await db.get(id))?.metadata?.v as number
+      await db.release()
+    }
+
+    const result = await brain.repackHistory()
+    expect(result.foldedGenerations).toBeGreaterThan(0)
+    expect(result.segmentsCreated).toBeGreaterThan(0)
+
+    // The folded per-generation directories are PHYSICALLY gone…
+    const genDirs = fs
+      .readdirSync(path.join(dir, '_generations'), { withFileTypes: true })
+      .filter((e) => e.isDirectory() && /^\d+$/.test(e.name)).length
+    expect(genDirs).toBeLessThanOrEqual(4) // live window (3) + at most the newest
+    // …and the segment tier exists (the filesystem adapter stores objects
+    // gzipped, so the manifest may live at either spelling).
+    const segDir = path.join(dir, SEGMENTS_PREFIX)
+    expect(
+      fs.existsSync(path.join(segDir, 'manifest.json')) ||
+        fs.existsSync(path.join(segDir, 'manifest.json.gz'))
+    ).toBe(true)
+    expect(fs.readdirSync(segDir).some((f) => f.endsWith('.bgs'))).toBe(true)
+
+    // Same asOf answers from the packed tier, same process…
+    for (const g of [2, 4, 6]) {
+      const db = await brain.asOf(g)
+      expect((await db.get(id))?.metadata?.v).toBe(before[g])
+      await db.release()
+    }
+    await brain.close()
+
+    // …and across a COLD REOPEN (manifest discovery, no live dirs to list).
+    const reopened = await openBrain(dir)
+    for (const g of [2, 4, 6]) {
+      const db = await reopened.asOf(g)
+      expect((await db.get(id))?.metadata?.v).toBe(before[g])
+      await db.release()
+    }
+    expect((await reopened.get(id))?.metadata?.v).toBe(10) // live state untouched
+    await reopened.close()
+  })
+
+  it('repack + bounded reclaim compose: whole segments drop, horizon is loud', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'reclaim-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 8; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+    await brain.repackHistory()
+
+    // Reclaim down to the 3 newest generations — packed segments below the
+    // horizon drop whole; asOf below throws loudly.
+    const res = await brain.compactHistory({ maxGenerations: 3 })
+    expect(res.removedGenerations).toBeGreaterThan(0)
+    await expect(brain.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    expect((await brain.get(id))?.metadata?.v).toBe(8)
+    await brain.close()
+  })
+
+  it('generationDigest: reopen-stable, divergence-sensitive, loud below the horizon', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'digest-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 6; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+    await brain.repackHistory()
+
+    const gen = brain.generation()
+    const atHead = await brain.generationDigest(gen)
+    const atMid = await brain.generationDigest(3)
+    expect(atHead).toMatch(/^[0-9a-f]{8}$/)
+    expect(atMid).not.toBe(atHead) // more history ⇒ different digest
+    await brain.close()
+
+    // Reopen-stable: same history, same digests (packed prefix stability).
+    const reopened = await openBrain(dir)
+    expect(await reopened.generationDigest(gen)).toBe(atHead)
+    expect(await reopened.generationDigest(3)).toBe(atMid)
+
+    // New history diverges the head digest.
+    await reopened.update({ id, metadata: { v: 7 } })
+    await reopened.flush()
+    expect(await reopened.generationDigest(reopened.generation())).not.toBe(atHead)
+
+    // Below the horizon: LOUD, never a silent pin of reclaimed history.
+    await reopened.compactHistory({ maxGenerations: 2 })
+    await expect(reopened.generationDigest(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    await reopened.close()
+  })
+
+  it('a spent time budget is a consistent no-op; the next pass resumes', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'budget-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 6; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+
+    const bounded = await brain.repackHistory({ timeBudgetMs: 0 })
+    expect(bounded).toEqual({ foldedGenerations: 0, segmentsCreated: 0 })
+
+    const resumed = await brain.repackHistory()
+    expect(resumed.foldedGenerations).toBeGreaterThan(0)
+    const db = await brain.asOf(3)
+    expect((await db.get(id))?.metadata?.v).toBeDefined()
+    await db.release()
+    await brain.close()
+  })
+})

tests/integration/ifabsent-upsert-blob-concurrency.test.ts

@@ -0,0 +1,212 @@
+/**
+ * @module tests/integration/ifabsent-upsert-blob-concurrency
+ * @description The 8.0.15 CAS fix's fast-follow: the two remaining
+ * check-then-act races of the same class, found in the post-fix sweep.
+ *
+ * 1. `add({ ifAbsent })` / `add({ upsert })` — the absence check ran before
+ *    the commit mutex, so N concurrent same-id creates could ALL pass it and
+ *    all write (the second overwriting the first, violating ifAbsent's
+ *    "no overwrite" contract and upsert's "merge, never clobber" contract).
+ *    Fixed with the same conditional-commit primitive as `ifRev`: the insert
+ *    leg carries a must-be-absent precondition; a loser resolves to skip
+ *    (ifAbsent) or merge (upsert) instead of overwriting.
+ *
+ * 2. `BlobStorage` reference counts — `write()`'s dedup decision and
+ *    `delete()`'s decrement-then-remove were unserialized read-modify-writes
+ *    over `blob-meta:<hash>`; concurrent same-content writes could lose
+ *    references, turning a later delete into premature removal of bytes
+ *    another file still needs. Fixed with a per-hash mutex.
+ *
+ * Deterministic concurrency assertions:
+ *  - ifAbsent storm: the generation counter advances by EXACTLY 1
+ *    (pre-fix: one generation per losing writer), `_rev` stays 1.
+ *  - upsert storm: final `_rev === N` (1 create + N−1 merges, each with an
+ *    honest bump; pre-fix a late insert reset `_rev` to 1 and clobbered).
+ *  - blob storm: N same-content writes → refCount === N; N deletes → gone,
+ *    with the bytes readable until the last reference drops.
+ */
+import { describe, it, expect, beforeAll, afterAll } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+import { BlobStorage } from '../../src/storage/blobStorage.js'
+
+describe('ifAbsent/upsert insert race + blob refCount (conditional commit fast-follow)', () => {
+  let dir: string
+  let brain: any
+  let seq = 0
+  const freshId = (): string =>
+    `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+  beforeAll(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-upsert-race-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterAll(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('N concurrent add({ifAbsent}) → exactly ONE write (generation +1), no overwrite', async () => {
+    const id = freshId()
+    const genBefore = brain.generationStore.generation()
+
+    const ids = await Promise.all(
+      Array.from({ length: 8 }, (_, i) =>
+        brain.add({
+          id,
+          ifAbsent: true,
+          data: `contender ${i}`,
+          type: NounType.Thing,
+          metadata: { writer: i }
+        })
+      )
+    )
+
+    // Every caller resolves to the same canonical id.
+    expect(new Set(ids).size).toBe(1)
+
+    // THE contract: exactly one write landed. Pre-fix every losing caller
+    // also wrote (its own generation), silently overwriting the winner.
+    expect(brain.generationStore.generation()).toBe(genBefore + 1)
+
+    const after = await brain.get(id)
+    expect(after._rev).toBe(1)
+    expect(typeof after.metadata.writer).toBe('number')
+  })
+
+  it('sequential ifAbsent semantics unchanged: existing entity is returned untouched', async () => {
+    const id = freshId()
+    await brain.add({ id, data: 'original', type: NounType.Thing, metadata: { keep: true } })
+    const returned = await brain.add({
+      id,
+      ifAbsent: true,
+      data: 'impostor',
+      type: NounType.Thing,
+      metadata: { keep: false }
+    })
+    expect(returned).toBe(id)
+    const after = await brain.get(id)
+    expect(after.metadata.keep).toBe(true)
+    expect(after.data).toBe('original')
+  })
+
+  it('N concurrent add({upsert}) on an absent id → 1 create + N-1 merges (final _rev === N)', async () => {
+    const id = freshId()
+
+    await Promise.all(
+      Array.from({ length: 8 }, (_, i) =>
+        brain.add({
+          id,
+          upsert: true,
+          data: 'shared upsert target',
+          type: NounType.Thing,
+          metadata: { [`k${i}`]: true }
+        })
+      )
+    )
+
+    const after = await brain.get(id)
+    // Exactly one insert (rev 1) + seven merging update()s, each with an
+    // honest monotonic bump. Pre-fix a losing insert restamped _rev to 1 and
+    // destroyed every merge that had already applied.
+    expect(after._rev).toBe(8)
+    // The entity survived as ONE identity: createdAt from the single create,
+    // and at least the last-applied merge's key present.
+    expect(Object.keys(after.metadata).filter((k) => k.startsWith('k')).length)
+      .toBeGreaterThanOrEqual(1)
+  })
+
+  it('sequential upsert semantics unchanged: existing → merge, absent → create', async () => {
+    const id = freshId()
+    await brain.add({ id, upsert: true, data: 'v1', type: NounType.Thing, metadata: { a: 1 } })
+    expect((await brain.get(id))._rev).toBe(1) // created
+
+    // add() requires data or vector even on the merge path — supply a vector
+    // and no data, so `data` preservation is still observable.
+    const vec = Array.from({ length: 384 }, (_, i) => (i % 7) / 7 - 0.5)
+    await brain.add({ id, upsert: true, vector: vec, type: NounType.Thing, metadata: { b: 2 } })
+    const after = await brain.get(id)
+    expect(after._rev).toBe(2) // merged, not overwritten
+    expect(after.metadata.a).toBe(1)
+    expect(after.metadata.b).toBe(2)
+    expect(after.data).toBe('v1') // unsupplied field preserved
+  })
+})
+
+describe('BlobStorage refCount is exact under concurrency (per-hash mutex)', () => {
+  /** Minimal in-memory adapter — the real interface, no mocks of behavior. */
+  function memAdapter() {
+    const kv = new Map<string, Buffer>()
+    return {
+      get: async (k: string) => kv.get(k),
+      put: async (k: string, v: Buffer) => void kv.set(k, v),
+      delete: async (k: string) => void kv.delete(k),
+      list: async (prefix: string) =>
+        [...kv.keys()].filter((k) => k.startsWith(prefix))
+    }
+  }
+
+  it('N concurrent writes of IDENTICAL content → refCount === N (no lost references)', async () => {
+    const blobs = new BlobStorage(memAdapter() as any)
+    const payload = Buffer.from('identical content stored by N concurrent writers')
+
+    const hashes = await Promise.all(
+      Array.from({ length: 10 }, () => blobs.write(payload))
+    )
+    expect(new Set(hashes).size).toBe(1)
+    const meta = await blobs.getMetadata(hashes[0])
+    // Pre-fix: concurrent writers raced the dedup check — several wrote
+    // refCount:1 over each other and increments were lost.
+    expect(meta?.refCount).toBe(10)
+  })
+
+  it('references drop exactly; bytes survive live-zero (temporal immutability) until reclaim', async () => {
+    const blobs = new BlobStorage(memAdapter() as any)
+    const payload = Buffer.from('shared bytes, two referencing files')
+
+    const hash = await blobs.write(payload)
+    await blobs.write(payload) // second reference (concurrent-equivalent path)
+
+    await blobs.release(hash) // drop one reference
+    expect((await blobs.read(hash)).toString()).toBe(payload.toString()) // still readable
+    expect((await blobs.getMetadata(hash))?.refCount).toBe(1)
+
+    await blobs.release(hash) // last LIVE reference — bytes still exist (history may need them)
+    expect(await blobs.has(hash)).toBe(true)
+    expect((await blobs.getMetadata(hash))?.refCount).toBe(0)
+
+    // Reclamation is compaction's job: zero-zero → physically removed.
+    expect(await blobs.reclaimIfUnreferenced(hash)).toBe(true)
+    expect(await blobs.has(hash)).toBe(false)
+    await expect(blobs.read(hash)).rejects.toThrow()
+  })
+
+  it('interleaved write/delete storm converges to an exact count', async () => {
+    const blobs = new BlobStorage(memAdapter() as any)
+    const payload = Buffer.from('storm payload')
+    const hash = BlobStorage.hash(payload)
+
+    // 12 writes and 5 releases racing: net 7 references, blob alive.
+    await Promise.all([
+      ...Array.from({ length: 12 }, () => blobs.write(payload)),
+      ...Array.from({ length: 5 }, () => blobs.release(hash))
+    ])
+    const meta = await blobs.getMetadata(hash)
+    // Releases against a not-yet-written hash floor at zero, so the net can
+    // only be >= 12 - 5. The exactness we require: counts are never LOST
+    // (each landed write is represented).
+    expect(meta?.refCount).toBeGreaterThanOrEqual(7)
+    expect(await blobs.has(hash)).toBe(true)
+  })
+})

tests/integration/ifrev-concurrent-cas.test.ts

@@ -0,0 +1,187 @@
+/**
+ * @module tests/integration/ifrev-concurrent-cas
+ * @description Regression for the P0 report that `update({ ifRev })` CAS was
+ * not atomic under concurrent in-process calls: N concurrent updates all
+ * carrying the same `ifRev` ALL fulfilled (0 conflicts, last-writer-wins) —
+ * the check ran before the commit mutex, so interleaved callers all passed it
+ * before any apply landed. In production that silently lost 7 of 8
+ * ifRev-guarded ledger writes and let two workers both "acquire" an advisory
+ * lock built on exactly-one-winner semantics.
+ *
+ * The fix is a conditional commit: the generation store's `precommit` hook
+ * re-verifies `ifRev` against the authoritative before-image UNDER the commit
+ * mutex, atomically with the apply (the per-record analogue of
+ * `ifAtGeneration`, which always ran there). These tests pin:
+ *  - exactly-one-winner for N concurrent same-rev `update({ ifRev })`
+ *  - the same for `transact()` per-op ifRev (whole batch rejected)
+ *  - `_rev` monotonicity under concurrent plain updates (no ifRev)
+ *  - the CAS retry loop converging exactly (the ledger/credit-consume shape)
+ *  - sequential conflict behavior unchanged
+ */
+import { describe, it, expect, beforeAll, afterAll } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+
+const isRevConflict = (e: unknown): boolean =>
+  (e as Error)?.name === 'RevisionConflictError' ||
+  String(e).includes('RevisionConflict')
+
+describe('ifRev CAS is atomic under concurrency (conditional commit)', () => {
+  let dir: string
+  let brain: any
+  let seq = 0
+  const freshId = (): string =>
+    `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+  beforeAll(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-cas-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterAll(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('N concurrent update({ifRev}) → exactly 1 winner, N-1 RevisionConflictError', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'contended entity',
+      type: NounType.Thing,
+      metadata: { credits: 8 }
+    })
+    const rev = (await brain.get(id))._rev
+
+    const results = await Promise.allSettled(
+      Array.from({ length: 8 }, (_, i) =>
+        brain.update({ id, metadata: { writer: i }, merge: false, ifRev: rev })
+      )
+    )
+
+    const wins = results.filter((r) => r.status === 'fulfilled')
+    const conflicts = results.filter(
+      (r) => r.status === 'rejected' && isRevConflict(r.reason)
+    )
+    expect(wins.length).toBe(1)
+    expect(conflicts.length).toBe(7)
+
+    // Exactly one bump; the surviving metadata belongs to the single winner.
+    const after = await brain.get(id)
+    expect(after._rev).toBe(rev + 1)
+    expect(typeof after.metadata.writer).toBe('number')
+  })
+
+  it('transact() per-op ifRev → exactly 1 batch wins, others rejected whole', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'transact contended',
+      type: NounType.Thing,
+      metadata: { state: 'initial' }
+    })
+    const rev = (await brain.get(id))._rev
+
+    const results = await Promise.allSettled(
+      Array.from({ length: 6 }, (_, i) =>
+        brain.transact([
+          { op: 'update', id, metadata: { batch: i }, merge: false, ifRev: rev }
+        ])
+      )
+    )
+    expect(results.filter((r) => r.status === 'fulfilled').length).toBe(1)
+    expect(
+      results.filter((r) => r.status === 'rejected' && isRevConflict(r.reason))
+        .length
+    ).toBe(5)
+    expect((await brain.get(id))._rev).toBe(rev + 1)
+  })
+
+  it('_rev is monotonic under concurrent plain updates (no ifRev): N updates → +N', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'plain concurrent updates',
+      type: NounType.Thing,
+      metadata: { v: 0 }
+    })
+    const rev = (await brain.get(id))._rev
+
+    await Promise.all(
+      Array.from({ length: 8 }, (_, i) => brain.update({ id, metadata: { v: i } }))
+    )
+    // Every applied update gets its own honest bump (previously all stamped
+    // the same stale rev+1 they computed before the commit).
+    expect((await brain.get(id))._rev).toBe(rev + 8)
+  })
+
+  it('the CAS retry loop converges exactly (the ledger shape: 8 workers, 8 consumes)', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'credit budget',
+      type: NounType.Thing,
+      metadata: { credits: 8 }
+    })
+
+    // Each worker: read → CAS-decrement → retry on conflict. The docs' documented
+    // pattern; with atomic CAS this MUST land on exactly 0 with zero lost updates.
+    const consumeOne = async (): Promise<void> => {
+      for (let attempt = 0; attempt < 50; attempt++) {
+        const cur = await brain.get(id)
+        if (cur.metadata.credits <= 0) throw new Error('budget exhausted')
+        try {
+          await brain.update({
+            id,
+            metadata: { ...cur.metadata, credits: cur.metadata.credits - 1 },
+            merge: false,
+            ifRev: cur._rev
+          })
+          return
+        } catch (e) {
+          if (!isRevConflict(e)) throw e
+        }
+      }
+      throw new Error('no convergence after 50 attempts')
+    }
+
+    await Promise.all(Array.from({ length: 8 }, () => consumeOne()))
+    expect((await brain.get(id)).metadata.credits).toBe(0)
+  })
+
+  it('sequential conflict behavior is unchanged (stale ifRev throws, fresh succeeds)', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'sequential control',
+      type: NounType.Thing,
+      metadata: { n: 1 }
+    })
+    const rev = (await brain.get(id))._rev
+
+    await brain.update({ id, metadata: { n: 2 }, ifRev: rev })
+    await expect(
+      brain.update({ id, metadata: { n: 3 }, ifRev: rev })
+    ).rejects.toMatchObject({ name: 'RevisionConflictError' })
+    // Fresh rev succeeds.
+    const fresh = (await brain.get(id))._rev
+    await brain.update({ id, metadata: { n: 3 }, ifRev: fresh })
+    expect((await brain.get(id)).metadata.n).toBe(3)
+  })
+
+  it('forward-ref add+update of the same entity in one transact() batch still works', async () => {
+    const id = freshId()
+    await brain.transact([
+      { op: 'add', id, data: 'created in batch', type: NounType.Thing, metadata: { step: 1 } },
+      { op: 'update', id, metadata: { step: 2 } }
+    ])
+    const after = await brain.get(id)
+    expect(after.metadata.step).toBe(2)
+    expect(after._rev).toBe(2) // add stamps 1, in-batch update sequences to 2
+  })
+})

tests/integration/lens-consistency.test.ts

@@ -0,0 +1,138 @@
+/**
+ * @module tests/integration/lens-consistency
+ * @description The three metadata "lenses" over one corpus must agree with
+ * canonical ground truth id-for-id, warm AND after a cold reopen:
+ *   - combined:      find({ type: T, where: { subtype: S } })
+ *   - subtype-only:  find({ where: { subtype: S } })
+ *   - type-only:     find({ type: T })
+ * Ported from the fresh-brain probe that closed the type+subtype lens-drop
+ * investigation (a restored pre-8.2.2 torn capture had entities visible to the
+ * subtype-only lens but dropped by the combined lens — "0 of 2 migrated, all
+ * gates green"). The corpus is seeded through the REAL write API — never
+ * restored bytes — which is what made the original datapoint decisive. The
+ * invariants: every lens matches an unfiltered canonical scan exactly (no
+ * missing ids, no extras) and combined ⊆ subtype-only always holds.
+ */
+import { describe, it, expect, beforeAll, afterAll } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+
+/** The corpus: 7 (type, subtype) pairs, uneven counts, incl. the incident's 2-of-a-pair shape. */
+const CORPUS: Array<{ type: string; subtype: string; count: number }> = [
+  { type: 'proposition', subtype: 'decision', count: 2 }, // the incident shape: "0 of 2"
+  { type: 'concept', subtype: 'decision', count: 3 },
+  { type: 'task', subtype: 'decision', count: 2 },
+  { type: 'concept', subtype: 'action', count: 4 },
+  { type: 'message', subtype: 'note', count: 5 },
+  { type: 'message', subtype: 'ship', count: 3 },
+  { type: 'document', subtype: 'guide', count: 4 }
+]
+
+/** Canonical ground truth: unfiltered enumeration, post-filtered IN THE TEST. */
+async function groundTruth(
+  brain: any,
+  match: { type?: string; subtype?: string }
+): Promise<Set<string>> {
+  const ids = new Set<string>()
+  let cursor: string | undefined
+  for (;;) {
+    const page = await brain.storage.getNounsWithPagination({ limit: 500, cursor })
+    for (const noun of page.items) {
+      // Hydrated shape: `type`/`subtype` are TOP-LEVEL; `metadata` holds only
+      // custom user fields (vfsType is one — the VFS plumbing marker).
+      const n = noun as any
+      if (n.metadata?.vfsType) continue // VFS plumbing is not corpus
+      if (!n.type || !n.subtype) continue
+      if (match.type && n.type !== match.type) continue
+      if (match.subtype && n.subtype !== match.subtype) continue
+      ids.add(n.id)
+    }
+    if (!page.hasMore) break
+    cursor = page.nextCursor
+  }
+  return ids
+}
+
+const idSet = (results: Array<{ id: string }>): Set<string> => new Set(results.map((r) => r.id))
+
+/** Every lens vs ground truth, id-for-id, for every pair in the corpus. */
+async function assertAllLenses(brain: any): Promise<void> {
+  const types = [...new Set(CORPUS.map((c) => c.type))]
+  const subtypes = [...new Set(CORPUS.map((c) => c.subtype))]
+
+  for (const { type, subtype } of CORPUS) {
+    const combined = idSet(await brain.find({ type, where: { subtype }, limit: 1000 }))
+    const subtypeOnly = idSet(await brain.find({ where: { subtype }, limit: 1000 }))
+    const truthPair = await groundTruth(brain, { type, subtype })
+    const truthSubtype = await groundTruth(brain, { subtype })
+
+    expect([...combined].sort()).toEqual([...truthPair].sort()) // no drops, no extras
+    expect([...subtypeOnly].sort()).toEqual([...truthSubtype].sort())
+    for (const id of combined) expect(subtypeOnly.has(id)).toBe(true) // combined ⊆ subtype-only
+  }
+
+  for (const type of types) {
+    const typeOnly = idSet(await brain.find({ type, limit: 1000 }))
+    const truthType = await groundTruth(brain, { type })
+    expect([...typeOnly].sort()).toEqual([...truthType].sort())
+  }
+
+  // Count cross-check against the corpus definition itself.
+  for (const subtype of subtypes) {
+    const expected = CORPUS.filter((c) => c.subtype === subtype).reduce((s, c) => s + c.count, 0)
+    const got = (await brain.find({ where: { subtype }, limit: 1000 })).length
+    expect(got).toBe(expected)
+  }
+}
+
+describe('lens consistency — combined vs subtype-only vs canonical ground truth', () => {
+  let dir: string
+  let brain: any
+
+  beforeAll(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-lens-'))
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir }, silent: true, dimensions: 384 })
+    await brain.init()
+    // Seed through the REAL write API — never restored bytes.
+    let i = 0
+    for (const { type, subtype, count } of CORPUS) {
+      for (let k = 0; k < count; k++) {
+        await brain.add({ data: `${type} ${subtype} ${i++}`, type, subtype, metadata: { k } })
+      }
+    }
+    await brain.flush()
+  })
+  afterAll(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('WARM: all lenses agree with ground truth id-for-id', async () => {
+    await assertAllLenses(brain)
+  })
+
+  it('COLD REOPEN: all lenses still agree after close + reopen from disk', async () => {
+    await brain.close()
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir }, silent: true, dimensions: 384 })
+    await brain.init()
+    await assertAllLenses(brain)
+  })
+
+  it('after an update() flips type AND subtype, every lens tracks the move exactly', async () => {
+    // The historical cross-bucket-staleness path: change (concept, action) -> (task, review).
+    const victims = await brain.find({ type: 'concept', where: { subtype: 'action' }, limit: 1 })
+    expect(victims.length).toBe(1)
+    const id = victims[0].id
+    await brain.update({ id, type: 'task', subtype: 'review' })
+
+    const oldCombined = idSet(await brain.find({ type: 'concept', where: { subtype: 'action' }, limit: 1000 }))
+    expect(oldCombined.has(id)).toBe(false) // unposted from the old buckets
+    const newCombined = idSet(await brain.find({ type: 'task', where: { subtype: 'review' }, limit: 1000 }))
+    expect(newCombined.has(id)).toBe(true) // posted to the new buckets
+    const subtypeOnly = idSet(await brain.find({ where: { subtype: 'review' }, limit: 1000 }))
+    expect(subtypeOnly.has(id)).toBe(true)
+  })
+})

tests/integration/migration-7x-to-8x.test.ts

@@ -191,4 +191,39 @@ describe('7.x → 8.0 layout migration', () => {
     expect(await captureReference(reopened)).toEqual(ref)
     expect(fs.existsSync(path.join(dir, 'branches'))).toBe(false)
   })
+
+  it('PARITY GUARD: branch-scoped non-entity state survives migration (not silently drained)', async () => {
+    const dir = freshLegacyDir()
+
+    // Simulate a 7.x engine that wrote durable state under the HEAD branch
+    // OUTSIDE entities/ (a branch-scoped index/blob/registry). The migration
+    // rescues entities/ only; the guard must PRESERVE the rest rather than let
+    // removeRawPrefix silently delete it — the VFS `_cow/` stranding lesson.
+    const raw: any = new FileSystemStorage(dir)
+    await raw.init()
+    await raw.writeRawObject('branches/main/_legacy_engine/state', { keep: 'me', n: 42 })
+    await raw.flush?.()
+    raw.stopFlushRequestWatcher?.()
+    await raw.releaseWriterLock?.()
+
+    // Migrate (entities collapse to the root as usual).
+    const brain = await openBrain(dir)
+    brains.push(brain)
+    await brain.init()
+    expect(await captureReference(brain)).toEqual(reference)
+    await brain.close()
+    brains.splice(brains.indexOf(brain), 1)
+
+    // The non-entity branch-scoped state is PRESERVED (branch NOT drained) —
+    // recoverable, not lost.
+    const check: any = new FileSystemStorage(dir)
+    await check.init()
+    expect(await check.readRawObject('branches/main/_legacy_engine/state')).toMatchObject({
+      keep: 'me',
+      n: 42
+    })
+    await check.flush?.()
+    check.stopFlushRequestWatcher?.()
+    await check.releaseWriterLock?.()
+  })
 })

tests/integration/multi-process-safety.test.ts

@@ -110,6 +110,89 @@ describe('Multi-process safety + read-only mode', () => {
       // Don't track `blocked` for afterEach cleanup since init failed.
     })
 
+    it('takes over a STALE foreign lock (dead PID + old heartbeat) and claims atomically', async () => {
+      const { mkdirSync, writeFileSync, readFileSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      const os = await import('node:os')
+      mkdirSync(join(dir, 'locks'), { recursive: true })
+      const tenMinutesAgo = new Date(Date.now() - 10 * 60 * 1000).toISOString()
+      writeFileSync(join(dir, 'locks', '_writer.lock'), JSON.stringify({
+        pid: 999999999, // no such process — provably dead
+        hostname: os.hostname(),
+        startedAt: tenMinutesAgo,
+        lastHeartbeat: tenMinutesAgo,
+        version: '8.0.0',
+        rootDir: dir
+      }))
+
+      writer = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await writer.init() // stale takeover must succeed
+
+      const lock = JSON.parse(readFileSync(join(dir, 'locks', '_writer.lock'), 'utf-8'))
+      expect(lock.pid).toBe(process.pid) // the atomic wx claim installed OUR lock
+    })
+
+    it('the writer-locked error carries the machine-readable contract (code + lockInfo)', async () => {
+      const { mkdirSync, writeFileSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      const os = await import('node:os')
+      mkdirSync(join(dir, 'locks'), { recursive: true })
+      const otherPid = (process as any).ppid || 1
+      writeFileSync(join(dir, 'locks', '_writer.lock'), JSON.stringify({
+        pid: otherPid,
+        hostname: os.hostname(),
+        startedAt: new Date().toISOString(),
+        lastHeartbeat: new Date().toISOString(),
+        version: '8.7.0',
+        rootDir: dir
+      }))
+
+      const blocked = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      const err: any = await blocked.init().catch((e) => e)
+      expect(err.code).toBe('BRAINY_WRITER_LOCKED')
+      expect(err.lockInfo?.pid).toBe(otherPid)
+    })
+
+    it('release drains an in-flight heartbeat — no phantom lock re-created after unlink', async () => {
+      // The race (8.9.0): clearInterval stops FUTURE heartbeat ticks, but a
+      // tick already in flight could land its lock rewrite AFTER release's
+      // unlink — re-creating the lock as a phantom that blocks the next
+      // writer until the stale TTL. Simulate the in-flight tick explicitly
+      // and prove release waits for it.
+      writer = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await writer.init()
+      const storage: any = (writer as any).storage
+
+      // An in-flight refresh that is ALREADY PAST its ownership guards
+      // (captured the lock info before release ran) and lands its atomic
+      // rewrite slowly — the exact straggler shape; absent the drain it
+      // writes after the unlink.
+      const { join: joinPath } = await import('node:path')
+      const capturedInfo = { ...storage.writerLockInfo }
+      const lockPath = joinPath(dir, 'locks', '_writer.lock')
+      const slowTick = (async () => {
+        await new Promise((r) => setTimeout(r, 100))
+        await storage.writeFileAtomic(
+          lockPath,
+          JSON.stringify({ ...capturedInfo, lastHeartbeat: new Date().toISOString() })
+        )
+      })()
+      storage.writerHeartbeatInFlight = slowTick.catch(() => {})
+
+      await writer.close() // → releaseWriterLock must drain slowTick first
+      await slowTick.catch(() => {}) // both paths fully settled either way
+      writer = null
+
+      const { existsSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      expect(existsSync(join(dir, 'locks', '_writer.lock'))).toBe(false)
+
+      // And the directory is immediately claimable — no stale-TTL wait.
+      const next = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await expect(next.init()).resolves.toBeUndefined()
+      await next.close()
+    })
+
     it('allows a second in-process writer with a warning (same PID)', async () => {
       // Two Brainy instances in the same Node process: not the dangerous
       // cross-process case. Should succeed (with a console warning).

tests/integration/onchange-feed.test.ts

@@ -0,0 +1,228 @@
+/**
+ * @module tests/integration/onchange-feed
+ * @description The `brain.onChange` in-process change feed — the authoritative
+ * post-commit signal for every canonical mutation, regardless of origin.
+ * Pins the delivery contract:
+ *  - every operation fires exactly once per affected record with the
+ *    post-commit payload (deletes carry the record's LAST state);
+ *  - batch operations emit one event per item; transact items share one
+ *    generation;
+ *  - VFS writes (a Tier-1 blind spot for router-synthesized events) emit;
+ *  - events arrive in commit order with monotonic generations;
+ *  - a losing `ifRev` CAS emits NOTHING (aborted commits are invisible);
+ *  - a throwing listener is isolated; unsubscribe stops delivery;
+ *  - clear()/restore() emit one store-level "refetch everything" event.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import type { BrainyChangeEvent } from '../../src/events/changeFeed.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+
+let seq = 0
+const freshId = (): string =>
+  `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+/** Let the microtask-dispatched feed drain. */
+const drained = (): Promise<void> => new Promise((r) => setTimeout(r, 0))
+
+describe('brain.onChange — the in-process change feed', () => {
+  let dir: string
+  let brain: any
+  let events: BrainyChangeEvent[]
+  let off: () => void
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-onchange-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+    events = []
+    off = brain.onChange((e: BrainyChangeEvent) => events.push(e))
+  })
+
+  afterEach(async () => {
+    off()
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('add → update → remove: one fully-described event each; remove carries the last state', async () => {
+    const id = await brain.add({
+      id: freshId(),
+      data: 'a document',
+      type: NounType.Document,
+      metadata: { status: 'draft' }
+    })
+    await brain.update({ id, metadata: { status: 'published' } })
+    await brain.remove(id)
+    await drained()
+
+    expect(events.map((e) => e.op)).toEqual(['add', 'update', 'remove'])
+    expect(events.every((e) => e.kind === 'entity' && e.id === id)).toBe(true)
+
+    expect(events[0].entity).toMatchObject({ id, type: 'document', metadata: { status: 'draft' } })
+    expect(events[1].entity).toMatchObject({ id, metadata: { status: 'published' } })
+    // The delete payload is the record's LAST committed state.
+    expect(events[2].entity).toMatchObject({ id, type: 'document', metadata: { status: 'published' } })
+
+    // Post-commit ordering: generations are monotonic.
+    const gens = events.map((e) => e.generation!)
+    expect([...gens].sort((a, b) => a - b)).toEqual(gens)
+    expect(new Set(gens).size).toBe(3)
+  })
+
+  it('relate → updateRelation → unrelate: relation events with endpoints + type', async () => {
+    const a = await brain.add({ id: freshId(), data: 'a', type: NounType.Thing })
+    const b = await brain.add({ id: freshId(), data: 'b', type: NounType.Thing })
+    events.length = 0
+
+    const rel = await brain.relate({ from: a, to: b, type: VerbType.RelatedTo, metadata: { w: 1 } })
+    await brain.updateRelation({ id: rel, metadata: { w: 2 } })
+    await brain.unrelate(rel)
+    await drained()
+
+    expect(events.map((e) => e.op)).toEqual(['relate', 'updateRelation', 'unrelate'])
+    expect(events.every((e) => e.kind === 'relation' && e.id === rel)).toBe(true)
+    for (const e of events) {
+      expect(e.relation).toMatchObject({ id: rel, from: a, to: b })
+    }
+  })
+
+  it('remove() cascades: deleting an entity emits unrelate for its relationships too', async () => {
+    const a = await brain.add({ id: freshId(), data: 'src', type: NounType.Thing })
+    const b = await brain.add({ id: freshId(), data: 'tgt', type: NounType.Thing })
+    const rel = await brain.relate({ from: a, to: b, type: VerbType.Contains })
+    events.length = 0
+
+    await brain.remove(a)
+    await drained()
+
+    const removeEvent = events.find((e) => e.op === 'remove')
+    const unrelateEvent = events.find((e) => e.op === 'unrelate')
+    expect(removeEvent).toMatchObject({ kind: 'entity', id: a })
+    expect(unrelateEvent).toMatchObject({
+      kind: 'relation',
+      id: rel,
+      relation: { from: a, to: b }
+    })
+    // Entity + cascaded verb share the one remove() generation.
+    expect(removeEvent!.generation).toBe(unrelateEvent!.generation)
+  })
+
+  it('batches emit one event per item; removeMany deletes carry payloads', async () => {
+    const ids = [freshId(), freshId(), freshId()]
+    await brain.addMany({
+      items: ids.map((id, i) => ({
+        id,
+        data: `item ${i}`,
+        type: NounType.Thing,
+        metadata: { n: i }
+      }))
+    })
+    await drained()
+    expect(events.filter((e) => e.op === 'add').length).toBe(3)
+
+    events.length = 0
+    await brain.removeMany({ ids })
+    await drained()
+    const removes = events.filter((e) => e.op === 'remove')
+    expect(removes.length).toBe(3)
+    // Every delete is fully described (enriched, not id-only).
+    for (const e of removes) {
+      expect(e.entity).toBeDefined()
+      expect(e.entity!.type).toBe('thing')
+      expect(typeof e.entity!.metadata.n).toBe('number')
+    }
+  })
+
+  it('transact(): per-item events sharing ONE generation; a rejected batch emits nothing', async () => {
+    const x = freshId()
+    const y = freshId()
+    await brain.transact([
+      { op: 'add', id: x, data: 'x', type: NounType.Thing, metadata: { k: 1 } },
+      { op: 'add', id: y, data: 'y', type: NounType.Thing },
+      { op: 'relate', from: x, to: y, type: VerbType.RelatedTo }
+    ])
+    await drained()
+
+    expect(events.map((e) => e.op)).toEqual(['add', 'add', 'relate'])
+    expect(new Set(events.map((e) => e.generation)).size).toBe(1)
+
+    // Rejected batch (stale per-op CAS) → zero events.
+    events.length = 0
+    await expect(
+      brain.transact([{ op: 'update', id: x, metadata: { k: 2 }, ifRev: 999 }])
+    ).rejects.toMatchObject({ name: 'RevisionConflictError' })
+    await drained()
+    expect(events.length).toBe(0)
+  })
+
+  it('a losing ifRev CAS update emits NOTHING; the winner emits once', async () => {
+    const id = await brain.add({ id: freshId(), data: 'contended', type: NounType.Thing })
+    const rev = (await brain.get(id))._rev
+    events.length = 0
+
+    const results = await Promise.allSettled(
+      Array.from({ length: 4 }, (_, i) =>
+        brain.update({ id, metadata: { w: i }, merge: false, ifRev: rev })
+      )
+    )
+    await drained()
+
+    expect(results.filter((r) => r.status === 'fulfilled').length).toBe(1)
+    expect(events.filter((e) => e.op === 'update').length).toBe(1) // exactly the winner
+  })
+
+  it('VFS writes emit (the router-synthesis blind spot)', async () => {
+    await brain.vfs.writeFile('/notes/hello.txt', 'hello feed')
+    await drained()
+    // A VFS write is entities + containment relations under the hood — the
+    // feed sees it because VFS delegates to canonical brain methods.
+    expect(events.length).toBeGreaterThan(0)
+    expect(events.some((e) => e.kind === 'entity' && e.op === 'add')).toBe(true)
+  })
+
+  it('listener errors are isolated; unsubscribe stops delivery', async () => {
+    const good: BrainyChangeEvent[] = []
+    const offBad = brain.onChange(() => {
+      throw new Error('subscriber bug')
+    })
+    const offGood = brain.onChange((e: BrainyChangeEvent) => good.push(e))
+
+    const id = await brain.add({ id: freshId(), data: 'p', type: NounType.Thing })
+    await drained()
+    expect(good.length).toBeGreaterThan(0) // sibling unaffected by the throwing listener
+
+    offBad()
+    offGood()
+    good.length = 0
+    events.length = 0
+    off() // unsubscribe the outer listener too
+    await brain.update({ id, metadata: { after: true } })
+    await drained()
+    expect(events.length).toBe(0)
+    expect(good.length).toBe(0)
+
+    // Re-subscribe for afterEach symmetry.
+    off = brain.onChange((e: BrainyChangeEvent) => events.push(e))
+  })
+
+  it('clear() emits one store-level event meaning "refetch everything"', async () => {
+    await brain.add({ id: freshId(), data: 'doomed', type: NounType.Thing })
+    events.length = 0
+    await brain.clear()
+    await drained()
+    const store = events.filter((e) => e.kind === 'store')
+    expect(store).toHaveLength(1)
+    expect(store[0].op).toBe('clear')
+    expect(store[0].generation).toBeUndefined()
+  })
+})

tests/integration/read-your-writes-contract.test.ts

@@ -0,0 +1,133 @@
+/**
+ * @module tests/integration/read-your-writes-contract
+ * @description The read-your-writes contract — brainy's spine guarantee that a
+ * write which has RETURNED is immediately readable, under the single writer, by
+ * BOTH id and every index (metadata filter, vector search, graph traversal),
+ * with NO await, delay, or retry between the write returning and the read.
+ *
+ * Why it holds by construction on the canonical + JS-index path: every mutation
+ * runs its canonical writes AND all derived-index projections (HNSW, metadata
+ * index, graph index) as operations inside ONE transaction that commits before
+ * the method returns — there is no post-return "eventual" indexing window. get()
+ * additionally rides the storage write-through cache (read-after-write within
+ * the process). The generation counter is the {seq} a caller can pin: a write
+ * returns generation N and every subsequent live read observes ≥ N.
+ *
+ * This pins the guarantee so no future change can quietly move index maintenance
+ * off the commit path (which would turn a returned write into a not-yet-queryable
+ * one — the "a 200 is still not durability/queryability" failure the spine
+ * program exists to make impossible). The native accelerator must honor the same
+ * in-commit contract; a background consolidation/optimization must never gate
+ * queryability of an acknowledged write.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+
+let seq = 0
+const freshId = (): string =>
+  `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+describe('read-your-writes contract (single-writer, no await between write and read)', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-ryw-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('add(): the entity is immediately readable by id, metadata filter, AND vector search', async () => {
+    const id = freshId()
+    const marker = `ryw-${id}`
+    await brain.add({ id, data: 'read your writes probe alpha', type: NounType.Concept, metadata: { marker } })
+
+    // id read (write-through cache / canonical)
+    expect(await brain.get(id)).not.toBeNull()
+    // metadata-filter read (metadata index committed in-transaction)
+    const byMeta = await brain.find({ where: { marker }, limit: 10 })
+    expect(byMeta.map((r: any) => r.id)).toContain(id)
+    // vector read (HNSW committed in-transaction)
+    const byVector = await brain.find({ query: 'read your writes probe alpha', limit: 10 })
+    expect(byVector.map((r: any) => r.id)).toContain(id)
+    // type-filter read
+    const byType = await brain.find({ type: NounType.Concept, limit: 100 })
+    expect(byType.map((r: any) => r.id)).toContain(id)
+  })
+
+  it('update(): the new metadata is immediately queryable and the old value is immediately gone', async () => {
+    const id = freshId()
+    await brain.add({ id, data: 'updatable', type: NounType.Thing, metadata: { phase: 'before' } })
+    await brain.update({ id, metadata: { phase: 'after' } })
+
+    const after = await brain.find({ where: { phase: 'after' }, limit: 10 })
+    expect(after.map((r: any) => r.id)).toContain(id)
+    const before = await brain.find({ where: { phase: 'before' }, limit: 10 })
+    expect(before.map((r: any) => r.id)).not.toContain(id)
+  })
+
+  it('relate(): the edge is immediately traversable', async () => {
+    const a = await brain.add({ id: freshId(), data: 'A', type: NounType.Thing })
+    const b = await brain.add({ id: freshId(), data: 'B', type: NounType.Thing })
+    await brain.relate({ from: a, to: b, type: VerbType.RelatedTo })
+    expect((await brain.related(a)).map((r: any) => r.to)).toContain(b)
+  })
+
+  it('remove(): the entity is immediately gone from id, metadata, and vector reads', async () => {
+    const id = freshId()
+    const marker = `gone-${id}`
+    await brain.add({ id, data: 'to be removed promptly', type: NounType.Thing, metadata: { marker } })
+    expect(await brain.get(id)).not.toBeNull()
+
+    await brain.remove(id)
+    expect(await brain.get(id)).toBeNull()
+    expect((await brain.find({ where: { marker }, limit: 10 })).map((r: any) => r.id)).not.toContain(id)
+    expect((await brain.find({ query: 'to be removed promptly', limit: 10 })).map((r: any) => r.id)).not.toContain(id)
+  })
+
+  it('transact(): every item of a committed batch is immediately readable by id, index, and traversal', async () => {
+    const a = freshId()
+    const b = freshId()
+    const marker = `batch-${a}`
+    const db = await brain.transact([
+      { op: 'add', id: a, data: 'batch node A', type: NounType.Thing, metadata: { marker } },
+      { op: 'add', id: b, data: 'batch node B', type: NounType.Thing, metadata: { marker } },
+      { op: 'relate', from: a, to: b, type: VerbType.Contains }
+    ])
+    expect(db.generation).toBeGreaterThan(0)
+
+    expect(await brain.get(a)).not.toBeNull()
+    expect(await brain.get(b)).not.toBeNull()
+    const byMeta = (await brain.find({ where: { marker }, limit: 10 })).map((r: any) => r.id)
+    expect(byMeta).toContain(a)
+    expect(byMeta).toContain(b)
+    expect((await brain.related(a)).map((r: any) => r.to)).toContain(b)
+  })
+
+  it('the write returns a generation ({seq}) and a live read observes it — the awaitable-read primitive', async () => {
+    const id = freshId()
+    const before = brain.generationStore.generation()
+    await brain.add({ id, data: 'generation stamped', type: NounType.Thing })
+    const after = brain.generationStore.generation()
+    // The write advanced the generation…
+    expect(after).toBeGreaterThan(before)
+    // …and a read pinned to that generation observes the write (asOf ≥ N sees it).
+    const pinned = await brain.asOf(after)
+    expect(await pinned.get(id)).not.toBeNull()
+  })
+})

tests/integration/readdir-no-duplicate-replace.test.ts

@@ -0,0 +1,56 @@
+/**
+ * @module tests/integration/readdir-no-duplicate-replace
+ * @description A re-created path must NOT show up twice in readdir. The reported
+ * defect (memory-vm) was readdir serving DUPLICATE entries for re-created paths,
+ * rooted in a delete that left a ghost (partial removal) whose next delete read
+ * null metadata and skipped unposting the stale Contains edge. With full-removal
+ * deletes there is no ghost, so the edge is always unposted and the path appears
+ * exactly once after any number of delete→recreate cycles.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+
+describe('readdir shows a re-created path exactly once (no duplicate on replace)', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-readdir-'))
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir }, silent: true, dimensions: 384 })
+    await brain.init()
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('delete then re-create the same file path — readdir lists it once', async () => {
+    await brain.vfs.mkdir('/d', { recursive: true })
+    await brain.vfs.writeFile('/d/x.txt', 'v1')
+    expect(await brain.vfs.readdir('/d')).toEqual(['x.txt'])
+
+    await brain.vfs.unlink('/d/x.txt')
+    expect(await brain.vfs.readdir('/d')).toEqual([])
+
+    await brain.vfs.writeFile('/d/x.txt', 'v2')
+    const listing = (await brain.vfs.readdir('/d')) as string[]
+    expect(listing).toEqual(['x.txt']) // exactly once — no ghost duplicate
+    expect(listing.filter((n) => n === 'x.txt')).toHaveLength(1)
+  })
+
+  it('survives several delete→recreate cycles without accumulating duplicates', async () => {
+    await brain.vfs.mkdir('/c', { recursive: true })
+    for (let i = 0; i < 4; i++) {
+      await brain.vfs.writeFile('/c/f.txt', `gen ${i}`)
+      await brain.vfs.unlink('/c/f.txt')
+    }
+    await brain.vfs.writeFile('/c/f.txt', 'final')
+    const listing = (await brain.vfs.readdir('/c')) as string[]
+    expect(listing).toEqual(['f.txt'])
+    expect(await brain.vfs.readFile('/c/f.txt')).toBeDefined()
+  })
+})

tests/integration/restore-nondestructive.test.ts

@@ -0,0 +1,168 @@
+/**
+ * @module tests/integration/restore-nondestructive
+ * @description Regression for a consumer-reported recovery hazard: `restore()`
+ * removed the entire live brain directory and THEN `fs.cp`'d the snapshot in, so
+ * a copy failure (most dangerously ENOSPC — `fs.cp` also materializes the holes
+ * of sparse mmap blobs, ballooning a snapshot that would otherwise fit) left the
+ * store destroyed with only a partial copy: the recovery tool could destroy the
+ * brain it was meant to recover.
+ *
+ * Fix: the snapshot is copied into a staging area (sparse-aware) BEFORE any live
+ * data is touched; only after the copy succeeds and a marker is fsync'd does an
+ * atomic per-entry swap move it into place. A copy failure leaves the live store
+ * exactly as it was; a crash mid-swap is resumed forward on the next open.
+ *
+ * These tests exercise the property directly: a forced copy failure leaves live
+ * data intact, a normal restore round-trips correctly, an interrupted-but-
+ * committed staging area is completed on reopen, and the sparse copy preserves
+ * holes byte-for-byte.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+
+const STAGING = '_restore_staging'
+const MARKER = '.restore-manifest.json'
+
+describe('non-destructive restore (BRAINY-RESTORE-DESTRUCTIVE)', () => {
+  let root: string
+  let snap: string
+  let brain: any
+
+  const openBrain = async (dir: string) => {
+    const b = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await b.init()
+    return b
+  }
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    const base = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-restore-'))
+    root = path.join(base, 'store')
+    snap = path.join(base, 'snapshot')
+    brain = await openBrain(root)
+  })
+
+  afterEach(async () => {
+    try { await brain.close() } catch { /* already closed in a test */ }
+    fs.rmSync(path.dirname(root), { recursive: true, force: true })
+  })
+
+  it('a copy failure leaves the LIVE store completely intact (the core property)', async () => {
+    const keep = await brain.add({ id: '00000000-0000-4000-8000-0000000000c1', data: 'in snapshot', type: NounType.Thing })
+    await brain.flush()
+    await brain['storage'].snapshotToDirectory(snap)
+    // Mutate live AFTER the snapshot — this state must survive a failed restore.
+    const extra = await brain.add({ id: '00000000-0000-4000-8000-0000000000c2', data: 'live only', type: NounType.Thing })
+
+    // Force the staging copy to fail on its second entry (partial staging).
+    const storage = brain['storage']
+    const realCopy = storage.copyTreeSparse.bind(storage)
+    let calls = 0
+    storage.copyTreeSparse = async (src: string, dest: string) => {
+      if (++calls === 2) throw new Error('simulated ENOSPC')
+      return realCopy(src, dest)
+    }
+
+    await expect(brain.restore(snap, { confirm: true })).rejects.toThrow(/untouched/)
+
+    // Live data — BOTH the snapshot entity and the live-only mutation — survives.
+    expect(await brain.get(keep)).not.toBeNull()
+    expect(await brain.get(extra)).not.toBeNull()
+    // The half-written staging area was cleaned up.
+    expect(fs.existsSync(path.join(root, STAGING))).toBe(false)
+  })
+
+  it('a normal restore round-trips to the snapshot state', async () => {
+    const a = await brain.add({ id: '00000000-0000-4000-8000-0000000000c3', data: 'A', type: NounType.Thing })
+    await brain.flush()
+    await brain['storage'].snapshotToDirectory(snap)
+    // Diverge from the snapshot: add B, remove A.
+    const b = await brain.add({ id: '00000000-0000-4000-8000-0000000000c4', data: 'B', type: NounType.Thing })
+    await brain.remove(a)
+    expect(await brain.get(a)).toBeNull()
+    expect(await brain.get(b)).not.toBeNull()
+
+    await brain.restore(snap, { confirm: true })
+
+    // Back to exactly the snapshot: A present, B gone.
+    expect(await brain.get(a)).not.toBeNull()
+    expect(await brain.get(b)).toBeNull()
+    expect(fs.existsSync(path.join(root, STAGING))).toBe(false)
+  })
+
+  it('an interrupted-but-committed restore is completed on the next open (resume forward)', async () => {
+    const a = await brain.add({ id: '00000000-0000-4000-8000-0000000000c5', data: 'A', type: NounType.Thing })
+    await brain.flush()
+    await brain['storage'].snapshotToDirectory(snap)
+    const b = await brain.add({ id: '00000000-0000-4000-8000-0000000000c6', data: 'B', type: NounType.Thing })
+    await brain.flush()
+    await brain.close()
+
+    // Simulate the state right after the staging copy committed but before the
+    // swap ran: a _restore_staging holding the snapshot's entries + the marker.
+    const staging = path.join(root, STAGING)
+    fs.mkdirSync(staging, { recursive: true })
+    const entries: string[] = []
+    for (const entry of fs.readdirSync(snap)) {
+      if (entry === 'locks' || entry === STAGING) continue
+      fs.cpSync(path.join(snap, entry), path.join(staging, entry), { recursive: true })
+      entries.push(entry)
+    }
+    fs.writeFileSync(path.join(staging, MARKER), JSON.stringify({ entries }))
+
+    // Reopen → init resumes the swap → the store is the snapshot (A, no B).
+    brain = await openBrain(root)
+    expect(await brain.get(a)).not.toBeNull()
+    expect(await brain.get(b)).toBeNull()
+    expect(fs.existsSync(staging)).toBe(false)
+  })
+
+  it('an uncommitted staging area (no marker) is discarded on open, live untouched', async () => {
+    const a = await brain.add({ id: '00000000-0000-4000-8000-0000000000c7', data: 'A live', type: NounType.Thing })
+    await brain.flush()
+    await brain.close()
+
+    // A staging dir with NO marker = a copy that never committed → debris.
+    const staging = path.join(root, STAGING)
+    fs.mkdirSync(staging, { recursive: true })
+    fs.writeFileSync(path.join(staging, 'entities'), 'garbage partial copy')
+
+    brain = await openBrain(root)
+    expect(await brain.get(a)).not.toBeNull() // live authoritative
+    expect(fs.existsSync(staging)).toBe(false) // debris removed
+  })
+
+  it('the sparse copy preserves holes: content byte-identical, allocation far below apparent size', async () => {
+    const base = path.dirname(root)
+    const src = path.join(base, 'sparse-src.bin')
+    const dest = path.join(base, 'sparse-dest.bin')
+    const SIZE = 32 * 1024 * 1024 // 32 MiB apparent
+    const DATA = Buffer.from('hello sparse world')
+
+    // Create a sparse source: 32 MiB of holes with a little real data at the start.
+    const fh = fs.openSync(src, 'w')
+    fs.ftruncateSync(fh, SIZE)
+    fs.writeSync(fh, DATA, 0, DATA.length, 0)
+    fs.closeSync(fh)
+
+    const storage = brain['storage']
+    const stat = fs.statSync(src)
+    await storage.copyFileSparse(src, dest, stat.size, stat.mode)
+
+    // Same apparent size and byte-identical content…
+    const destStat = fs.statSync(dest)
+    expect(destStat.size).toBe(SIZE)
+    expect(fs.readFileSync(dest).equals(fs.readFileSync(src))).toBe(true)
+    // …but the destination is actually sparse (allocated bytes far below size).
+    expect(destStat.blocks * 512).toBeLessThan(SIZE / 2)
+  })
+})

tests/integration/rollback-trapdoor.test.ts

@@ -0,0 +1,156 @@
+/**
+ * @module tests/integration/rollback-trapdoor
+ * @description Regression for a consumer-reported data-integrity bug: a failed
+ * CANONICAL rollback undo left the record durable while the request errored (a
+ * post-commit response lie). Reproduced by a failure-injection coherence harness:
+ * a late index op throws → rollback runs → an earlier op's canonical undo
+ * (storage.deleteNounMetadata) fails persistently → the record stays on disk
+ * while Transaction.rollback threw TransactionRollbackError AND lied with
+ * state='rolled_back'.
+ *
+ * Fix (David's ruling — adopt-forward when safe, else fail loud):
+ *  - SINGLE-op add whose only damage is a durably-present orphan → adopt it
+ *    forward: commit the generation (the record IS what add() wanted), return
+ *    success + a loud warn; the derived index heals on rebuild/repairIndex.
+ *  - MULTI-op batch, OR any restorative LOSS → StoreInconsistentError naming the
+ *    unreconciled ids; the brain enters write-quarantine (reads OK, writes
+ *    refused) until repairIndex() reconciles and lifts it. Transaction state is
+ *    'inconsistent', never the 'rolled_back' lie.
+ *
+ * The injection mirrors that harness: fault index.addToIndex (trigger rollback) and
+ * storage.deleteNounMetadata (fail the canonical undo).
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+import { StoreInconsistentError } from '../../src/db/errors.js'
+
+let seq = 0
+const freshId = (): string =>
+  `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+describe('rollback trapdoor — failed canonical undo (data integrity)', () => {
+  let dir: string
+  let brain: any
+  let restore: Array<() => void>
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-trapdoor-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+    restore = []
+  })
+
+  afterEach(async () => {
+    for (const r of restore) r()
+    try { await brain.close() } catch { /* ignore */ }
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  /** Fault the metadata-index add so a write's rollback is triggered. */
+  const faultIndexAdd = () => {
+    const idx = brain['metadataIndex']
+    const orig = idx.addToIndex.bind(idx)
+    idx.addToIndex = async () => { throw new Error('injected: index add failed') }
+    restore.push(() => { idx.addToIndex = orig })
+  }
+  /** Fault the canonical additive undo (delete of a just-created record). */
+  const faultCanonicalDelete = () => {
+    const storage = brain['storage']
+    const orig = storage.deleteNounMetadata.bind(storage)
+    storage.deleteNounMetadata = async () => { throw new Error('injected: canonical undo failed') }
+    restore.push(() => { storage.deleteNounMetadata = orig })
+  }
+
+  it('ADOPT-FORWARD: a single-op add whose canonical undo fails commits as a durable, get-able record (no throw, no quarantine)', async () => {
+    const id = freshId()
+    const genBefore = brain.generationStore.generation()
+
+    faultIndexAdd()        // triggers rollback
+    faultCanonicalDelete() // the trapdoor: the metadata record can't be un-written
+
+    // add() must NOT throw — the record it wanted is durable, so it is adopted.
+    const returned = await brain.add({ id, data: 'orphan adopted', type: NounType.Thing })
+    expect(returned).toBe(id)
+
+    // Un-fault so reads/subsequent writes use the real methods.
+    for (const r of restore) r()
+    restore = []
+
+    // The record is durable and get-able, and the generation advanced.
+    expect(await brain.get(id)).not.toBeNull()
+    expect(brain.generationStore.generation()).toBe(genBefore + 1)
+
+    // The store is NOT quarantined — a subsequent write succeeds.
+    const ok = await brain.add({ id: freshId(), data: 'still writable', type: NounType.Thing })
+    expect(await brain.get(ok)).not.toBeNull()
+  })
+
+  it('FAIL-LOUD: a multi-op transact whose canonical undo fails throws StoreInconsistentError and write-quarantines the brain', async () => {
+    // A pre-existing record to prove reads keep working under quarantine.
+    const anchor = await brain.add({ id: freshId(), data: 'anchor', type: NounType.Thing })
+    await brain.flush()
+
+    faultIndexAdd()
+    faultCanonicalDelete()
+
+    const a = freshId()
+    const b = freshId()
+    await expect(
+      brain.transact([
+        { op: 'add', id: a, data: 'batch A', type: NounType.Thing },
+        { op: 'add', id: b, data: 'batch B', type: NounType.Thing }
+      ])
+    ).rejects.toBeInstanceOf(StoreInconsistentError)
+
+    // Writes are refused (quarantine); reads still work.
+    await expect(
+      brain.add({ id: freshId(), data: 'blocked', type: NounType.Thing })
+    ).rejects.toThrow(/WRITE-QUARANTINED/)
+    expect(await brain.get(anchor)).not.toBeNull()
+
+    // repairIndex() reconciles and lifts the quarantine → writes resume.
+    for (const r of restore) r()
+    restore = []
+    await brain.repairIndex()
+
+    const ok = await brain.add({ id: freshId(), data: 'writable again', type: NounType.Thing })
+    expect(await brain.get(ok)).not.toBeNull()
+  })
+
+  it('the StoreInconsistentError names the unreconciled records with dispositions', async () => {
+    faultIndexAdd()
+    faultCanonicalDelete()
+
+    const a = freshId()
+    const b = freshId()
+    let caught: unknown
+    await brain
+      .transact([
+        { op: 'add', id: a, data: 'A', type: NounType.Thing },
+        { op: 'add', id: b, data: 'B', type: NounType.Thing }
+      ])
+      .catch((e: unknown) => (caught = e))
+
+    expect(caught).toBeInstanceOf(StoreInconsistentError)
+    const err = caught as StoreInconsistentError
+    expect(err.records.length).toBeGreaterThan(0)
+    // Every unreconciled record here is a durably-present orphan.
+    expect(err.records.every((r) => r.disposition === 'orphan')).toBe(true)
+    expect(err.message).toMatch(/WRITE-QUARANTINED/)
+
+    // Recover so afterEach can close cleanly.
+    for (const r of restore) r()
+    restore = []
+    await brain.repairIndex()
+  })
+})

tests/integration/temporal-vfs.test.ts

@@ -0,0 +1,200 @@
+/**
+ * @module tests/integration/temporal-vfs
+ * @description VFS content joins the Model-B immutability model. Previously
+ * the temporal model had a hole exactly where files were concerned: entity
+ * records were versioned per write (before-images), but the content BYTES
+ * lived under an eager refCount GC — `rm` could physically delete bytes that
+ * in-window history still referenced, and overwrite never released the old
+ * hash at all (an unbounded silent leak that only accidentally preserved
+ * history). Now blob bytes are retention-protected: a content blob referenced
+ * by any generation inside the retention window (or live) survives, and the
+ * ONE reclamation point is history compaction.
+ *
+ * Pins:
+ *  - `readFile(path, { asOf })` returns each version's exact bytes.
+ *  - `history(path)` lists versions ascending with generation/hash/size.
+ *  - overwrite releases the superseded live reference (leak fixed) while
+ *    history protects the bytes; rm keeps bytes readable via asOf.
+ *  - compaction past the last referencing generation physically reclaims
+ *    bytes (zero live + zero history), and never reclaims in-window content —
+ *    including the cross-file dedup case where an old file's history and a
+ *    newer file's history share one hash.
+ *  - overwrite refreshes the entity's `data` (embedding text) — the stale
+ *    `.data` defect.
+ *  - the backfill/scrub recounts exactly from the generation records.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+
+describe('temporal VFS — blob immutability under Model B', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-temporal-vfs-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  /** The file's blob metadata via the storage layer (test introspection). */
+  async function blobMeta(hash: string) {
+    return brain['storage'].blobStorage.getMetadata(hash)
+  }
+
+  it('readFile(path, {asOf}) returns each version‘s exact bytes; history(path) lists them', async () => {
+    const p = '/pages/home.json'
+    await brain.vfs.writeFile(p, 'v1 — original')
+    const g1 = brain.generationStore.generation()
+    await brain.vfs.writeFile(p, 'v2 — edited')
+    await brain.vfs.writeFile(p, 'v3 — final')
+
+    // Live read = newest.
+    expect((await brain.vfs.readFile(p)).toString()).toBe('v3 — final')
+
+    // History lists the file's write-generations ascending, distinct hashes.
+    const versions = await brain.vfs.history(p)
+    expect(versions.length).toBeGreaterThanOrEqual(3)
+    const gens = versions.map((v: any) => v.generation)
+    expect([...gens].sort((a: number, b: number) => a - b)).toEqual(gens)
+
+    // Every listed version's exact bytes are readable.
+    const texts: string[] = []
+    for (const v of versions) {
+      texts.push((await brain.vfs.readFile(p, { asOf: v.generation })).toString())
+    }
+    expect(texts).toContain('v1 — original')
+    expect(texts).toContain('v2 — edited')
+    expect(texts).toContain('v3 — final')
+
+    // Direct generation read too (the incident shape: recover the past bytes).
+    expect((await brain.vfs.readFile(p, { asOf: g1 })).toString()).toBe('v1 — original')
+  })
+
+  it('overwrite releases the superseded LIVE reference (leak fixed) while history protects the bytes', async () => {
+    const p = '/notes/leak.txt'
+    await brain.vfs.writeFile(p, 'old content A')
+    const oldHash = (await brain.vfs.history(p))[0].hash
+    const g1 = brain.generationStore.generation()
+
+    await brain.vfs.writeFile(p, 'new content B')
+
+    // The old hash's LIVE reference is released (previously it leaked forever)…
+    expect((await blobMeta(oldHash))?.refCount).toBe(0)
+    // …but the bytes survive (history-protected) and asOf still reads them.
+    expect((await brain.vfs.readFile(p, { asOf: g1 })).toString()).toBe('old content A')
+  })
+
+  it('rm keeps the bytes readable via asOf inside the window (no premature deletion)', async () => {
+    const p = '/docs/doomed.txt'
+    await brain.vfs.writeFile(p, 'still recoverable after rm')
+    const g = brain.generationStore.generation()
+    const hash = (await brain.vfs.history(p))[0].hash
+
+    await brain.vfs.unlink(p)
+
+    // Live path is gone…
+    await expect(brain.vfs.readFile(p)).rejects.toThrow()
+    // …the bytes are not: zero live refs, history-protected.
+    expect((await blobMeta(hash))?.refCount).toBe(0)
+    expect((await brain['storage'].blobStorage.read(hash)).toString()).toBe(
+      'still recoverable after rm'
+    )
+    // (Path-level asOf resolution of a DELETED path is future work — the
+    // bytes + entity history are intact; recovery reads go through the hash
+    // or a pre-delete generation view.)
+    void g
+  })
+
+  it('compaction is the ONE reclamation point: past-window bytes are reclaimed, in-window preserved', async () => {
+    const p = '/pages/compact.json'
+    await brain.vfs.writeFile(p, 'version ONE')
+    const v1Hash = (await brain.vfs.history(p))[0].hash
+    await brain.vfs.writeFile(p, 'version TWO')
+    await brain.vfs.writeFile(p, 'version THREE')
+
+    // Sanity: v1's bytes exist (history-protected, live-released).
+    expect(await blobMeta(v1Hash)).toBeDefined()
+    expect((await blobMeta(v1Hash))?.refCount).toBe(0)
+
+    // Compact everything reclaimable (retain nothing beyond the live state).
+    await brain.compactHistory({ maxGenerations: 0 })
+
+    // v1's bytes are physically GONE — zero live, zero history.
+    expect(await blobMeta(v1Hash)).toBeUndefined()
+    // The live version is untouched.
+    expect((await brain.vfs.readFile(p)).toString()).toBe('version THREE')
+  })
+
+  it('cross-file dedup: a shared hash survives while ANY in-window generation references it', async () => {
+    const shared = 'identical bytes shared across files and time'
+    const pA = '/a.txt'
+    const pB = '/b.txt'
+
+    // A holds the shared content, then moves off it (A's history references it).
+    await brain.vfs.writeFile(pA, shared)
+    await brain.vfs.writeFile(pA, 'A moved on')
+    // B now holds the shared content live, then is removed (B's remove-gen
+    // before-image references it too).
+    await brain.vfs.writeFile(pB, shared)
+    const gBLive = brain.generationStore.generation()
+    const sharedHash = (await brain.vfs.history(pA))[0].hash
+    await brain.vfs.unlink(pB)
+
+    // Live refs are zero (A moved off; B removed) — bytes survive on history.
+    expect((await blobMeta(sharedHash))?.refCount).toBe(0)
+    expect((await brain['storage'].blobStorage.read(sharedHash)).toString()).toBe(shared)
+
+    // B's content at its live generation is still exactly readable… via asOf
+    // on A's early version too (same bytes, same blob).
+    const versionsA = await brain.vfs.history(pA)
+    const aV1 = versionsA[0]
+    expect((await brain.vfs.readFile(pA, { asOf: aV1.generation })).toString()).toBe(shared)
+    void gBLive
+
+    // Full compaction drops the last history references → bytes reclaimed.
+    await brain.compactHistory({ maxGenerations: 0 })
+    expect(await blobMeta(sharedHash)).toBeUndefined()
+  })
+
+  it('overwrite refreshes the entity data/embedding text (the stale-.data defect)', async () => {
+    const p = '/pages/data-fresh.txt'
+    await brain.vfs.writeFile(p, 'first words')
+    await brain.vfs.writeFile(p, 'second words entirely')
+
+    const entityId = await brain.vfs['pathResolver'].resolve(p)
+    const entity = await brain.get(entityId)
+    expect(entity.data).toBe('second words entirely')
+  })
+
+  it('the scrub recounts history references exactly from the generation records', async () => {
+    const p = '/pages/scrubbed.md'
+    await brain.vfs.writeFile(p, 'scrub v1')
+    await brain.vfs.writeFile(p, 'scrub v2')
+    const v1Hash = (await brain.vfs.history(p))[0].hash
+    await brain.flush()
+
+    const storage = brain['storage']
+    const before = (await blobMeta(v1Hash))?.historyRefCount ?? 0
+    expect(before).toBeGreaterThan(0)
+
+    // Corrupt the counter (simulates a legacy store / crash drift), then scrub.
+    await storage.blobStorage.setHistoryRefCount(v1Hash, 0)
+    const result = await storage.scrubBlobHistoryRefCounts()
+    expect(result.records).toBeGreaterThan(0)
+    expect((await blobMeta(v1Hash))?.historyRefCount).toBe(before)
+  })
+})

tests/integration/transact-durability-barrier.test.ts

@@ -0,0 +1,151 @@
+/**
+ * @module tests/integration/transact-durability-barrier
+ * @description Regression for a consumer-reported durability gap: a committed
+ * `transact()` reported success while its canonical entity writes were still
+ * only in the page cache (tmp+rename, not yet fsync'd), whereas the generation
+ * counter/manifest WERE fsync'd — so a hard kill could leave the counter ahead
+ * of the persisted entity bytes ("phantom progress" for any generation-based
+ * consumer resuming from the counter).
+ *
+ * Fix: `commitTransaction` opens a write barrier before running the planned
+ * operations and flushes it (fsync of every canonical write + the parent dir of
+ * every canonical delete) BEFORE advancing the counter and writing the manifest.
+ * These tests assert the ordering property directly by recording the storage's
+ * `syncRawObjects` calls: the entity writes are fsync'd in an earlier call than
+ * the manifest, and a precommit-rejected batch opens no barrier and advances
+ * nothing.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+import { MemoryStorage } from '../../src/storage/adapters/memoryStorage.js'
+
+const MANIFEST_REL = '_system/manifest.json'
+
+describe('transact durability barrier — entity writes fsync before the counter advances', () => {
+  let dir: string
+  let brain: any
+  let syncCalls: string[][]
+  let beginCount: number
+  let flushCount: number
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-durability-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+
+    // Instrument the real filesystem storage: record every fsync batch in order,
+    // and count barrier open/flush, delegating to the originals.
+    syncCalls = []
+    beginCount = 0
+    flushCount = 0
+    const storage = brain['storage']
+    const origSync = storage.syncRawObjects.bind(storage)
+    storage.syncRawObjects = async (paths: string[]) => {
+      syncCalls.push([...paths])
+      return origSync(paths)
+    }
+    const origBegin = storage.beginWriteBarrier.bind(storage)
+    storage.beginWriteBarrier = () => {
+      beginCount++
+      return origBegin()
+    }
+    const origFlush = storage.flushWriteBarrier.bind(storage)
+    storage.flushWriteBarrier = async () => {
+      flushCount++
+      return origFlush()
+    }
+  })
+
+  afterEach(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  /** Index of the first sync call whose paths satisfy `pred` (or -1). */
+  const findSync = (pred: (p: string) => boolean): number =>
+    syncCalls.findIndex((paths) => paths.some(pred))
+
+  it('a committed transact fsyncs the entity write BEFORE the manifest', async () => {
+    const id = '00000000-0000-4000-8000-00000000ab01'
+    await brain.transact([{ op: 'add', id, data: 'durable entity', type: NounType.Thing }])
+
+    // The barrier opened and flushed exactly once.
+    expect(beginCount).toBe(1)
+    expect(flushCount).toBe(1)
+
+    // The entity's canonical metadata write was fsync'd…
+    const entitySyncIdx = findSync((p) => p.includes(`/${id}/`) && p.includes('entities/nouns'))
+    expect(entitySyncIdx).toBeGreaterThanOrEqual(0)
+    // …and the manifest (the commit point) was fsync'd in a LATER call.
+    const manifestSyncIdx = findSync((p) => p === MANIFEST_REL)
+    expect(manifestSyncIdx).toBeGreaterThanOrEqual(0)
+    expect(entitySyncIdx).toBeLessThan(manifestSyncIdx)
+  })
+
+  it('a multi-op transact (add + relate) fsyncs both endpoints and the edge before the manifest', async () => {
+    const a = '00000000-0000-4000-8000-00000000ab02'
+    const b = '00000000-0000-4000-8000-00000000ab03'
+    await brain.transact([
+      { op: 'add', id: a, data: 'A', type: NounType.Thing },
+      { op: 'add', id: b, data: 'B', type: NounType.Thing },
+      { op: 'relate', from: a, to: b, type: VerbType.Contains }
+    ])
+
+    expect(flushCount).toBe(1)
+    const aIdx = findSync((p) => p.includes(`/${a}/`))
+    const bIdx = findSync((p) => p.includes(`/${b}/`))
+    const manifestIdx = findSync((p) => p === MANIFEST_REL)
+    expect(aIdx).toBeGreaterThanOrEqual(0)
+    expect(bIdx).toBeGreaterThanOrEqual(0)
+    expect(manifestIdx).toBeGreaterThanOrEqual(0)
+    expect(Math.max(aIdx, bIdx)).toBeLessThan(manifestIdx)
+
+    // Sanity: the data is actually there and traverses.
+    expect((await brain.related(a)).map((r: any) => r.to)).toContain(b)
+  })
+
+  it('a precommit-rejected batch opens no barrier and advances nothing', async () => {
+    const anchor = await brain.add({ id: '00000000-0000-4000-8000-00000000ab04', data: 'anchor', type: NounType.Thing })
+    const genBefore = brain.generationStore.generation()
+    syncCalls.length = 0
+    beginCount = 0
+    flushCount = 0
+
+    const never = '00000000-0000-4000-8000-00000000ab05'
+    await expect(
+      brain.transact([
+        { op: 'add', id: never, data: 'never', type: NounType.Thing },
+        { op: 'update', id: anchor, metadata: { poke: 1 }, ifRev: 999 }
+      ])
+    ).rejects.toMatchObject({ name: 'RevisionConflictError' })
+
+    // Precommit throws before the barrier opens (it lives just before execute):
+    // the barrier never opens, nothing is flushed, and the generation counter is
+    // unchanged. (A manifest sync CAN appear here from transact() flushing the
+    // anchor's previously-buffered single-op — that is the anchor's deferred
+    // durability, not this rejected batch committing; beginCount is the clean
+    // signal that this batch's commit path never reached execute.)
+    expect(beginCount).toBe(0)
+    expect(flushCount).toBe(0)
+    expect(brain.generationStore.generation()).toBe(genBefore)
+    expect(await brain.get(never)).toBeNull()
+  })
+
+  it('MemoryStorage does not implement the barrier (optional-chaining no-op)', () => {
+    const mem = new MemoryStorage() as any
+    // In-memory has no durability seam; the generation store treats the absent
+    // barrier methods as no-ops via optional chaining.
+    expect(mem.beginWriteBarrier).toBeUndefined()
+    expect(mem.flushWriteBarrier).toBeUndefined()
+  })
+})

tests/integration/transact-forward-ref-graph.test.ts

@@ -0,0 +1,140 @@
+/**
+ * @module tests/integration/transact-forward-ref-graph
+ * @description Regression for a consumer-reported native/JS parity bug:
+ * `transact([{op:'add', id:X}, {op:'relate', to:X}])` — the platform's first
+ * atomic add+relate consumer — threw in `EntityIdMapper.getOrAssign` at PLAN
+ * time on the native id mapper (an entity added in the same batch does not
+ * exist when the planner runs, so a strict mapper rightly refuses to assign;
+ * the permissive JS mapper masked the bug — and silently leaked an int
+ * assignment whenever the batch was later rejected at precommit).
+ *
+ * Fix: graph-index operations resolve endpoint ints at EXECUTE time (a lazy
+ * thunk, mirroring the existing lazy `generationFn`), after the batch's add
+ * operations have applied. These tests pin every forward-ref shape on the JS
+ * path, including the JS-observable deferral proof: a REJECTED batch leaves
+ * the id mapper untouched (pre-fix, plan-time resolution assigned ints for
+ * entities that never came to exist). The native side is locked in by the
+ * accelerator's own gate running this same shape.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/brainy.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+
+let seq = 0
+const freshId = (): string =>
+  `00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
+
+describe('transact() forward references into the graph index (native/JS parity)', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-fwd-ref-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      dimensions: 384,
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  it('CASE 1 (the exact repro): add + relate-to-that-add in ONE transact', async () => {
+    const agent = await brain.add({ id: freshId(), data: 'agent probe', type: NounType.Person })
+    const thread = freshId()
+
+    const db = await brain.transact([
+      { op: 'add', id: thread, data: 'thread PROBE-ONE', type: NounType.Thing },
+      { op: 'relate', from: agent, to: thread, type: VerbType.RelatedTo }
+    ])
+
+    // The batch committed as one generation and the edge traverses.
+    expect(db.generation).toBeGreaterThan(0)
+    const related = await brain.related(agent)
+    expect(related.map((r: any) => r.to)).toContain(thread)
+  })
+
+  it('both endpoints created in-batch: add A + add B + relate A→B', async () => {
+    const a = freshId()
+    const b = freshId()
+    await brain.transact([
+      { op: 'add', id: a, data: 'node A', type: NounType.Thing },
+      { op: 'add', id: b, data: 'node B', type: NounType.Thing },
+      { op: 'relate', from: a, to: b, type: VerbType.Contains }
+    ])
+    expect((await brain.related(a)).map((r: any) => r.to)).toContain(b)
+  })
+
+  it('bidirectional relate to an in-batch add: both edges traverse', async () => {
+    const hub = await brain.add({ id: freshId(), data: 'hub', type: NounType.Thing })
+    const spoke = freshId()
+    await brain.transact([
+      { op: 'add', id: spoke, data: 'spoke', type: NounType.Thing },
+      { op: 'relate', from: hub, to: spoke, type: VerbType.RelatedTo, bidirectional: true }
+    ])
+    expect((await brain.related(hub)).map((r: any) => r.to)).toContain(spoke)
+    expect((await brain.related(spoke)).map((r: any) => r.to)).toContain(hub)
+  })
+
+  it('add + relate + remove in ONE transact: the cascade covers the in-batch verb', async () => {
+    const keeper = await brain.add({ id: freshId(), data: 'keeper', type: NounType.Thing })
+    const doomed = freshId()
+    await brain.transact([
+      { op: 'add', id: doomed, data: 'doomed', type: NounType.Thing },
+      { op: 'relate', from: keeper, to: doomed, type: VerbType.Contains },
+      { op: 'remove', id: doomed }
+    ])
+    expect(await brain.get(doomed)).toBeNull()
+    expect((await brain.related(keeper)).length).toBe(0)
+  })
+
+  it('CASE 2 control: the same two ops split across two transacts still work', async () => {
+    const agent = await brain.add({ id: freshId(), data: 'agent two', type: NounType.Person })
+    const thread = freshId()
+    await brain.transact([{ op: 'add', id: thread, data: 'thread two', type: NounType.Thing }])
+    await brain.transact([{ op: 'relate', from: agent, to: thread, type: VerbType.RelatedTo }])
+    expect((await brain.related(agent)).map((r: any) => r.to)).toContain(thread)
+  })
+
+  it('a REJECTED forward-ref batch leaves the id mapper untouched (the deferral proof)', async () => {
+    const existing = await brain.add({
+      id: freshId(),
+      data: 'cas anchor',
+      type: NounType.Thing
+    })
+    const never = freshId()
+
+    // Reject the batch via a stale per-op CAS — planning completes, the
+    // commit precondition throws, nothing applies.
+    await expect(
+      brain.transact([
+        { op: 'add', id: never, data: 'never exists', type: NounType.Thing },
+        { op: 'relate', from: existing, to: never, type: VerbType.RelatedTo },
+        { op: 'update', id: existing, metadata: { poke: 1 }, ifRev: 999 }
+      ])
+    ).rejects.toMatchObject({ name: 'RevisionConflictError' })
+
+    // Nothing applied…
+    expect(await brain.get(never)).toBeNull()
+    // …and the id mapper was never asked to assign for the phantom entity —
+    // pre-fix, plan-time resolution leaked an int here on every rejected batch.
+    const idMapper = brain['metadataIndex'].getIdMapper()
+    expect(idMapper.getInt(never)).toBeUndefined()
+
+    // The same shape then succeeds cleanly on retry.
+    await brain.transact([
+      { op: 'add', id: never, data: 'exists now', type: NounType.Thing },
+      { op: 'relate', from: existing, to: never, type: VerbType.RelatedTo }
+    ])
+    expect((await brain.related(existing)).map((r: any) => r.to)).toContain(never)
+  })
+})

tests/integration/vfs-rename-containment.test.ts

@@ -0,0 +1,157 @@
+/**
+ * @module tests/integration/vfs-rename-containment
+ * @description A cross-directory rename must MOVE the containment edge, not
+ * accumulate one per parent. The pre-fix rename added the new parent's
+ * Contains edge but skipped removing the old one ("not critical") — leaving
+ * the entity a child of BOTH directories: readdir(oldDir) kept listing it,
+ * re-creating the old path showed the name twice (the duplicate-readdir /
+ * "cosmetic ghost" field report), and tree-walking consumers saw the file in
+ * two places. Also covers the repair sweep (repairIndex → vfs.repairContainment)
+ * that heals ghosts left by earlier versions, and move-to-root (whose edge
+ * used to be skipped entirely).
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import { Brainy, VerbType } from '../../src/index.js'
+
+describe('VFS rename moves the containment edge (no ghost in the old directory)', () => {
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, silent: true, dimensions: 384 })
+    await brain.init()
+    await brain.vfs.mkdir('/a', { recursive: true })
+    await brain.vfs.mkdir('/b', { recursive: true })
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+  })
+
+  it('cross-directory move: old dir stops listing it, new dir lists it exactly once', async () => {
+    await brain.vfs.writeFile('/a/x.txt', 'v1')
+    await brain.vfs.rename('/a/x.txt', '/b/x.txt')
+
+    expect(await brain.vfs.readdir('/a')).toEqual([])
+    expect(await brain.vfs.readdir('/b')).toEqual(['x.txt'])
+    expect(String(await brain.vfs.readFile('/b/x.txt'))).toBe('v1')
+  })
+
+  it('re-creating the old path lists each name exactly once in each directory', async () => {
+    await brain.vfs.writeFile('/a/x.txt', 'moved away')
+    await brain.vfs.rename('/a/x.txt', '/b/x.txt')
+    await brain.vfs.writeFile('/a/x.txt', 'new file at old path')
+
+    const a = (await brain.vfs.readdir('/a')) as string[]
+    const b = (await brain.vfs.readdir('/b')) as string[]
+    expect(a).toEqual(['x.txt']) // exactly once — the pre-fix ghost made this list the moved entity too
+    expect(b).toEqual(['x.txt'])
+    expect(String(await brain.vfs.readFile('/a/x.txt'))).toBe('new file at old path')
+    expect(String(await brain.vfs.readFile('/b/x.txt'))).toBe('moved away')
+  })
+
+  it('repeated moves never accumulate containment edges', async () => {
+    await brain.vfs.writeFile('/a/f.txt', 'wanderer')
+    for (let i = 0; i < 3; i++) {
+      await brain.vfs.rename('/a/f.txt', '/b/f.txt')
+      await brain.vfs.rename('/b/f.txt', '/a/f.txt')
+    }
+    expect(await brain.vfs.readdir('/a')).toEqual(['f.txt'])
+    expect(await brain.vfs.readdir('/b')).toEqual([])
+
+    // Exactly ONE containment edge exists on the entity.
+    const stat = await brain.vfs.stat('/a/f.txt')
+    const edges = await brain.related({ to: stat.entityId, type: VerbType.Contains })
+    expect(edges).toHaveLength(1)
+  })
+
+  it('move to the root gets a containment edge (used to be skipped → orphan)', async () => {
+    await brain.vfs.writeFile('/a/up.txt', 'to the top')
+    await brain.vfs.rename('/a/up.txt', '/up.txt')
+
+    const rootListing = (await brain.vfs.readdir('/')) as string[]
+    expect(rootListing).toContain('up.txt')
+    expect(await brain.vfs.readdir('/a')).toEqual([])
+    expect(String(await brain.vfs.readFile('/up.txt'))).toBe('to the top')
+  })
+})
+
+describe('repairIndex() heals pre-fix containment ghosts (vfs.repairContainment)', () => {
+  let brain: any
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, silent: true, dimensions: 384 })
+    await brain.init()
+    await brain.vfs.mkdir('/a', { recursive: true })
+    await brain.vfs.mkdir('/b', { recursive: true })
+  })
+  afterEach(async () => {
+    await brain.close?.().catch(() => {})
+  })
+
+  /** Reproduce the PRE-FIX defect state: file lives at /b/g.txt but a stale
+   *  vfs-contains edge from /a lingers (what old renames left behind). */
+  const synthesizeGhost = async () => {
+    await brain.vfs.writeFile('/b/g.txt', 'ghost target')
+    const fileId = (await brain.vfs.stat('/b/g.txt')).entityId
+    const aId = (await brain.vfs.stat('/a')).entityId
+    await brain.relate({
+      from: aId,
+      to: fileId,
+      type: VerbType.Contains,
+      subtype: 'vfs-contains',
+      metadata: { isVFS: true }
+    })
+    return { fileId, aId }
+  }
+
+  it('a stale old-parent edge is removed; listings become honest', async () => {
+    const { fileId } = await synthesizeGhost()
+    // The defect state is visible: /a lists a file whose path says /b.
+    expect((await brain.vfs.readdir('/a')) as string[]).toContain('g.txt')
+
+    await brain.repairIndex()
+
+    expect(await brain.vfs.readdir('/a')).toEqual([])
+    expect(await brain.vfs.readdir('/b')).toEqual(['g.txt'])
+    const edges = await brain.related({ to: fileId, type: VerbType.Contains })
+    expect(edges).toHaveLength(1)
+  })
+
+  it("a user's own knowledge Contains edge onto the file survives the repair", async () => {
+    const { fileId } = await synthesizeGhost()
+    // A knowledge-graph containment from a NON-directory entity (a collection
+    // curating the file) — same verb TYPE, not a vfs-contains edge. The repair
+    // must remove only VFS containment ghosts, never user knowledge edges.
+    // (Edges dedupe by (from,to,type), so the user edge needs its own source.)
+    const collectionId = await brain.add({
+      data: 'reading list',
+      type: 'collection',
+      metadata: { kind: 'curation' }
+    })
+    await brain.relate({ from: collectionId, to: fileId, type: VerbType.Contains, subtype: 'curates' })
+
+    await brain.repairIndex()
+
+    const edges = await brain.related({ to: fileId, type: VerbType.Contains })
+    const subtypes = edges.map((e: any) => e.subtype).sort()
+    // The stale vfs ghost edge is gone; the correct vfs edge + the user's
+    // curation edge remain.
+    expect(subtypes).toEqual(['curates', 'vfs-contains'])
+    expect(edges.find((e: any) => e.subtype === 'curates')?.from).toBe(collectionId)
+  })
+
+  it('a missing expected edge is restored (entity unreachable from its own directory)', async () => {
+    await brain.vfs.writeFile('/b/lost.txt', 'find me')
+    const fileId = (await brain.vfs.stat('/b/lost.txt')).entityId
+    // Simulate total edge loss (an older damage shape).
+    for (const e of await brain.related({ to: fileId, type: VerbType.Contains })) {
+      await brain.unrelate(e.id)
+    }
+    expect((await brain.vfs.readdir('/b')) as string[]).not.toContain('lost.txt')
+
+    await brain.repairIndex()
+
+    expect((await brain.vfs.readdir('/b')) as string[]).toContain('lost.txt')
+  })
+})

tests/regression/metadata-index-cleanup.unit.test.ts

@@ -205,10 +205,10 @@ describe('Metadata index cleanup after remove / removeMany', () => {
       }
     })
 
-    it('handles empty ids array gracefully', async () => {
-      const result = await brain.removeMany({ ids: [] })
-      expect(result.successful).toHaveLength(0)
-      expect(result.failed).toHaveLength(0)
+    it('refuses an empty ids array loudly (a silent no-op is not "graceful")', async () => {
+      // 8.8.2: an empty selector used to resolve successfully having deleted
+      // NOTHING — the caller believed the delete happened. Now it throws.
+      await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
     })
 
     it('handles large batch (> 1 chunk) without leaving stale index entries', async () => {

tests/transaction/TransactionManager.unit.test.ts

@@ -10,6 +10,7 @@
 
 import { describe, it, expect, beforeEach } from 'vitest'
 import { TransactionManager } from '../../src/transaction/TransactionManager.js'
+import { transactTimeoutBudget } from '../../src/transaction/Transaction.js'
 import { TransactionError } from '../../src/transaction/errors.js'
 
 describe('TransactionManager', () => {
@@ -105,14 +106,17 @@ describe('TransactionManager', () => {
       const result = await manager.executeTransactionWithResult(async (tx) => {
         tx.addOperation({
           execute: async () => {
-            await new Promise(resolve => setTimeout(resolve, 10))
+            await new Promise(resolve => setTimeout(resolve, 25))
             return async () => {}
           }
         })
         return 'done'
       })
 
-      expect(result.executionTimeMs).toBeGreaterThanOrEqual(10)
+      // Timer coalescing can fire a setTimeout up to a few ms EARLY under
+      // load, so assert well below the sleep — this tests that time is
+      // MEASURED, not the OS timer's precision.
+      expect(result.executionTimeMs).toBeGreaterThanOrEqual(20)
     })
   })
 
@@ -325,4 +329,45 @@ describe('TransactionManager', () => {
       expect(stats1).toEqual(stats2) // Same values
     })
   })
+
+  describe('Timeout budget + telemetry', () => {
+    it('transactTimeoutBudget: explicit override wins; default scales with batch size', () => {
+      expect(transactTimeoutBudget(1)).toBe(30_000) // small batches keep the 30s floor
+      expect(transactTimeoutBudget(15)).toBe(30_000) // the old flat cap's break-even point
+      expect(transactTimeoutBudget(100)).toBe(200_000) // 100 ops × 2s — bulk gets an honest budget
+      expect(transactTimeoutBudget(1000, 5_000)).toBe(5_000) // caller override is untouched
+    })
+
+    it('a tripped budget rolls back and names the operation, progress, and budget', async () => {
+      const rolledBack: string[] = []
+
+      const failing = manager.executeTransaction(
+        async (tx) => {
+          tx.addOperation({
+            name: 'slow-first-op',
+            execute: async () => {
+              await new Promise((r) => setTimeout(r, 30))
+              return async () => {
+                rolledBack.push('slow-first-op')
+              }
+            }
+          })
+          tx.addOperation({
+            name: 'never-reached',
+            execute: async () => undefined
+          })
+        },
+        { timeout: 5 } // the first op's 30ms sleep guarantees the pre-op-2 check trips
+      )
+
+      await expect(failing).rejects.toThrow(TransactionError)
+      const err = await failing.catch((e) => e)
+      expect(err.name).toBe('TransactionTimeoutError')
+      expect(err.message).toContain('operation 1/2') // which op, of how many
+      expect(err.message).toContain("('never-reached')") // its name
+      expect(err.message).toContain('budget 5ms') // the budget that tripped
+      expect(err.message).toContain('rolled back') // the retryability statement
+      expect(rolledBack).toEqual(['slow-first-op']) // the applied op was undone
+    })
+  })
 })

tests/unit/aggregation/AggregationIndex.test.ts

@@ -675,4 +675,104 @@ describe('AggregationIndex', () => {
       await reloaded.close()
     })
   })
+
+  // ============= Boot-order reconciliation =============
+  //
+  // The production boot pattern: defineAggregate() is synchronous and always
+  // beats the async init() that loads persisted state. The old code flagged a
+  // backfill at define time and init never cleared it — so the loaded state
+  // was wiped and the whole store re-walked on EVERY restart.
+
+  describe('boot-order reconciliation (define-before-init)', () => {
+    const DEF: AggregateDefinition = {
+      name: 'boot_agg',
+      source: { type: NounType.Event },
+      groupBy: ['category'],
+      metrics: { count: { op: 'count' } }
+    }
+
+    const entity = (id: string, category: string): Record<string, unknown> => ({
+      id,
+      noun: NounType.Event,
+      metadata: { category },
+      createdAt: Date.now(),
+      updatedAt: Date.now()
+    })
+
+    /** Simulate the previous session: define, contribute, flush. */
+    async function seedAndFlush(store: MemoryStorage): Promise<void> {
+      const first = new AggregationIndex(store)
+      await first.init()
+      first.defineAggregate(DEF)
+      first.onEntityAdded('e1', entity('e1', 'food'))
+      first.onEntityAdded('e2', entity('e2', 'food'))
+      first.onEntityAdded('e3', entity('e3', 'transport'))
+      await first.flush()
+    }
+
+    it('adopts persisted state when define beats init with an unchanged definition', async () => {
+      const store = new MemoryStorage()
+      await store.init()
+      await seedAndFlush(store)
+
+      const second = new AggregationIndex(store)
+      second.defineAggregate(DEF) // synchronous define FIRST — the real boot order
+      await second.init()
+      await second.ready()
+
+      expect(second.getPendingBackfills()).toEqual([])
+      const rows = second.queryAggregate({ name: 'boot_agg' })
+      const food = rows.find(r => r.groupKey.category === 'food')!
+      expect(food.metrics.count).toBe(2)
+    })
+
+    it('a write landing before adoption forces an exact rescan instead', async () => {
+      const store = new MemoryStorage()
+      await store.init()
+      await seedAndFlush(store)
+
+      const second = new AggregationIndex(store)
+      second.defineAggregate(DEF)
+      second.onEntityAdded('e4', entity('e4', 'food')) // lands before init settles
+      await second.init()
+      await second.ready()
+
+      // Adoption would lose e4's contribution — the engine must rescan.
+      expect(second.getPendingBackfills()).toEqual(['boot_agg'])
+    })
+
+    it('init never clobbers a changed app definition registered before it', async () => {
+      const store = new MemoryStorage()
+      await store.init()
+      await seedAndFlush(store)
+
+      const CHANGED: AggregateDefinition = {
+        ...DEF,
+        metrics: { count: { op: 'count' }, total: { op: 'sum', field: 'amount' } }
+      }
+      const second = new AggregationIndex(store)
+      second.defineAggregate(CHANGED)
+      await second.init()
+      await second.ready()
+
+      const def = second.getDefinitions().find(d => d.name === 'boot_agg')!
+      expect(Object.keys(def.metrics).sort()).toEqual(['count', 'total'])
+      expect(second.getPendingBackfills()).toEqual(['boot_agg'])
+    })
+
+    it('init alone restores persisted definitions with adopted state, no backfill', async () => {
+      const store = new MemoryStorage()
+      await store.init()
+      await seedAndFlush(store)
+
+      const second = new AggregationIndex(store)
+      await second.init()
+      await second.ready()
+
+      expect(second.hasAggregate('boot_agg')).toBe(true)
+      expect(second.getPendingBackfills()).toEqual([])
+      const rows = second.queryAggregate({ name: 'boot_agg' })
+      expect(rows.reduce((s, r) => s + (r.metrics.count as number), 0)).toBe(3)
+    })
+  })
 })

tests/unit/aggregation/materialize-failure-loud.test.ts

@@ -0,0 +1,51 @@
+/**
+ * @module tests/unit/aggregation/materialize-failure-loud
+ * @description Aggregation (Pattern C): a failed materialization must not be
+ * swallowed. `scheduleMaterialize`'s debounced `materializeOne(key).catch(() => {})`
+ * silently discarded the error, leaving the materialized Measurement entity stale
+ * with no signal. It now emits a loud warning (the value is still rebuildable via
+ * backfill-on-query, so it stays non-fatal — but visible). Load-bearing assertion:
+ * prodLog.warn fires instead of an empty catch.
+ */
+import { describe, it, expect, vi, afterEach } from 'vitest'
+import { AggregateMaterializer } from '../../../src/aggregation/materializer.js'
+import { prodLog } from '../../../src/utils/logger.js'
+import { NounType } from '../../../src/types/graphTypes.js'
+
+describe('Aggregation — swallowed materialize failure is surfaced', () => {
+  afterEach(() => {
+    vi.restoreAllMocks()
+    vi.useRealTimers()
+  })
+
+  it('logs loudly when materializeOne rejects', async () => {
+    vi.useFakeTimers()
+    const warn = vi.spyOn(prodLog, 'warn').mockImplementation(() => {})
+
+    // Brain access whose add()/update() always reject → doMaterialize throws.
+    const brain = {
+      add: vi.fn().mockRejectedValue(new Error('storage down')),
+      update: vi.fn().mockRejectedValue(new Error('storage down'))
+    }
+    const m = new AggregateMaterializer(brain as any, 10)
+    const def: any = {
+      name: 'sales',
+      source: { type: NounType.Document },
+      groupBy: ['region'],
+      metrics: { total: { op: 'count' } },
+      materialize: { debounceMs: 10 }
+    }
+    m.scheduleMaterialize(
+      'sales',
+      def,
+      { region: 'us' },
+      { groupKey: { region: 'us' }, metrics: { total: { count: 1 } } } as any
+    )
+
+    await vi.advanceTimersByTimeAsync(20) // fire the debounce timer
+    await Promise.resolve()
+
+    expect(warn).toHaveBeenCalled()
+    expect(String(warn.mock.calls[0][0])).toMatch(/Failed to materialize aggregate group/)
+  })
+})

tests/unit/brainy/batch-operations.test.ts

@@ -533,8 +533,10 @@ describe('Brainy Batch Operations', () => {
       expect(result.successful).toHaveLength(0)
       
       await brain.updateMany({ items: [] })
-      await brain.removeMany({ ids: [] })
-      // Should not throw
+      // removeMany is the exception (8.8.2): an empty id list is a refused
+      // selector, not an empty batch — deleting "nothing" silently was the
+      // bug class (a positional/bare-array call looked identical).
+      await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
     })
     
     it('should validate batch size limits', async () => {

tests/unit/brainy/degraded-reads-surfaced.test.ts

@@ -0,0 +1,79 @@
+/**
+ * @module tests/unit/brainy/degraded-reads-surfaced
+ * @description Finding 10: a known-degraded derived index must be SURFACED, not
+ * silently served as authoritative. Two degraded sources:
+ *   (a) a non-fatal index rebuild failure at init() (`_indexRebuildFailed`), and
+ *   (b) an adopt-forward failed-rollback recovery (`commitSingleOp`'s `degraded`
+ *       ids, previously dropped on the floor by persistSingleOp).
+ * Both now fold into checkHealth()/validateIndexConsistency() and emit ONE loud
+ * read-path warning per degraded window; repairIndex() reconciles + clears them.
+ *
+ * The private fields are the observable contract of the fix, so the test drives
+ * them directly (a real failed rollback is exercised elsewhere).
+ */
+import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
+import { Brainy } from '../../../src/brainy.js'
+import { NounType } from '../../../src/types/graphTypes.js'
+import { prodLog } from '../../../src/utils/logger.js'
+
+const UUID = (suffix: string): string => `00000000-0000-4000-8000-0000000000${suffix}`
+
+describe('Finding 10 — degraded derived-index state is surfaced on reads', () => {
+  beforeEach(() => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+  })
+  afterEach(() => vi.restoreAllMocks())
+
+  it('checkHealth() reports adopt-forward degraded ids as unhealthy', async () => {
+    const brain = new Brainy({ storage: { type: 'memory' }, dimensions: 384, requireSubtype: false })
+    await brain.init()
+    ;(brain as any)._indexDegradedIds.add(UUID('de'))
+
+    const health = await brain.checkHealth()
+    expect(health.healthy).toBe(false)
+    expect(health.recommendation).toMatch(/repairIndex\(\)/)
+  })
+
+  it('find()/get() warn loudly while degraded, ONCE, then repairIndex() clears it', async () => {
+    const warn = vi.spyOn(prodLog, 'warn').mockImplementation(() => {})
+    const brain = new Brainy({ storage: { type: 'memory' }, dimensions: 384, requireSubtype: false })
+    await brain.init()
+    await brain.add({ id: UUID('a1'), data: 'x', type: NounType.Document })
+    ;(brain as any)._indexRebuildFailed = new Error('rebuild boom')
+
+    await brain.find({ type: NounType.Document })
+    await brain.get(UUID('a1')) // second read: must NOT double-warn
+    const degradedWarns = warn.mock.calls.filter((c) =>
+      String(c[0]).includes('derived index is INCOMPLETE')
+    )
+    expect(degradedWarns.length).toBe(1)
+
+    await brain.repairIndex()
+    expect((brain as any)._indexRebuildFailed).toBeNull()
+    expect((brain as any)._indexDegradedIds.size).toBe(0)
+
+    warn.mockClear()
+    await brain.find({ type: NounType.Document })
+    expect(warn.mock.calls.filter((c) => String(c[0]).includes('INCOMPLETE')).length).toBe(0)
+  })
+
+  it('persistSingleOp records receipt.degraded (widened return type, not dropped)', async () => {
+    const brain = new Brainy({ storage: { type: 'memory' }, dimensions: 384, requireSubtype: false })
+    await brain.init()
+    // Simulate a degraded receipt by wrapping the generation store's commitSingleOp.
+    const gs: any = (brain as any).generationStore
+    const realCommit = gs.commitSingleOp.bind(gs)
+    vi.spyOn(gs, 'commitSingleOp').mockImplementation(async (args: any) => {
+      const r = await realCommit(args)
+      return { ...r, degraded: [...(args.touched.nouns ?? [])] }
+    })
+
+    await brain.add({ id: UUID('b2'), data: 'y', type: NounType.Document })
+    expect((brain as any)._indexDegradedIds.size).toBeGreaterThan(0)
+    expect((brain as any)._indexDegradedIds.has(UUID('b2'))).toBe(true)
+
+    const health = await brain.checkHealth()
+    expect(health.healthy).toBe(false)
+    expect(health.recommendation).toMatch(/repairIndex\(\)/)
+  })
+})

tests/unit/brainy/migration-gate-family-scoped.test.ts

@@ -0,0 +1,94 @@
+/**
+ * @module tests/unit/brainy/migration-gate-family-scoped
+ * @description The coordinated migration LOCK is FAMILY-SCOPED: a read waits only
+ * on the index families it actually consults. A one-time native migration of ONE
+ * family (its `isMigrating()` held) must NOT block a read served entirely from
+ * canonical storage (get / VFS content + dir) or from a HEALTHY family — only a
+ * read that needs the migrating family blocks. Regression for the whole-brain
+ * gate that hung getStats / readdir / readFile behind an unrelated family's
+ * migration until the wait timed out.
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { Brainy } from '../../../src/brainy.js'
+import { MigrationInProgressError } from '../../../src/errors/brainyError.js'
+
+const seed = async () => {
+  const brain = new Brainy({
+    storage: { type: 'memory' },
+    dimensions: 384,
+    requireSubtype: false,
+    silent: true,
+    // Short so a genuine block resolves fast; a real block would otherwise wait
+    // the 30 s default — this test asserts the wait does NOT happen for scoped
+    // reads, and DOES (bounded) for the one that needs the migrating family.
+    migrationWaitTimeoutMs: 300
+  })
+  await brain.init()
+  for (let i = 0; i < 5; i++) {
+    await brain.add({ data: `doc ${i}`, type: 'document', metadata: { i } })
+  }
+  await brain.vfs.writeFile('/notes/hello.txt', 'canonical bytes — no index needed')
+  await brain.flush()
+  return brain
+}
+
+/** Force a provider to report an in-flight (stuck) migration. */
+const jam = (provider: unknown) => {
+  ;(provider as { isMigrating: () => boolean }).isMigrating = () => true
+}
+
+describe('migration LOCK is family-scoped', () => {
+  beforeEach(() => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+  })
+
+  it('a stuck VECTOR migration does not block canonical or graph/metadata reads', async () => {
+    const brain = await seed()
+    const childId = (
+      (await brain.vfs.readdir('/notes', { withFileTypes: true })) as Array<{ entityId: string }>
+    )[0].entityId
+
+    jam((brain as any).index) // vector provider migrating; graph + metadata healthy
+
+    // None of these consult the vector family → they serve immediately.
+    await expect(brain.getStats()).resolves.toBeDefined()
+    await expect(brain.vfs.readdir('/notes')).resolves.toHaveLength(1) // graph traversal
+    await expect(brain.vfs.readFile('/notes/hello.txt')).resolves.toBeDefined() // canonical blob
+    await expect(brain.get(childId)).resolves.not.toBeNull() // canonical by id
+    await expect(brain.find({ where: { i: 1 } })).resolves.toBeDefined() // metadata filter
+  })
+
+  it('a stuck VECTOR migration STILL blocks a read that needs the vector family', async () => {
+    const brain = await seed()
+    jam((brain as any).index)
+
+    // A semantic query consults the vector index — it must wait, and (bounded by
+    // migrationWaitTimeoutMs) surface the retryable MigrationInProgressError
+    // rather than serve a half-built result.
+    await expect(brain.find({ query: 'doc' })).rejects.toBeInstanceOf(MigrationInProgressError)
+  })
+
+  it('a stuck GRAPH migration blocks traversal but not vector/canonical reads', async () => {
+    const brain = await seed()
+    const childId = (
+      (await brain.vfs.readdir('/notes', { withFileTypes: true })) as Array<{ entityId: string }>
+    )[0].entityId
+
+    jam((brain as any).graphIndex) // graph provider migrating; vector + metadata healthy
+
+    // Canonical + vector + metadata are unaffected.
+    await expect(brain.get(childId)).resolves.not.toBeNull()
+    await expect(brain.find({ query: 'doc' })).resolves.toBeDefined()
+    await expect(brain.find({ where: { i: 1 } })).resolves.toBeDefined()
+
+    // A graph traversal needs the migrating family → it blocks (bounded).
+    await expect(brain.related({ from: childId })).rejects.toBeInstanceOf(MigrationInProgressError)
+  })
+
+  it('with no migration in flight, every read serves (the fast path is a no-op)', async () => {
+    const brain = await seed()
+    await expect(brain.getStats()).resolves.toBeDefined()
+    await expect(brain.find({ query: 'doc' })).resolves.toBeDefined()
+    await expect(brain.vfs.readdir('/notes')).resolves.toHaveLength(1)
+  })
+})

tests/unit/db/fact-log.test.ts

@@ -0,0 +1,237 @@
+/**
+ * @module tests/unit/db/fact-log
+ * @description The generation fact log in isolation: wire-format round-trip
+ * (positional msgpack facts, bin16 uuids, body-less tombstones), crc32c
+ * framing with torn-tail detection, open-time truncation to committed truth
+ * (the log can only ever be AHEAD after a crash; open cuts it back), rotation
+ * with a manifest-first flip, exactly-once scans with the frozen telemetry
+ * shape, and the mmap segment handoff excluding the mutable tail.
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+import {
+  FactLog,
+  FACTS_PREFIX,
+  type CommitFact,
+  type FactLogStorage,
+  storageSupportsFactLog
+} from '../../../src/db/factLog.js'
+import { crc32c } from '../../../src/utils/crc32c.js'
+
+const UUID = (n: number): string =>
+  `00000000-0000-4000-8000-${String(n).padStart(12, '0')}`
+
+const fact = (generation: number, overrides?: Partial<CommitFact>): CommitFact => ({
+  generation,
+  timestamp: 1_700_000_000_000 + generation,
+  ops: [
+    {
+      kind: 'noun',
+      id: UUID(generation),
+      record: { metadata: { noun: 'document', title: `doc ${generation}` }, vector: { v: [1, 2] } }
+    }
+  ],
+  ...overrides
+})
+
+describe('crc32c known-answer vectors', () => {
+  it('matches the RFC 3720 test vectors', () => {
+    expect(crc32c(new TextEncoder().encode('123456789'))).toBe(0xe3069283)
+    expect(crc32c(new Uint8Array(32))).toBe(0x8a9136aa)
+  })
+})
+
+describe('fact log — round-trip, framing, reconcile, rotation, scan', () => {
+  let storage: FactLogStorage
+  let log: FactLog
+
+  beforeEach(async () => {
+    const mem: any = new MemoryStorage()
+    await mem.init()
+    expect(storageSupportsFactLog(mem)).toBe(true)
+    storage = mem
+    log = new FactLog(storage)
+    await log.open(0)
+  })
+
+  it('facts round-trip byte-exactly: ops, tombstones, meta, blobHashes', async () => {
+    await log.append(fact(1))
+    await log.append(
+      fact(2, {
+        ops: [
+          { kind: 'verb', id: UUID(21), record: { metadata: { verb: 'contains' }, vector: null } },
+          { kind: 'noun', id: UUID(22), record: null } // TOMBSTONE
+        ],
+        meta: { source: 'test' },
+        blobHashes: ['abc123', 'abc123'] // multiset — duplicates preserved
+      })
+    )
+    await log.sync()
+
+    const scan = log.scanFacts()
+    expect(scan.headGeneration).toBe(2)
+    const all: CommitFact[] = []
+    for await (const batch of scan.batches()) all.push(...batch.facts)
+
+    expect(all).toHaveLength(2)
+    expect(all[0].generation).toBe(1)
+    expect(all[0].ops[0].id).toBe(UUID(1))
+    expect(all[0].ops[0].record?.metadata).toEqual({ noun: 'document', title: 'doc 1' })
+    expect(all[1].ops[0].kind).toBe('verb')
+    expect(all[1].ops[1].record).toBeNull() // the tombstone is body-less
+    expect(all[1].meta).toEqual({ source: 'test' })
+    expect(all[1].blobHashes).toEqual(['abc123', 'abc123'])
+    expect(scan.summary().factsYielded).toBe(2)
+  })
+
+  it('appends are monotonic — a replayed/duplicate generation throws', async () => {
+    await log.append(fact(5))
+    await expect(log.append(fact(5))).rejects.toThrow(/non-monotonic/)
+    await expect(log.append(fact(3))).rejects.toThrow(/non-monotonic/)
+    await expect(log.append(fact(6))).resolves.toBeUndefined() // gaps are fine (aborted reservations)
+  })
+
+  it('a torn tail (partial frame) is detected and ignored — intact prefix survives', async () => {
+    await log.append(fact(1))
+    await log.append(fact(2))
+    await log.sync()
+
+    // Simulate a crash mid-append: chop bytes off the tail file.
+    const tailPath = `${FACTS_PREFIX}/seg-${'1'.padStart(20, '0')}.bfl`
+    const bytes = (await storage.readRawBytes(tailPath))!
+    await storage.writeRawBytes(tailPath, bytes.subarray(0, bytes.length - 7))
+
+    const reopened = new FactLog(storage)
+    await reopened.open(2)
+    expect(reopened.headGeneration()).toBe(1) // fact 2's frame was torn → gone
+
+    const all: CommitFact[] = []
+    for await (const b of reopened.scanFacts().batches()) all.push(...b.facts)
+    expect(all.map((f) => f.generation)).toEqual([1])
+  })
+
+  it('open() truncates facts beyond committed truth (the crash-ahead shape)', async () => {
+    await log.append(fact(1))
+    await log.append(fact(2))
+    await log.append(fact(3))
+    await log.sync()
+
+    // The store's committed generation is 1 — facts 2..3 never committed.
+    const reopened = new FactLog(storage)
+    await reopened.open(1)
+    expect(reopened.headGeneration()).toBe(1)
+
+    const all: CommitFact[] = []
+    for await (const b of reopened.scanFacts().batches()) all.push(...b.facts)
+    expect(all.map((f) => f.generation)).toEqual([1])
+
+    // And appends continue cleanly from the truncated head.
+    await reopened.append(fact(2))
+    expect(reopened.headGeneration()).toBe(2)
+  })
+
+  it('a cleared store (committed=0) truncates everything', async () => {
+    await log.append(fact(1))
+    await log.append(fact(2))
+    await log.sync()
+    const reopened = new FactLog(storage)
+    await reopened.open(0)
+    expect(reopened.headGeneration()).toBe(0)
+  })
+
+  it('scan honors fromGeneration/toGeneration inclusively and filters kinds', async () => {
+    for (let g = 1; g <= 6; g++) await log.append(fact(g))
+    await log.sync()
+
+    const scan = log.scanFacts({ fromGeneration: 2, toGeneration: 4 })
+    const all: CommitFact[] = []
+    for await (const b of scan.batches()) all.push(...b.facts)
+    expect(all.map((f) => f.generation)).toEqual([2, 3, 4])
+
+    const verbsOnly = log.scanFacts({ kinds: ['verb'] })
+    for await (const b of verbsOnly.batches()) {
+      for (const f of b.facts) expect(f.ops.every((op) => op.kind === 'verb')).toBe(true)
+    }
+  })
+
+  it('batch telemetry carries the frozen shape', async () => {
+    for (let g = 1; g <= 5; g++) await log.append(fact(g))
+    await log.sync()
+
+    const scan = log.scanFacts({ batchSize: 2 })
+    expect(scan.approxFactCount).toBe(5)
+    const batches = []
+    for await (const b of scan.batches()) batches.push(b)
+    expect(batches.length).toBe(3)
+    expect(batches[0]).toMatchObject({ firstGeneration: 1, lastGeneration: 2, factCount: 2 })
+    expect(batches[0].byteSize).toBeGreaterThan(0)
+    expect(typeof batches[0].segmentId).toBe('string')
+    expect(scan.summary()).toEqual({ factsYielded: 5, segmentsRead: 1 })
+  })
+
+  it('survives reopen: head and content come back from disk', async () => {
+    for (let g = 1; g <= 3; g++) await log.append(fact(g))
+    await log.sync()
+
+    const reopened = new FactLog(storage)
+    await reopened.open(3)
+    expect(reopened.headGeneration()).toBe(3)
+    const all: CommitFact[] = []
+    for await (const b of reopened.scanFacts().batches()) all.push(...b.facts)
+    expect(all.map((f) => f.generation)).toEqual([1, 2, 3])
+  })
+
+  it('segmentPaths excludes the mutable tail (mmap handoff = sealed only)', async () => {
+    await log.append(fact(1))
+    await log.sync()
+    expect(log.segmentPaths()).toEqual([]) // only a tail exists — nothing sealed
+  })
+
+  describe('scanFacts liveness contract (Stage-2 D1)', () => {
+    it('a wedged store fails LOUDLY within the first-batch bound — never a silent hang', async () => {
+      // Force a sealed segment (tiny rotateBytes) so the scan must READ from
+      // storage, then wedge that read: the exact production shape (a
+      // backlogged brain whose segment read never returned).
+      const mem: any = new MemoryStorage()
+      await mem.init()
+      const wedgeable = new FactLog(mem, { rotateBytes: 1 })
+      await wedgeable.open(0)
+      await wedgeable.append(fact(1))
+      await wedgeable.append(fact(2)) // second append rotates → seg 1 sealed
+      await wedgeable.sync()
+
+      const realRead = mem.readRawBytes.bind(mem)
+      mem.readRawBytes = (p: string) =>
+        p.includes('facts/seg-') ? new Promise(() => {}) : realRead(p) // hangs forever
+
+      const scan = wedgeable.scanFacts({ firstBatchTimeoutMs: 200 })
+      const started = Date.now()
+      await expect(scan.batches().next()).rejects.toThrow(/no first batch within 200ms/)
+      expect(Date.now() - started).toBeLessThan(5_000) // bound held, not a hang
+    })
+
+    it('a healthy scan is unaffected — first batch well inside the bound, all facts delivered', async () => {
+      for (let g = 1; g <= 5; g++) await log.append(fact(g))
+      await log.sync()
+      const scan = log.scanFacts({ batchSize: 2 })
+      const all: CommitFact[] = []
+      for await (const b of scan.batches()) all.push(...b.facts)
+      expect(all.map((f) => f.generation)).toEqual([1, 2, 3, 4, 5])
+      expect(scan.summary().factsYielded).toBe(5)
+    })
+
+    it('consumer think-time between pulls never counts against the producer', async () => {
+      for (let g = 1; g <= 4; g++) await log.append(fact(g))
+      await log.sync()
+      // Bound tighter than the consumer's pause: only the FIRST pull is
+      // raced, so a slow consumer after batch 1 must not trip the deadline.
+      const gen = log.scanFacts({ batchSize: 2, firstBatchTimeoutMs: 150 }).batches()
+      const first = await gen.next()
+      expect(first.done).toBe(false)
+      await new Promise((r) => setTimeout(r, 400)) // dawdle past the bound
+      const second = await gen.next()
+      expect(second.done).toBe(false)
+      expect((await gen.next()).done).toBe(true)
+    })
+  })
+})

tests/unit/db/generation-segments.test.ts

@@ -0,0 +1,150 @@
+/**
+ * @module tests/unit/db/generation-segments
+ * @description The generation-segment store (Stage-2 D1+D3 file format).
+ * Laws: (1) fold → read round-trips deltas and records byte-faithfully via
+ * sidecar point-reads; (2) the manifest is the ONLY discovery path — reopen
+ * reads one file, never a listing; (3) a lost/corrupt sidecar rebuilds from
+ * its segment loudly, a damaged SEGMENT fails loudly (never silent wrong
+ * data); (4) D3 reclaim drops whole segments only and bumps compactedBelow;
+ * (5) the packed digest is deterministic across reopen; (6) immutability —
+ * fold refuses overlap with sealed ranges.
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+import {
+  GenerationSegmentStore,
+  SEGMENTS_PREFIX,
+  type FoldGeneration
+} from '../../../src/db/generationSegments.js'
+
+const UUID = (n: number): string => `00000000-0000-4000-8000-${String(n).padStart(12, '0')}`
+
+const gen = (g: number, recordCount = 2): FoldGeneration => ({
+  generation: g,
+  timestamp: 1_700_000_000_000 + g,
+  delta: { generation: g, nouns: [UUID(g)], verbs: [], bytes: 123 + g },
+  records: Array.from({ length: recordCount }, (_, i) => ({
+    kind: (i % 2 === 0 ? 'noun' : 'verb') as 'noun' | 'verb',
+    id: UUID(g * 100 + i),
+    record: { metadata: { noun: 'document', v: g }, vector: { v: [g, i] } }
+  }))
+})
+
+describe('db/GenerationSegmentStore — the D1+D3 packed tier', () => {
+  let storage: MemoryStorage
+  let store: GenerationSegmentStore
+
+  beforeEach(async () => {
+    storage = new MemoryStorage()
+    await storage.init()
+    store = new GenerationSegmentStore(storage as any)
+    await store.open()
+  })
+
+  it('fold → read round-trips deltas and records via sidecar point-reads', async () => {
+    const meta = await store.fold([gen(1), gen(2), gen(3)])
+    expect(meta).toMatchObject({ firstGeneration: 1, lastGeneration: 3, frames: 3 })
+    expect(meta.checksum).toBeGreaterThan(0)
+
+    expect(store.hasGeneration(2)).toBe(true)
+    expect(store.hasGeneration(4)).toBe(false)
+
+    const d2 = await store.readDelta(2)
+    expect(d2?.delta).toEqual({ generation: 2, nouns: [UUID(2)], verbs: [], bytes: 125 })
+    expect(d2?.timestamp).toBe(1_700_000_000_002)
+
+    const records = await store.readRecords(3)
+    expect(records).toHaveLength(2)
+    expect(records![0]).toEqual({
+      kind: 'noun',
+      id: UUID(300),
+      record: { metadata: { noun: 'document', v: 3 }, vector: { v: [3, 0] } }
+    })
+    // Point read by id, both kinds.
+    expect(await store.readRecord(3, 'verb', UUID(301))).toEqual({
+      metadata: { noun: 'document', v: 3 },
+      vector: { v: [3, 1] }
+    })
+    expect(await store.readRecord(3, 'noun', UUID(999))).toBeNull()
+  })
+
+  it('reopen discovers everything from the manifest alone — no listing', async () => {
+    await store.fold([gen(1), gen(2)])
+    await store.fold([gen(3), gen(4)])
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(reopened.segments()).toHaveLength(2)
+    expect(reopened.hasGeneration(4)).toBe(true)
+    expect((await reopened.readDelta(1))?.timestamp).toBe(1_700_000_000_001)
+  })
+
+  it('a lost sidecar rebuilds from its segment; a damaged segment fails LOUDLY', async () => {
+    const meta = await store.fold([gen(1), gen(2)])
+    const idxPath = `${SEGMENTS_PREFIX}/seg-${String(1).padStart(20, '0')}.idx`
+    await storage.deleteRawObject(idxPath)
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    // Rebuild path: still serves correct data.
+    expect((await reopened.readRecords(2))!).toHaveLength(2)
+
+    // Now damage the SEGMENT itself: flip a payload byte → CRC mismatch, loud.
+    const segPath = `${SEGMENTS_PREFIX}/${meta.file}`
+    const bytes = (await storage.readRawBytes(segPath))!
+    bytes[bytes.length - 3] ^= 0xff
+    await storage.writeRawBytes(segPath, bytes)
+    const damaged = new GenerationSegmentStore(storage as any)
+    await damaged.open()
+    ;(damaged as any).sidecars.clear()
+    await storage.deleteRawObject(idxPath) // force the sequential rebuild over damaged bytes
+    await expect(damaged.readRecords(2)).rejects.toThrow(/CRC mismatch|damaged/)
+  })
+
+  it('D3 reclaim drops whole segments only and bumps compactedBelow', async () => {
+    await store.fold([gen(1), gen(2)])
+    await store.fold([gen(3), gen(4)])
+    await store.fold([gen(5), gen(6)])
+
+    // Horizon mid-segment-2 (below 4): only segment 1 is FULLY below → drops.
+    const r1 = await store.dropSegmentsBelow(4)
+    expect(r1).toEqual({ dropped: 1, compactedBelow: 3 })
+    expect(store.hasGeneration(1)).toBe(false)
+    expect(store.hasGeneration(3)).toBe(true) // partial segment survives whole
+
+    // Bytes actually gone.
+    expect(await storage.readRawBytes(`${SEGMENTS_PREFIX}/seg-${String(1).padStart(20, '0')}.bgs`)).toBeNull()
+
+    // Horizon past everything: the rest drop; compactedBelow is durable.
+    const r2 = await store.dropSegmentsBelow(7)
+    expect(r2.dropped).toBe(2)
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(reopened.compactedBelow()).toBe(7)
+    expect(reopened.segments()).toHaveLength(0)
+  })
+
+  it('the packed digest is deterministic across reopen and changes with history', async () => {
+    await store.fold([gen(1), gen(2), gen(3)])
+    const atSeal = await store.digestThroughPacked(3)
+    const midSegment = await store.digestThroughPacked(2)
+    expect(atSeal).not.toBeNull()
+    expect(midSegment).not.toBeNull()
+    expect(midSegment).not.toBe(atSeal)
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(await reopened.digestThroughPacked(3)).toBe(atSeal)
+    expect(await reopened.digestThroughPacked(2)).toBe(midSegment)
+
+    await reopened.fold([gen(4)])
+    expect(await reopened.digestThroughPacked(4)).not.toBe(atSeal)
+  })
+
+  it('sealed segments are immutable — fold refuses overlap, requires ascending input', async () => {
+    await store.fold([gen(1), gen(2)])
+    await expect(store.fold([gen(2), gen(3)])).rejects.toThrow(/overlaps the packed tier/)
+    await expect(store.fold([gen(4), gen(4)])).rejects.toThrow(/strictly ascending/)
+    await expect(store.fold([])).rejects.toThrow(/at least one generation/)
+  })
+})

tests/unit/db/generationStore.test.ts

@@ -249,7 +249,11 @@ describe('db/GenerationStore', () => {
       store.release(pinned)
       const result = await store.compact()
       expect(result.removedGenerations).toBeGreaterThan(0)
-      const remaining = await storage.listRawObjects(GENERATIONS_PREFIX)
+      // History record-sets only — the fact log (at `_generations/facts/`) is
+      // deliberately NOT reclaimed by history compaction.
+      const remaining = (await storage.listRawObjects(GENERATIONS_PREFIX)).filter(
+        (p: string) => !p.startsWith(`${GENERATIONS_PREFIX}/facts/`)
+      )
       expect(remaining).toEqual([])
     })
 
@@ -484,5 +488,95 @@ describe('db/GenerationStore', () => {
       expect(result.removedGenerations).toBe(2)
       store.release(2)
     })
+
+    it('timeBudgetMs bounds a pass; the next pass resumes the same prefix', async () => {
+      await manyGens(4)
+      // A spent budget (0ms) stops before reclaiming anything — an early stop
+      // is a consistent prefix, never a partial generation.
+      const bounded = await store.compact({ timeBudgetMs: 0 })
+      expect(bounded.removedGenerations).toBe(0)
+      expect(bounded.horizon).toBe(0)
+      // The next (unbounded) pass picks up exactly where the bounded one
+      // stopped and completes the same work.
+      const resumed = await store.compact()
+      expect(resumed.removedGenerations).toBe(4)
+      expect(resumed.horizon).toBe(4)
+    })
+  })
+
+  // ==========================================================================
+  describe('history-bytes running total (the O(1) retention check)', () => {
+    /** A fresh walk with the cache dropped — ground truth for the invariant. */
+    async function groundTruthBytes(): Promise<number> {
+      ;(store as any).historyBytesTotal = null
+      return store.historyBytes()
+    }
+
+    it('is seeded once, then maintained through commits WITHOUT re-walks', async () => {
+      await commitWrite(ID_A, 1)
+      await commitWrite(ID_A, 2)
+      const seeded = await store.historyBytes()
+      expect(seeded).toBe(await groundTruthBytes())
+
+      // From here every read must come from the running total, not a walk:
+      // getDelta re-reads are the walk's cost — commits must not trigger any.
+      const getDeltaSpy = vi.spyOn(store as any, 'getDelta')
+      await commitWrite(ID_B, 1)
+      const afterCommit = await store.historyBytes()
+      expect(getDeltaSpy).not.toHaveBeenCalled()
+      getDeltaSpy.mockRestore()
+      expect(afterCommit).toBe(await groundTruthBytes())
+    })
+
+    it('stays exact through single-op group commits and compaction', async () => {
+      await commitWrite(ID_A, 1)
+      await store.historyBytes() // seed
+      // Single-op path: buffered generations flushed as one group commit.
+      await store.commitSingleOp({
+        touched: { nouns: [ID_B] },
+        execute: async () => {
+          await storage.saveNounMetadata(ID_B, metadataFixture(1))
+        }
+      })
+      await store.flushPendingSingleOps()
+      expect(await store.historyBytes()).toBe(await groundTruthBytes())
+
+      await store.historyBytes() // re-seed after ground-truth reset
+      await store.compact({ maxGenerations: 1 })
+      expect(await store.historyBytes()).toBe(await groundTruthBytes())
+    })
+
+    it('historyStats reports counts, bytes, range, and horizon read-only', async () => {
+      await commitWrite(ID_A, 1)
+      await commitWrite(ID_B, 1)
+      const stats = await store.historyStats()
+      expect(stats.generations).toBe(2)
+      expect(stats.bytes).toBe(await store.historyBytes())
+      expect(stats.oldestGeneration).toBe(1)
+      expect(stats.newestGeneration).toBe(2)
+      expect(stats.oldestTimestamp).toBeLessThanOrEqual(stats.newestTimestamp!)
+      expect(stats.horizon).toBe(0)
+      // Read-only: nothing was reclaimed by asking.
+      expect(store.committedGeneration()).toBe(2)
+
+      await store.compact({ maxGenerations: 1 })
+      const after = await store.historyStats()
+      expect(after.generations).toBe(1)
+      expect(after.oldestGeneration).toBe(2)
+      expect(after.horizon).toBe(1)
+    })
+
+    it('empty history reports null range and zero bytes', async () => {
+      const stats = await store.historyStats()
+      expect(stats).toMatchObject({
+        generations: 0,
+        bytes: 0,
+        oldestGeneration: null,
+        newestGeneration: null,
+        oldestTimestamp: null,
+        newestTimestamp: null,
+        horizon: 0
+      })
+    })
   })
 })

tests/unit/db/pending-flush-durability.test.ts

@@ -0,0 +1,106 @@
+/**
+ * @module tests/unit/db/pending-flush-durability
+ * @description Finding 8: the async group-commit flush that persists single-op
+ * generation HISTORY must not swallow persist failures. Before the fix a failed
+ * background flush was a bare `prodLog.warn` — writes kept succeeding while their
+ * before-images silently piled up in memory, never durable and unbounded.
+ *
+ * The contract now:
+ *  - a single transient flush failure does NOT trip the alarm (tolerance), and
+ *  - after PENDING_FLUSH_FAILURE_THRESHOLD consecutive failures the store LATCHES
+ *    and REFUSES further writes with a typed PendingFlushDurabilityError rather
+ *    than accumulate un-durable history, and
+ *  - it self-heals: once a flush finally succeeds the latch lifts and writes resume.
+ * Live canonical data is never touched — only the immutable history is stuck.
+ *
+ * Fake timers keep the background retry/coalesce timers from firing, so the exact
+ * number of flush attempts (and thus the latch point) is deterministic — the test
+ * drives every flush explicitly.
+ */
+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+import { GenerationStore } from '../../../src/db/generationStore.js'
+import { PendingFlushDurabilityError } from '../../../src/db/errors.js'
+import { NounType } from '../../../src/types/graphTypes.js'
+
+const ID = (suffix: string): string => `00000000-0000-4000-8000-0000000000${suffix}`
+const meta = (v: number): Record<string, unknown> => ({
+  noun: NounType.Document,
+  subtype: 'note',
+  data: `payload-v${v}`,
+  version: v,
+  _rev: v
+})
+
+describe('db/GenerationStore pending-flush durability (finding 8)', () => {
+  let storage: MemoryStorage
+  let store: GenerationStore
+
+  beforeEach(async () => {
+    vi.useFakeTimers()
+    storage = new MemoryStorage()
+    await storage.init()
+    store = new GenerationStore(storage)
+    await store.open()
+    // High size threshold so single-ops never auto-flush — the test owns every flush.
+    ;(store as any).pendingFlushThreshold = 1000
+  })
+
+  afterEach(() => {
+    vi.useRealTimers()
+  })
+
+  /** One single-op write that buffers a pending generation. */
+  const singleOp = (id: string, v: number) =>
+    store.commitSingleOp({
+      touched: { nouns: [id], verbs: [] },
+      execute: async () => {
+        await storage.saveNounMetadata(id, meta(v))
+      }
+    })
+
+  it('latches after repeated flush failures, refuses writes, then self-heals', async () => {
+    // Buffer a pending generation while storage is healthy.
+    await singleOp(ID('aa'), 1)
+
+    // Fault the HISTORY flush (raw generation-record writes) — NOT the live
+    // canonical write, which uses a different path (saveNounMetadata).
+    const fault = Object.assign(new Error('EIO simulated'), { code: 'EIO' })
+    const spy = vi.spyOn(storage as any, 'writeRawObject').mockRejectedValue(fault)
+
+    // Each explicit flush fails and is accounted; the third trips the latch.
+    for (let i = 0; i < 3; i++) {
+      await expect(store.flushPendingSingleOps()).rejects.toThrow('EIO simulated')
+    }
+
+    // Writes are now refused LOUDLY rather than piling up un-durable history.
+    await expect(singleOp(ID('bb'), 1)).rejects.toBeInstanceOf(PendingFlushDurabilityError)
+    // The live counter did not advance for the refused write (no generation consumed).
+    expect(store.generation()).toBe(1)
+
+    // Storage recovers → an explicit flush drains the retained tier and lifts the latch.
+    spy.mockRestore()
+    await expect(store.flushPendingSingleOps()).resolves.toBeUndefined()
+
+    // Writes resume; the next single-op commits normally.
+    const receipt = await singleOp(ID('cc'), 1)
+    expect(receipt.generation).toBe(2)
+  })
+
+  it('a single transient flush failure does NOT latch — writes continue', async () => {
+    await singleOp(ID('aa'), 1)
+
+    const spy = vi.spyOn(storage as any, 'writeRawObject').mockRejectedValue(new Error('blip'))
+    await expect(store.flushPendingSingleOps()).rejects.toThrow('blip') // failure #1, below threshold
+    spy.mockRestore()
+
+    // Below the latch threshold → the store still accepts writes.
+    const receipt = await singleOp(ID('bb'), 1)
+    expect(receipt.generation).toBe(2)
+
+    // A subsequent healthy flush drains the tier and resets the failure counter.
+    await expect(store.flushPendingSingleOps()).resolves.toBeUndefined()
+    expect((store as any).pendingFlushFailures).toBe(0)
+    expect((store as any).pendingFlushError).toBeNull()
+  })
+})

tests/unit/get-index-status-readiness.test.ts

@@ -0,0 +1,59 @@
+/**
+ * @module tests/unit/get-index-status-readiness
+ * @description Pattern-A / Finding 9: getIndexStatus reported `populated: size>0`
+ * and only `migrating`, so a native index that loaded its count but not its
+ * serving structure reported populated:true — a k8s readiness probe would 200 a
+ * brain that serves []. It now folds in the honest isReady() signal + the
+ * _indexRebuildFailed / _indexDegradedIds degraded states (mirroring
+ * validateIndexConsistency / checkHealth).
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { Brainy, NounType } from '../../src/index.js'
+
+describe('getIndexStatus honest readiness (Finding 9)', () => {
+  let brain: any
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, dimensions: 384 })
+    await brain.init()
+    await brain.add({ data: 'x', type: NounType.Concept })
+    await brain.flush()
+  })
+
+  it('a not-ready provider makes populated honest (false) and exposes ready:false', async () => {
+    brain.index.isReady = () => false // count present, serving structure NOT loaded
+    const status = await brain.getIndexStatus()
+    expect(status.hnswIndex.populated).toBe(false)
+    expect(status.hnswIndex.ready).toBe(false)
+    delete brain.index.isReady
+  })
+
+  it('a ready provider reports populated:true + ready:true', async () => {
+    brain.index.isReady = () => true
+    const status = await brain.getIndexStatus()
+    expect(status.hnswIndex.populated).toBe(true)
+    expect(status.hnswIndex.ready).toBe(true)
+    delete brain.index.isReady
+  })
+
+  it('rebuildFailed / rebuildError surface the init-degraded state', async () => {
+    brain._indexRebuildFailed = new Error('boom')
+    const status = await brain.getIndexStatus()
+    expect(status.rebuildFailed).toBe(true)
+    expect(status.rebuildError).toBe('boom')
+    brain._indexRebuildFailed = null
+  })
+
+  it('degradedIds surfaces the adopt-forward degraded set', async () => {
+    brain._indexDegradedIds.add('00000000-0000-4000-8000-0000000000de')
+    const status = await brain.getIndexStatus()
+    expect(status.degradedIds).toBe(1)
+    brain._indexDegradedIds.clear()
+  })
+
+  it('a JS-baseline provider (no isReady) omits ready and falls back to size>0', async () => {
+    const status = await brain.getIndexStatus()
+    expect(status.hnswIndex.ready).toBeUndefined()
+    expect(status.hnswIndex.populated).toBe(brain.index.size() > 0)
+  })
+})

tests/unit/graph/graph-fastpath-honest-readiness.test.ts

@@ -0,0 +1,95 @@
+/**
+ * @module tests/unit/graph/graph-fastpath-honest-readiness
+ * @description Pattern-A / Finding 2: getVerbsBySource/ByTarget gated the graph
+ * fast path on `graphIndex.isInitialized` (a proxy that reads true once the
+ * manifest/count loaded even if the source→target adjacency did NOT → the fast
+ * path then returned a silent []). The honest gate skips the fast path when the
+ * provider reports isReady()===false, falling to the correct canonical shard
+ * scan; and a one-shot probe self-heals a no-isReady provider whose adjacency
+ * did not cold-load.
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { Brainy, NounType, VerbType } from '../../../src/index.js'
+
+describe('graph fast-path honest readiness (Finding 2)', () => {
+  let brain: any
+  let storage: any
+  let a: string
+  let b: string
+  let c: string
+
+  beforeEach(async () => {
+    process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
+    brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, dimensions: 384 })
+    await brain.init()
+    a = await brain.add({ data: 'a', type: NounType.Concept })
+    b = await brain.add({ data: 'b', type: NounType.Concept })
+    c = await brain.add({ data: 'c', type: NounType.Concept })
+    await brain.relate({ from: a, to: b, type: VerbType.Contains })
+    await brain.relate({ from: a, to: c, type: VerbType.Contains })
+    await brain.flush()
+    storage = brain.storage
+    // Warm + wire the storage graph index (lazy) so we can stub it.
+    await storage.getVerbsBySource(a)
+  })
+
+  it('not-ready provider → shard scan returns the REAL edges, not a silent []', async () => {
+    const gi = storage.graphIndex
+    // Simulate a cold native provider: count/manifest loaded (isInitialized) but
+    // the source→target adjacency is NOT (isReady false; fast-path lookup empty).
+    gi.isReady = () => false
+    const origBySource = gi.getVerbIdsBySource.bind(gi)
+    gi.getVerbIdsBySource = async () => [] // cold adjacency — old fast path trusted this
+    storage._graphFastPathProbed = true // isolate the GATE (probe skips isReady-capable anyway)
+    try {
+      const verbs = await storage.getVerbsBySource(a)
+      // Honest gate skipped the cold fast path → canonical shard scan → real edges.
+      expect(verbs.length).toBe(2)
+    } finally {
+      delete gi.isReady
+      gi.getVerbIdsBySource = origBySource
+    }
+  })
+
+  it('ready provider → fast path is used and an empty result is trusted', async () => {
+    const gi = storage.graphIndex
+    gi.isReady = () => true
+    let fastPathCalls = 0
+    const origBySource = gi.getVerbIdsBySource.bind(gi)
+    gi.getVerbIdsBySource = async (...args: any[]) => { fastPathCalls++; return origBySource(...args) }
+    storage._graphFastPathProbed = true
+    try {
+      // `c` has no OUTGOING edges — a genuinely empty result via the fast path.
+      const verbs = await storage.getVerbsBySource(c)
+      expect(verbs).toEqual([])
+      expect(fastPathCalls).toBeGreaterThan(0) // fast path was taken, not a shard scan
+    } finally {
+      delete gi.isReady
+      gi.getVerbIdsBySource = origBySource
+    }
+  })
+
+  it('no-isReady provider whose adjacency did not cold-load → probe rebuilds once', async () => {
+    const gi = storage.graphIndex
+    if ('isReady' in gi) delete gi.isReady // force the "unknown" (no honest signal) path
+    // Force the probe to see a cold adjacency: a known edge resolves to no ints.
+    const origBySource = gi.getVerbIdsBySource.bind(gi)
+    let cold = true
+    gi.getVerbIdsBySource = async (...args: any[]) => (cold ? [] : origBySource(...args))
+    let rebuilds = 0
+    const origRebuild = gi.rebuild.bind(gi)
+    gi.rebuild = async (...args: any[]) => { rebuilds++; cold = false; return origRebuild(...args) }
+    storage._graphFastPathProbed = false // re-arm the one-shot probe
+    try {
+      await storage.getVerbsBySource(a)
+      expect(rebuilds).toBe(1)                    // probe detected cold adjacency + rebuilt once
+      expect(storage._graphFastPathProbed).toBe(true) // latched — no second rebuild
+      rebuilds = 0
+      await storage.getVerbsByTarget(b)
+      expect(rebuilds).toBe(0)                    // probe does not re-run
+    } finally {
+      gi.getVerbIdsBySource = origBySource
+      gi.rebuild = origRebuild
+    }
+  })
+})

tests/unit/graph/lsm-partial-load-failclosed.test.ts

@@ -0,0 +1,76 @@
+/**
+ * @module tests/unit/graph/lsm-partial-load-failclosed
+ * @description LSMTree partial-load fail-closed guard (Finding 3 of the honest
+ * index-readiness spine plan). `loadSSTables()` used to swallow a per-SSTable
+ * load failure with only a `prodLog.warn`, then `loadManifest()` still published
+ * the FULL persisted `totalRelationships` count — so on a partial cold load,
+ * `size()`/`isHealthy()` reported the full count and "healthy" while a subset of
+ * the tree's edges were silently unqueryable. Fixed by failing the whole load
+ * closed on ANY per-SSTable failure: `loadManifest()`'s existing catch then
+ * resets `sstables`/`totalRelationships`/`sstablesByLevel` to honest-empty, so
+ * `size()` reports 0 and the graph layer's existing `size()===0` self-heal
+ * rebuilds from canonical records — the dishonest partial state can no longer be
+ * observed.
+ */
+import { describe, it, expect } from 'vitest'
+import { LSMTree } from '../../../src/graph/lsm/LSMTree.js'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+
+describe('LSMTree partial SSTable load fails closed (honest size())', () => {
+  it('one SSTable fails to load → size() reports 0, not the manifest count', async () => {
+    const storage = new MemoryStorage()
+    await storage.init()
+    const prefix = 'test-lsm-partial'
+
+    const t1 = new LSMTree(storage, { storagePrefix: prefix, enableCompaction: false })
+    await t1.init()
+    await t1.add('a', 'b'); await t1.flush()   // SSTable #1
+    await t1.add('c', 'd'); await t1.flush()   // SSTable #2
+    expect(t1.size()).toBeGreaterThanOrEqual(2)
+    await t1.close()
+
+    // Find an SSTable id from the persisted manifest.
+    const manifest = await storage.getMetadata(`${prefix}-manifest`)
+    const sstableIds = Object.keys((manifest!.data as any).sstables)
+    expect(sstableIds.length).toBeGreaterThanOrEqual(2)
+
+    // Reopen with ONE SSTable load forced to fail (deterministic).
+    const t2 = new LSMTree(storage, { storagePrefix: prefix, enableCompaction: false })
+    const realGet = storage.getMetadata.bind(storage)
+    ;(storage as any).getMetadata = async (key: string) => {
+      if (key === `${prefix}-${sstableIds[1]}`) throw new Error('simulated SSTable load failure')
+      return realGet(key)
+    }
+    await t2.init()
+
+    // BEFORE THE FIX: t2.size() === (full manifest count)  ← LIE (partial load, full count)
+    // AFTER THE FIX:  t2.size() === 0                       ← fail-closed → self-heal rebuilds
+    expect(t2.size()).toBe(0)
+    expect(t2.isHealthy()).toBe(true) // honestly-empty-but-initialized, not "healthy with a lie"
+
+    await t2.close()
+  })
+
+  it('a clean load with no failures still reports the full honest count', async () => {
+    const storage = new MemoryStorage()
+    await storage.init()
+    const prefix = 'test-lsm-clean'
+
+    const t1 = new LSMTree(storage, { storagePrefix: prefix, enableCompaction: false })
+    await t1.init()
+    await t1.add('a', 'b'); await t1.flush()
+    await t1.add('c', 'd'); await t1.flush()
+    const expectedSize = t1.size()
+    expect(expectedSize).toBeGreaterThanOrEqual(2)
+    await t1.close()
+
+    // Reopen with no injected failures — a normal, fully-successful cold load.
+    const t2 = new LSMTree(storage, { storagePrefix: prefix, enableCompaction: false })
+    await t2.init()
+
+    expect(t2.size()).toBe(expectedSize)
+    expect(t2.isHealthy()).toBe(true)
+
+    await t2.close()
+  })
+})

tests/unit/hnsw/flush-failure.test.ts

@@ -0,0 +1,157 @@
+/**
+ * HNSW deferred-flush durability tests (Finding 6 — spine-plan Part B,
+ * "blind catch" audit).
+ *
+ * `JsHnswVectorIndex.flush()` used to swallow a per-node
+ * `persistNodeConnections` failure (and a system-record `saveHNSWSystem`
+ * failure) behind `console.error`, then unconditionally clear the dirty set
+ * and report the pre-flush node count as "success". A transient write fault
+ * therefore silently dropped that node's connections from durable storage
+ * forever, with `flush()` having lied about it. These tests pin the fix:
+ * a node (or the system record) that fails to persist stays in the
+ * dirty/retry set, and `flush()` throws {@link HnswFlushError} instead of
+ * returning a count. The immediate-mode first-noun `saveHNSWSystem` swallow
+ * (while `addItem()` still returned the id) is covered too.
+ */
+
+import { describe, it, expect, vi } from 'vitest'
+import { v4 as uuidv4 } from 'uuid'
+import { JsHnswVectorIndex, HnswFlushError } from '../../../src/hnsw/hnswIndex.js'
+import { euclideanDistance } from '../../../src/utils/index.js'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+
+// Helper: generate a random vector of given dimension (mirrors lazy-vectors.test.ts).
+function randomVector(dim: number): number[] {
+  return Array.from({ length: dim }, () => Math.random() * 2 - 1)
+}
+
+describe('HNSW deferred-flush durability (Finding 6)', () => {
+  const dim = 8
+
+  it('a clean flush persists every dirty node and clears the dirty set', async () => {
+    const storage = new MemoryStorage()
+    const index = new JsHnswVectorIndex(
+      { M: 4, efConstruction: 50, efSearch: 20 },
+      euclideanDistance,
+      { useParallelization: false, storage, persistMode: 'deferred' }
+    )
+
+    for (let i = 0; i < 5; i++) {
+      await index.addItem({ id: uuidv4(), vector: randomVector(dim) })
+    }
+
+    // Sanity: inserting several items in deferred mode does dirty something.
+    expect((index as any).dirtyNodes.size).toBeGreaterThan(0)
+
+    const flushed = await index.flush()
+    expect(typeof flushed).toBe('number')
+    expect((index as any).dirtyNodes.size).toBe(0)
+    expect((index as any).dirtySystem).toBe(false)
+  })
+
+  it('retains a node whose connections failed to persist and surfaces HnswFlushError instead of reporting success', async () => {
+    const storage = new MemoryStorage()
+    const index = new JsHnswVectorIndex(
+      { M: 4, efConstruction: 50, efSearch: 20 },
+      euclideanDistance,
+      { useParallelization: false, storage, persistMode: 'deferred' }
+    )
+
+    const ids: string[] = []
+    for (let i = 0; i < 5; i++) {
+      const id = uuidv4()
+      ids.push(id)
+      await index.addItem({ id, vector: randomVector(dim) })
+    }
+
+    // The very first node is guaranteed to become a neighbor of the second
+    // insert (it is the only existing node in the graph at that point), so it
+    // is deterministically present in the dirty set before any fault.
+    const failId = ids[0]
+    const dirtyBefore = (index as any).dirtyNodes as Set<string>
+    expect(dirtyBefore.has(failId)).toBe(true)
+
+    const originalSave = storage.saveVectorIndexData.bind(storage)
+    const spy = vi
+      .spyOn(storage, 'saveVectorIndexData')
+      .mockImplementation(async (nounId, hnswData) => {
+        if (nounId === failId) {
+          throw Object.assign(new Error('simulated write fault'), { code: 'EIO' })
+        }
+        return originalSave(nounId, hnswData)
+      })
+
+    await expect(index.flush()).rejects.toBeInstanceOf(HnswFlushError)
+
+    const dirtyAfter = (index as any).dirtyNodes as Set<string>
+    // The failed node stays dirty for the next retry ...
+    expect(dirtyAfter.has(failId)).toBe(true)
+    // ... and every node that DID persist successfully leaves the dirty set —
+    // the failure of one node must not re-dirty (or fail to clear) the rest.
+    expect(dirtyAfter.size).toBe(1)
+
+    // Recovery: once the fault clears, the retained node persists and the
+    // flush reports success again (the intended retry path).
+    spy.mockRestore()
+    const flushed = await index.flush()
+    expect(typeof flushed).toBe('number')
+    expect((index as any).dirtyNodes.size).toBe(0)
+  })
+
+  it('surfaces a system-record persist failure as HnswFlushError and keeps dirtySystem set for retry', async () => {
+    const storage = new MemoryStorage()
+    const index = new JsHnswVectorIndex(
+      { M: 4, efConstruction: 50, efSearch: 20 },
+      euclideanDistance,
+      { useParallelization: false, storage, persistMode: 'deferred' }
+    )
+
+    // First noun in deferred mode marks dirtySystem (entryPoint/maxLevel).
+    await index.addItem({ id: uuidv4(), vector: randomVector(dim) })
+    expect((index as any).dirtySystem).toBe(true)
+
+    const spy = vi
+      .spyOn(storage, 'saveHNSWSystem')
+      .mockRejectedValue(
+        Object.assign(new Error('simulated system write fault'), { code: 'EIO' })
+      )
+
+    let caught: unknown
+    try {
+      await index.flush()
+    } catch (error) {
+      caught = error
+    }
+
+    expect(caught).toBeInstanceOf(HnswFlushError)
+    expect((caught as HnswFlushError).systemFailed).toBe(true)
+    // The system record must stay dirty — a lost entry point/maxLevel update
+    // must be retried, not silently dropped.
+    expect((index as any).dirtySystem).toBe(true)
+
+    spy.mockRestore()
+    const flushed = await index.flush()
+    expect(typeof flushed).toBe('number')
+    expect((index as any).dirtySystem).toBe(false)
+  })
+
+  it('surfaces the immediate-mode first-noun system persist failure via a rejecting addItem()', async () => {
+    const storage = new MemoryStorage()
+    const index = new JsHnswVectorIndex(
+      { M: 4, efConstruction: 50, efSearch: 20 },
+      euclideanDistance,
+      { useParallelization: false, storage } // default persistMode: 'immediate'
+    )
+
+    vi.spyOn(storage, 'saveHNSWSystem').mockRejectedValue(
+      Object.assign(new Error('simulated system write fault'), { code: 'EIO' })
+    )
+
+    // Previously this swallowed the error (console.error) and addItem()
+    // still resolved with the id — stranding a brand-new index whose root
+    // (entry point) was never actually persisted. Now it must reject.
+    await expect(
+      index.addItem({ id: uuidv4(), vector: randomVector(dim) })
+    ).rejects.toThrow('simulated system write fault')
+  })
+})