feat: two-tier history reads + the repacker + generationDigest — D1+D3 wired end-to-end

The packed tier goes live inside GenerationStore:

- Two-tier reads: getDelta / readBeforeImage / readGenerationRecords
  fall through live-tier → sealed segments (live-tier-wins: a crash
  mid-fold leaves a duplicate representation, never a gap). Cold-open
  seeds committedRanges from the segment manifest via interval merge —
  packed generations resolve without their directories existing.
- repackHistory({timeBudgetMs, batchGenerations}): folds cold
  generations (older than the newest 1024) oldest-first into sealed
  segments, deleting per-generation directories only after segment +
  manifest are durable. Public API + automatic time-bounded pass at
  close() (before compaction, so reclaim can drop whole segments);
  re-representation only — the sole history transform under the
  archival profile. Early stop = consistent prefix, next pass resumes.
- compact(): packed generations reclaim logically in the loop and
  physically at whole-segment boundaries via dropSegmentsBelow (the
  frozen partial-segments-wait rule).
- generationDigest(g) (D8): deterministic content digest through g —
  sealed-segment checksum chain + live-tier delta hashes; O(segments +
  live window); RangeError out of range, GenerationCompactedError
  below the horizon (a gate can never silently pin reclaimed history).

Four end-to-end pins: asOf answers byte-identical across fold + cold
reopen with folded dirs physically gone; repack+reclaim composition;
digest reopen-stability/divergence/loud-horizon; budget no-op+resume.

.github/workflows/ci.yml

@@ -34,5 +34,7 @@ jobs:
         with:
           bun-version: latest
       - run: npm ci
+      # test:bun imports the built dist/, so build first.
+      - run: npm run build
       # Bun as a runtime is the supported Bun story (`bun add` / `bun run`).
       - run: npm run test:bun

.gitignore

@@ -93,9 +93,14 @@ docs/internal/
 # Rust/Cargo build artifacts
 src/embeddings/candle-wasm/target/
 src/embeddings/candle-wasm/Cargo.lock
-src/embeddings/wasm/pkg/
 
-# But keep the pre-built WASM (committed for users without Rust)
+# Ignore the wasm-pack output dir's CONTENTS (note the `/*`, not `/`, so the
+# re-includes below can take effect — git cannot re-include a file whose parent
+# DIRECTORY is excluded). Keep the pre-built WASM committed: it ships in the npm
+# package anyway, it lets consumers + CI build without a Rust/wasm-pack toolchain,
+# and versioning it makes the shipped artifact reproducible (not "whatever the
+# maintainer last built").
+src/embeddings/wasm/pkg/*
 !src/embeddings/wasm/pkg/*.wasm
 !src/embeddings/wasm/pkg/*.js
 !src/embeddings/wasm/pkg/*.d.ts

CHANGELOG.md

@@ -2,6 +2,274 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19)
+
+- docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78)
+- fix: release drains in-flight writer-lock heartbeat — no phantom lock after unlink (70e4bc8)
+- feat: flush() never compacts — history maintenance moves to close() with bounded passes (300d9f2)
+
+
+### [8.8.2](https://github.com/soulcraftlabs/brainy/compare/v8.8.1...v8.8.2) (2026-07-19)
+
+- fix: one field-resolution law across aggregation hooks, source.where, removeMany, and find() spellings (945d92d)
+- chore: push public docs to the soulcraft.com ingest door on release (42037d0)
+
+
+### [8.8.1](https://github.com/soulcraftlabs/brainy/compare/v8.8.0...v8.8.1) (2026-07-18)
+
+- fix: O(1) adaptive retention accounting + historyStats fleet audit (6207e48)
+- fix: import dedup off-switch honesty + brain-owned lifecycle for the background pass (4fcef7b)
+
+
+### [8.8.0](https://github.com/soulcraftlabs/brainy/compare/v8.7.1...v8.8.0) (2026-07-17)
+
+- feat: OS-limit detection for pool-scale deployments (16a73b8)
+
+
+### [8.7.1](https://github.com/soulcraftlabs/brainy/compare/v8.7.0...v8.7.1) (2026-07-17)
+
+- fix: race-proof writer-lock acquisition + machine-readable conflict through init (01a3b46)
+
+
+### [8.7.0](https://github.com/soulcraftlabs/brainy/compare/v8.6.0...v8.7.0) (2026-07-17)
+
+- feat: scaled transact budgets + labeled timeout diagnostics + envelope docs (6ef9fcb)
+
+
+### [8.6.0](https://github.com/soulcraftlabs/brainy/compare/v8.5.2...v8.6.0) (2026-07-17)
+
+- feat: brain.auditGraph() — read-only graph-truth audit (2a03fae)
+
+
+### [8.5.2](https://github.com/soulcraftlabs/brainy/compare/v8.5.1...v8.5.2) (2026-07-17)
+
+- fix: exception-safe aggregation backfill + generation-verified adoption + loud open-path guards (a77b064)
+
+
+### [8.5.1](https://github.com/soulcraftlabs/brainy/compare/v8.5.0...v8.5.1) (2026-07-17)
+
+- fix: aggregation state adoption on reopen + single-flight backfill + query-cap ratchet removal (da55be7)
+- docs: external-backups/sparse-storage guide + generation fact log concept (593bb8b)
+
+
+### [8.5.0](https://github.com/soulcraftlabs/brainy/compare/v8.4.0...v8.5.0) (2026-07-15)
+
+- test: tolerant timing assertion in the execution-time measure test (4dc0a92)
+- feat: committedGeneration capability + pinned durability/stability contracts (d1ecee1)
+- docs: RELEASES.md entry for 8.5.0 (provider fact-log access + shared verifier) (e4f37cd)
+- feat: provider access to the fact log + shared stamp verifier via internals (352e356)
+
+
+### [8.4.0](https://github.com/soulcraftlabs/brainy/compare/v8.3.3...v8.4.0) (2026-07-15)
+
+- docs: RELEASES.md entry for 8.4.0 (generation fact log + family stamp) (4a60b43)
+- feat: entity-tree family stamp — sourceGeneration + rollup coherence at open (2888ae6)
+- feat: generation fact log — after-image commit records, dual-written at every commit point (38b0041)
+
+
+### [8.3.3](https://github.com/soulcraftlabs/brainy/compare/v8.3.2...v8.3.3) (2026-07-15)
+
+- docs: RELEASES.md entry for 8.3.3 (rename containment fix + repair) (c3feafd)
+- test: lens-consistency regression — combined vs subtype-only vs canonical ground truth (4fb41f9)
+- fix: VFS rename moves the containment edge — no ghost in the old directory (af8c179)
+
+
+### [8.3.2](https://github.com/soulcraftlabs/brainy/compare/v8.3.1...v8.3.2) (2026-07-14)
+
+- docs: RELEASES.md entry for 8.3.2 (honest counters) (0932ecd)
+- fix: honest counters — removal never re-reads the removed record + repairIndex recounts and persists all rollups (2e2ba9c)
+
+
+### [8.3.1](https://github.com/soulcraftlabs/brainy/compare/v8.3.0...v8.3.1) (2026-07-14)
+
+- docs: RELEASES.md entry for 8.3.1 (full-removal deletes + family-scoped gate) (c0c68ac)
+- fix: full-removal canonical deletes + family-scoped migration gate (366f9a9)
+- docs: cite the cross-layer integrity contract generically in comments and notes (1d26988)
+
+
+### [8.3.0](https://github.com/soulcraftlabs/brainy/compare/v8.2.8...v8.3.0) (2026-07-13)
+
+- docs: RELEASES.md entry for 8.3.0 (heal-cost + cross-layer integrity contract) (7692c6f)
+- perf: parallel + id-only canonical enumeration (heal-cost dominant term) (ec5b933)
+- feat: registered-blob family contract — declared index blobs are undeletable (bfa1762)
+- feat: validateIndexConsistency delegates to provider invariants (6bcb54f)
+
+
+### [8.2.8](https://github.com/soulcraftlabs/brainy/compare/v8.2.7...v8.2.8) (2026-07-13)
+
+- fix: honest index readiness — no silently-empty queries on a cold index (d0f69c7)
+
+
+### [8.2.7](https://github.com/soulcraftlabs/brainy/compare/v8.2.6...v8.2.7) (2026-07-13)
+
+- fix: restore loadBinaryBlob fault-propagation (native column-store lockstep) (b6c7039)
+
+
+### [8.2.6](https://github.com/soulcraftlabs/brainy/compare/v8.2.5...v8.2.6) (2026-07-13)
+
+- docs: RELEASES.md entry for 8.2.6 (write/index-spine hardening) (a873852)
+- chore: hold loadBinaryBlob fault-propagation for the cortex column-store lockstep (36c10c1)
+- fix: aggregation surfaces materialize/state-load failures loudly (02eff64)
+- fix: surface a degraded derived index on reads instead of serving it silently (ba958d9)
+- fix: saveBinaryBlob never acks a durable write that stored nothing (7feba49)
+- fix: refuse writes when single-op history cannot be made durable (54c1836)
+- fix: clear() wipes the full native/derived footprint, not a subset (d8301f8)
+- fix: surface segment/entity read faults loudly instead of masking as absent (af5d2f3)
+- fix: spine hardening pass 1 (part) — count symmetry, honest partial-load, flush durability, read-fault propagation (119087a)
+- test: pin the read-your-writes contract under the single writer (eb9c4eb)
+
+
+### [8.2.5](https://github.com/soulcraftlabs/brainy/compare/v8.2.4...v8.2.5) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.5 (honest rollback-failure response) (a7c7aa5)
+- fix: honest response when a transaction rollback cannot complete (711d2f0)
+
+
+### [8.2.4](https://github.com/soulcraftlabs/brainy/compare/v8.2.3...v8.2.4) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.4 (non-destructive restore) (4574695)
+- fix: non-destructive, crash-resumable restore (a2f4f6a)
+
+
+### [8.2.3](https://github.com/soulcraftlabs/brainy/compare/v8.2.2...v8.2.3) (2026-07-12)
+
+- docs: RELEASES.md entry for 8.2.3 (transact durability barrier) (be5ce0b)
+- fix: transact durability barrier — committed transactions are durable on return (3b8fa51)
+
+
+### [8.2.2](https://github.com/soulcraftlabs/brainy/compare/v8.2.1...v8.2.2) (2026-07-11)
+
+- docs: RELEASES.md entry for 8.2.2 (transaction timeout rollback) (ed97006)
+- fix: transaction timeout rolls back applied operations (no torn state) (508a8e3)
+
+
+### [8.2.1](https://github.com/soulcraftlabs/brainy/compare/v8.2.0...v8.2.1) (2026-07-10)
+
+- test: update graph-index operation constructors to the VerbEndpointInts signature (62a449d)
+- docs: RELEASES.md entry for 8.2.1 (transact forward-ref parity fix) (7089782)
+- fix: transact forward references resolve graph endpoint ints at execute time (a175406)
+
+
+### [8.2.0](https://github.com/soulcraftlabs/brainy/compare/v8.1.0...v8.2.0) (2026-07-10)
+
+- docs: RELEASES.md entry for 8.2.0 (temporal VFS) (98ceadc)
+- feat: temporal VFS — file content joins the Model-B immutability model (a3467e1)
+- docs: pin the write-path invariant in the plugin contract (the onChange change-feed guarantee) (4af8fb3)
+
+
+### [8.1.0](https://github.com/soulcraftlabs/brainy/compare/v8.0.17...v8.1.0) (2026-07-10)
+
+- docs: RELEASES.md entry for 8.1.0 (brain.onChange change feed) (4e9be08)
+- feat: brain.onChange — the in-process change feed for every committed mutation (fd5edb5)
+
+
+### [8.0.17](https://github.com/soulcraftlabs/brainy/compare/v8.0.16...v8.0.17) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.17 (canonical count recovery + dead-machinery sweep) (6b8b9cb)
+- fix: count recovery scans the canonical layout; remove the dead 7.x hnsw sharding machinery (352e2da)
+
+
+### [8.0.16](https://github.com/soulcraftlabs/brainy/compare/v8.0.15...v8.0.16) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.16 (atomic ifAbsent/upsert + exact blob refCounts) (54e7c0e)
+- fix: atomic ifAbsent/upsert inserts + exact blob reference counts under concurrency (867939e)
+
+
+### [8.0.15](https://github.com/soulcraftlabs/brainy/compare/v8.0.14...v8.0.15) (2026-07-08)
+
+- docs: RELEASES.md entry for 8.0.15 (atomic ifRev CAS) (b1fe25a)
+- fix: ifRev CAS is atomic — the revision check now runs under the commit mutex (9a3d1bd)
+
+
+### [8.0.14](https://github.com/soulcraftlabs/brainy/compare/v8.0.13...v8.0.14) (2026-07-07)
+
+- docs: RELEASES.md entry for 8.0.14 (migration preserves branch-scoped non-entity state) (64188a3)
+- fix: 7→8 migration preserves branch-scoped non-entity state instead of deleting it (a93bb4e)
+
+
+### [8.0.13](https://github.com/soulcraftlabs/brainy/compare/v8.0.12...v8.0.13) (2026-07-07)
+
+- docs: RELEASES.md entry for 8.0.13 (accurate boot log for established stores) (38e8de5)
+- fix: an established store no longer boot-logs "New installation" (3086916)
+
+
+### [8.0.12](https://github.com/soulcraftlabs/brainy/compare/v8.0.11...v8.0.12) (2026-07-07)
+
+- docs: RELEASES.md entry for 8.0.12 (7→8 VFS recovery, zero-rebuild cold open, strict query operators) (d9017e7)
+- fix: recover VFS content blobs stranded by a 7→8 upgrade, in place on open (c0f6ccd)
+- fix: validate where-operators and align the in-memory matcher to the documented set (6821e19)
+- docs: correct rc-era time-travel staleness + record the embedding-model ordering constraint (68da660)
+- fix: cold-open no longer re-derives durable indexes — complete the readiness contract for all three providers (61c247c)
+- docs: RELEASES.md entry for 8.0.11 (exit-hang class closed for every op shape) (4fde94b)
+
+
+### [8.0.11](https://github.com/soulcraftlabs/brainy/compare/v8.0.10...v8.0.11) (2026-07-02)
+
+- fix: no script shape can hang on brainy's internals — unref every maintenance timer + one-shot beforeExit (30eacbd)
+- docs: RELEASES.md entry for 8.0.10 (clean process exit after close) (2da2736)
+
+
+### [8.0.10](https://github.com/soulcraftlabs/brainy/compare/v8.0.9...v8.0.10) (2026-07-02)
+
+- fix: a bare script now exits cleanly after close() — release every process keep-alive (c540d63)
+
+
+### [8.0.9](https://github.com/soulcraftlabs/brainy/compare/v8.0.8...v8.0.9) (2026-07-02)
+
+- feat: guarded plugin auto-detection — installing @soulcraft/cor is the opt-in (588267b)
+
+
+### [8.0.8](https://github.com/soulcraftlabs/brainy/compare/v8.0.7...v8.0.8) (2026-07-02)
+
+- docs: plugins are explicit opt-in — correct the README scale section and plugins config comment (e420369)
+
+
+### [8.0.7](https://github.com/soulcraftlabs/brainy/compare/v8.0.1...v8.0.7) (2026-07-02)
+
+- docs: GA version is 8.0.7 — npm retired 8.0.0-8.0.6 (January dev-cycle unpublishes) (5db2c41)
+- chore(release): 8.0.1 (48bea9e)
+- docs: flagship README for the 8.0 GA; GA version is 8.0.1 (e44620e)
+- docs: rename the native provider to @soulcraft/cor across public docs and JSDoc (bf4a333)
+- chore(release): 8.0.0 (a3c2717)
+- docs: RELEASES.md 8.0.0 GA entry (RC notes become history) (4584d0b)
+- feat: promote the 8.0 u64-id line to main for the 8.0.0 GA (55d57f8)
+- fix(8.0): byte-copy _id_mapper/* in the pre-upgrade backup (cor's write-new nuance) (a30ed72)
+- chore(release): 7.33.5 (29b9d5f)
+- fix: metadata cold-read guard — no more silent [] on cold find({where}) (7.33.5) (9dc4c5e)
+- fix(8.0): metadata cold-read guard — no more silent [] on cold find({where}) (79e8709)
+- docs(8.0): add module JSDoc to typeValidation.ts (the one file missing a module block) (ab53fa0)
+- feat(8.0): auto pre-upgrade backup — hard-link snapshot before the 7.x→8.0 migration (1aad1f6)
+- fix(ci): commit the prebuilt wasm pkg + build before test:bun (green CI on fresh clone) (ed178e2)
+- chore(release): 7.33.4 (2be3d0f)
+- fix: never serve a silent [] from find({connected}) on a cold-loaded graph (fd699d0)
+- chore(release): 7.33.3 (d1665bb)
+- fix: re-validate find() results against the predicate (index-integrity guard) (7b5db0d)
+- chore(release): 7.33.2 (9593a27)
+- fix: graph adjacency cold-load consistency guard — no more silent [] on connected (1694f68)
+- chore(release): 7.33.1 (811c7da)
+- fix: getNouns cursor pagination re-scanned the first page forever (permanent CPU loop) (6721c52)
+- chore(release): 7.33.0 (526aaad)
+- feat: visibility tier (public/internal/system) on nouns + verbs (3a62445)
+- chore(release): 7.32.2 (c53dd61)
+- refactor: rename BackupData → PortableGraph (the type is interchange, not a backup) (89036de)
+- chore(release): 7.32.1 (5e7379d)
+- fix: getNouns().totalCount reports true total, not page size; quiet benign mmap-vector log (edff637)
+- chore(release): 7.32.0 (adec0ba)
+- feat: portable graph export()/import() (BackupData v1) on brain.data() (a408d37)
+- chore(release): 7.31.8 (89c6d04)
+- fix: query-cap memory misread (MemAvailable + floor) + rootDirectory getter for native mmap fast-path (3f8e097)
+- chore(release): 7.31.7 (4f8159c)
+- fix: vfs.rename() issues a metadata-only update + rollback of fresh adds removes them (ac29b0e)
+- chore(release): 7.31.6 (9b52629)
+- fix: remap reserved fields from update() metadata patches to their canonical location (67e5fc8)
+- chore(release): 7.31.5 (e5ec658)
+- fix: feature-detect setVectorBackend before wiring the mmap-vector backend (a537b36)
+- chore(release): 7.31.4 (a8cbab6)
+- fix: feature-detect setConnectionsCodec before wiring the connections codec (747ab97)
+- chore(release): 7.31.3 (cfb051c)
+- fix: mmap-vector backend capacity NaN at the provider FFI boundary (eade6ff)
+
+
 ### [8.0.0-rc.9](https://github.com/soulcraftlabs/brainy/compare/v8.0.0-rc.8...v8.0.0-rc.9) (2026-07-01)
 
 - docs(8.0): RELEASES.md — rc.9 (migration LOCK + 6x cosine + ES2023/Node22 floor) (3a33987)

README.md

@@ -1,440 +1,217 @@
-# Brainy
-
 <p align="center">
-  <img src="https://raw.githubusercontent.com/soulcraftlabs/brainy/main/brainy.png" alt="Brainy Logo" width="200">
+  <img src="https://raw.githubusercontent.com/soulcraftlabs/brainy/main/brainy.png" alt="Brainy" width="180">
 </p>
 
-[![npm version](https://badge.fury.io/js/%40soulcraft%2Fbrainy.svg)](https://www.npmjs.com/package/@soulcraft/brainy)
-[![npm downloads](https://img.shields.io/npm/dm/@soulcraft/brainy.svg)](https://www.npmjs.com/package/@soulcraft/brainy)
-[![Documentation](https://img.shields.io/badge/docs-soulcraft.com-blue.svg)](https://soulcraft.com/docs)
-[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
-[![TypeScript](https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg)](https://www.typescriptlang.org/)
+<h1 align="center">Brainy</h1>
 
-**Three database paradigms. One API. Zero configuration.**
+<p align="center">
+  <b>Three database paradigms. One API. Zero configuration.</b><br>
+  The in-process knowledge database for TypeScript — vector search, graph traversal,<br>
+  and metadata filtering unified in a single query.
+</p>
 
-Built because we were tired of stitching together Pinecone + Neo4j + MongoDB and spending weeks on configuration before writing a single line of business logic. Brainy unifies vector search, graph traversal, and metadata filtering so you don't have to choose.
+<p align="center">
+  <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
+  <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
+  <a href="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml"><img src="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
+  <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>
+</p>
 
-**New here?** → **[What is Brainy? — plain-language overview, no jargon](docs/eli5.md)**
+<p align="center">
+  <a href="#quick-start">Quick start</a> ·
+  <a href="#one-query-three-engines">One query</a> ·
+  <a href="#feature-tour">Features</a> ·
+  <a href="#from-laptop-to-hundreds-of-millions">Scale with Cor</a> ·
+  <a href="#documentation">Docs</a>
+</p>
 
 ---
 
-## Install
+Built because we were tired of stitching a vector store to a graph database to a document store — and spending weeks on plumbing before writing a line of business logic. Brainy indexes every fact **three ways at once** and lets one call query them together:
+
+| You write | Brainy indexes it as | You query it with |
+|---|---|---|
+| `data: 'Ada wrote the first program'` | a **384-dim vector** (local embedding — no API key) | `find({ query: 'computing pioneers' })` |
+| `metadata: { field: 'CS', year: 1843 }` | **structured fields** (O(1) exact, O(log n) range) | `find({ where: { year: { lessThan: 1900 } } })` |
+| `relate({ from: ada, to: babbage })` | a **typed, directed graph edge** | `find({ connected: { to: babbage, depth: 2 } })` |
+
+It runs **inside your process** — no server, no Docker, nothing to operate — and persists to plain files you can snapshot with a hard link.
+
+**New here?** → **[What is Brainy? — plain-language overview, no jargon](docs/eli5.md)**
+
+## Quick start
 
 ```bash
-bun add @soulcraft/brainy        # fastest — recommended
-npm install @soulcraft/brainy    # Node.js — fully supported
+bun add @soulcraft/brainy        # Bun ≥ 1.1 — recommended
+npm install @soulcraft/brainy    # Node.js ≥ 22
 ```
 
-## Quick Start
-
 ```javascript
 import { Brainy, NounType, VerbType } from '@soulcraft/brainy'
 
-const brain = new Brainy()
+const brain = new Brainy()                       // in-memory; one line swaps to disk
 await brain.init()
 
-// Add knowledge — text auto-embeds, metadata auto-indexes
-const reactId = await brain.add({
+// Text auto-embeds locally; metadata auto-indexes
+const react = await brain.add({
   data: 'React is a JavaScript library for building user interfaces',
   type: NounType.Concept,
+  subtype: 'library',
   metadata: { category: 'frontend', year: 2013 }
 })
 
-const nextId = await brain.add({
-  data: 'Next.js framework for React with server-side rendering',
+const next = await brain.add({
+  data: 'Next.js is a React framework with server-side rendering',
   type: NounType.Concept,
-  metadata: { category: 'framework', year: 2016 }
+  subtype: 'framework',
+  metadata: { category: 'frontend', year: 2016 }
 })
 
-// Create a relationship
-await brain.relate({ from: nextId, to: reactId, type: VerbType.BuiltOn })
-
-// Query all three paradigms at once
-const results = await brain.find({
-  query: 'modern frontend frameworks',            // Vector similarity
-  where: { year: { greaterThan: 2015 } },         // Metadata filtering
-  connected: { to: reactId, depth: 2 }            // Graph traversal
-})
+await brain.relate({ from: next, to: react, type: VerbType.DependsOn, subtype: 'runtime' })
 ```
 
-**[Full API Reference](docs/api/README.md)** | **[soulcraft.com/docs](https://soulcraft.com/docs)**
-
----
-
-## Three Indexes, One Query
-
-Every piece of knowledge lives in three indexes simultaneously:
-
-- **`data`** → **Vector index** — Content for semantic search. Strings auto-embed into 384-dim vectors. Queried with `find({ query: '...' })`.
-- **`metadata`** → **Metadata index** — Structured fields for filtering. O(1) lookups. Queried with `find({ where: { ... } })`.
-- **`relate()`** → **Graph index** — Typed, directed relationships between entities. Traversed with `find({ connected: { ... } })`.
-
-```javascript
-// Data → vector index (semantic search)
-const articleId = await brain.add({
-  data: 'A deep dive into transformer architectures',
-  type: NounType.Document,
-  metadata: { author: 'Dr. Chen', year: 2024, tags: ['AI'] }  // → metadata index
-})
-
-// Relationships → graph index
-await brain.relate({ from: authorId, to: articleId, type: VerbType.Authored })
-
-// Query all three at once
-brain.find({
-  query: 'attention mechanisms',                  // Vector similarity
-  where: { year: { greaterThan: 2023 } },         // Metadata filter
-  connected: { from: authorId, depth: 1 }         // Graph traversal
-})
-```
-
-**[Data Model Reference](docs/DATA_MODEL.md)** | **[Query Operators](docs/QUERY_OPERATORS.md)**
-
----
-
-## Features
-
-### Triple Intelligence
-
-Vector search + graph traversal + metadata filtering in every query. No stitching services together — one `find()` call combines all three.
+## One query, three engines
 
 ```javascript
 const results = await brain.find({
-  query: 'machine learning',
-  where: { department: 'engineering', level: 'senior' },
-  connected: { from: teamLeadId, via: VerbType.WorksWith, depth: 2 }
+  query: 'modern frontend frameworks',           // vector — what it means
+  where: { year: { greaterThan: 2015 } },        // metadata — what it is
+  connected: { to: react, depth: 2 }             // graph — what it touches
 })
 ```
 
-### Hybrid Search
+Every clause is optional; any combination composes. Under the hood Brainy plans the query across an HNSW vector index, a roaring-bitmap field index, and an adjacency graph index — and re-validates every result against your predicate before returning it, so a corrupt index can never hand you a wrong answer.
 
-Automatically combines keyword (text) and semantic (vector) search. No configuration needed.
+## Feature tour
+
+### The database is a value
+
+Pin it, rewind it, fork it. Snapshot isolation without a server.
 
 ```javascript
-await brain.find({ query: 'David Smith' })             // Auto: text + semantic
-await brain.find({ query: 'AI concepts', searchMode: 'semantic' })  // Semantic only
-await brain.find({ query: 'exact id', searchMode: 'text' })         // Text only
+const db = brain.now()                           // pin current state — O(1)
+
+await brain.transact([                           // atomic all-or-nothing, CAS-guarded
+  { op: 'update', id: order, metadata: { status: 'paid' } },
+  { op: 'relate', from: invoice, to: order, type: VerbType.References, subtype: 'billing' }
+], { ifAtGeneration: db.generation })
+
+await db.get(order)                              // still 'pending' — pinned forever
+await brain.get(order)                           // 'paid' — live
+
+const lastWeek = await brain.asOf(Date.now() - 7 * 86_400_000)   // full query surface, past state
+const whatIf = await db.with([{ op: 'remove', id: order }])      // speculative — never touches disk
+await brain.now().persist('/backups/today')                      // instant hard-link snapshot
 ```
 
-### Query Operators
+**[Consistency model](docs/concepts/consistency-model.md)** · **[Snapshots & time travel](docs/guides/snapshots-and-time-travel.md)**
 
-Filter metadata with equality, comparison, array, existence, pattern, and logical operators:
+### Local embeddings — no API keys
+
+Strings embed on-device with a bundled MiniLM model (WASM). Semantic search works offline, in CI, and on air-gapped machines, at zero cost per call. Hybrid keyword + semantic ranking is the default:
 
 ```javascript
-await brain.find({
-  where: {
-    status: 'active',                          // Exact match
-    score: { greaterThan: 90 },                // Comparison
-    tags: { contains: 'ai' },                  // Array
-    anyOf: [{ role: 'admin' }, { role: 'owner' }]  // Logical OR
-  }
-})
+await brain.find({ query: 'David Smith' })                            // auto: text + semantic
+await brain.find({ query: 'AI concepts', searchMode: 'semantic' })    // semantic only
 ```
 
-**[Query Operators Reference](docs/QUERY_OPERATORS.md)** — all operators with indexed/in-memory matrix
+### A typed graph, not a bag of edges
 
-### Graph Relationships
-
-Typed, directed edges between entities. Traverse connections at any depth.
+42 entity types × 127 relationship types form a shared vocabulary for any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), yours. Your own taxonomy layers on with `subtype`, enforced at write time:
 
 ```javascript
-await brain.relate({ from: personId, to: projectId, type: VerbType.WorksOn })
+await brain.add({ data: 'Avery Brooks', type: NounType.Person, subtype: 'employee' })
 
-const results = await brain.find({
-  connected: { from: personId, via: VerbType.WorksOn, depth: 3 }
-})
+brain.counts.bySubtype(NounType.Person)          // O(1) — { employee: 12, customer: 847 }
+brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true })
 ```
 
-### Database as a Value
+**[Type system](docs/architecture/noun-verb-taxonomy.md)** · **[Subtypes & facets](docs/guides/subtypes-and-facets.md)**
 
-The whole database, pinned as an immutable value. Snapshot isolation, time travel, atomic transactions, instant hard-link snapshots.
+### Graph analytics built in
 
 ```javascript
-const db = brain.now()                         // Pin current state — O(1)
-
-// Atomic multi-write transaction (all-or-nothing, with CAS)
-await brain.transact([
-  { op: 'update', id: orderId, metadata: { status: 'paid' } },
-  { op: 'relate', from: invoiceId, to: orderId, type: VerbType.References, subtype: 'billing' }
-], { meta: { author: 'billing-service' }, ifAtGeneration: db.generation })
-
-await db.get(orderId)                          // Still 'pending' — pinned, forever
-await brain.get(orderId)                       // 'paid' — live
-
-// Time travel: full query surface at any past state
-const yesterday = await brain.asOf(new Date(Date.now() - 86_400_000))
-const past = await yesterday.find({ query: 'unpaid orders' })
-
-// What-if: speculative writes, nothing touches disk
-const whatIf = await db.with([{ op: 'remove', id: orderId }])
-
-// Instant backup: hard-link snapshot, opens read-only with Brainy.load()
-await brain.now().persist('/backups/today')
+await brain.graph.rank()                          // which entities matter most (centrality)
+await brain.graph.communities()                   // natural clusters
+await brain.graph.path(a, b)                      // how two things connect
+await brain.graph.subgraph([seed], { depth: 2 })  // bounded neighborhood → { nodes, edges }
+await brain.graph.export()                        // whole graph, one O(N+E) streaming pass
 ```
 
-**[Consistency Model](docs/concepts/consistency-model.md)** | **[Snapshots & Time Travel](docs/guides/snapshots-and-time-travel.md)**
+### Write-time aggregations
 
-### Virtual Filesystem
+`SUM` / `COUNT` / `AVG` / `MIN` / `MAX` with `GROUP BY` and time windows, maintained incrementally on every write — reads are O(1) lookups, not scans. **[Aggregation guide](docs/guides/aggregation.md)**
 
-File operations with semantic search built in.
-
-```javascript
-const vfs = brain.vfs
-
-await vfs.writeFile('/docs/readme.md', 'Project documentation')
-const content = await vfs.readFile('/docs/readme.md')
-const tree = await vfs.getTreeStructure('/docs', { maxDepth: 3 })
-
-// Semantic file search
-const matches = await vfs.search('React components with hooks')
-```
-
-**[VFS Quick Start](docs/vfs/QUICK_START.md)** | **[Common Patterns](docs/vfs/COMMON_PATTERNS.md)**
-
-### Import Anything
-
-CSV, Excel, PDF, URLs — auto-detected format, auto-classified entities.
+### Import anything
 
 ```javascript
 await brain.import('customers.csv')
-await brain.import('sales-data.xlsx')                              // all sheets processed
-await brain.import('research-paper.pdf')                           // tables extracted automatically
+await brain.import('sales.xlsx')                              // every sheet
+await brain.import('research-paper.pdf')                      // tables extracted
 await brain.import('https://api.example.com/data.json')
-await brain.import('./handbook.md', { vfsPath: '/imports/handbook' })  // preserve in VFS
 ```
 
-**[Import Guide](docs/guides/import-anything.md)**
+Entities auto-classify on the way in; `brain.extractEntities(text)` exposes the same NER ensemble directly. **[Import guide](docs/guides/import-anything.md)**
 
-### Entity Extraction
-
-AI-powered named entity recognition with 4-signal ensemble scoring.
+### A filesystem that understands content
 
 ```javascript
-const entities = await brain.extractEntities('John Smith founded Acme Corp in New York')
-// [
-//   { text: 'John Smith', type: NounType.Person, confidence: 0.95 },
-//   { text: 'Acme Corp', type: NounType.Organization, confidence: 0.92 },
-//   { text: 'New York', type: NounType.Location, confidence: 0.88 }
-// ]
+await brain.vfs.writeFile('/docs/readme.md', 'Project documentation')
+await brain.vfs.search('React components with hooks')          // semantic file search
 ```
 
-**[Neural Extraction Guide](docs/neural-extraction.md)**
+**[VFS quick start](docs/vfs/QUICK_START.md)**
 
-### Plugin System
+### Operations-grade by default
 
-Optional native acceleration via `@soulcraft/cortex` — SIMD distance calculations, CRoaring bitmaps, Candle ML embeddings.
+- **Single-writer, many-reader** — an exclusive lock protects the data directory; `Brainy.openReadOnly()` and the `brainy inspect` CLI examine a live brain from another process, safely.
+- **Self-upgrading data files** — a 7.x brain opens under 8.x and migrates itself behind an observable lock (`getIndexStatus().migration`), with an automatic pre-upgrade backup. No migration scripts.
+- **No silent wrong answers** — cold-open guards self-heal or throw typed errors (`MetadataIndexNotReadyError`, `GraphIndexNotReadyError`); they never return `[]` for data that exists.
+
+**[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)**
+
+## From laptop to hundreds of millions
+
+Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**:
+
+```bash
+npm install @soulcraft/cor
+```
 
 ```javascript
-const brain = new Brainy({ plugins: ['@soulcraft/cortex'] })
-await brain.init()
+const brain = new Brainy({ storage: { type: 'filesystem', path: './data' } })
+await brain.init()   // @soulcraft/cor detected — same code, native engines underneath
 ```
 
-Plugins are opt-in. Brainy never auto-imports packages unless listed in `plugins`.
+Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate.
 
-**[Plugin Documentation](docs/PLUGINS.md)**
+Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both.
 
----
+## Performance
 
-## Type System
+- JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41).
+- Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan.
+- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
 
-42 noun types and 127 verb types form a universal knowledge protocol:
+## Use cases
 
-```
-42 Nouns × 127 Verbs = 5,334 base relationship combinations
-```
-
-Model any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), education (`Student → completes → Course`), or your own.
-
-### Subtypes — sub-classification within a NounType *or* VerbType
-
-Both noun types and verb types are intentionally coarse. Use the top-level `subtype` field to sub-classify entities AND relationships within a type — flat string, no hierarchy, your choice of vocabulary:
-
-```javascript
-// Nouns: sub-classify entities
-await brain.add({
-  data: 'Avery Brooks — runs the AI lab',
-  type: NounType.Person,
-  subtype: 'employee'                 // 'customer', 'vendor', 'contractor', …
-})
-
-// Verbs: sub-classify relationships
-await brain.relate({
-  from: ceoId,
-  to: vpId,
-  type: VerbType.ReportsTo,
-  subtype: 'direct'                   // 'dotted-line', 'matrix', …
-})
-
-// Filter on the fast path — column-store hit, not metadata fallback:
-const employees = await brain.find({ type: NounType.Person, subtype: 'employee' })
-const directReports = await brain.related({ from: ceoId, subtype: 'direct' })
-
-// O(1) counts via the persisted rollups:
-brain.counts.bySubtype(NounType.Person)
-// → { employee: 12, customer: 847, vendor: 34 }
-
-brain.counts.byRelationshipSubtype(VerbType.ReportsTo)
-// → { direct: 12, 'dotted-line': 3 }
-```
-
-**Enforce the pairing.** Register a vocabulary per type or turn on brain-wide strict mode to ensure every entity AND relationship has both `type` AND `subtype`:
-
-```javascript
-// Per-type rule with a closed vocabulary
-brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true })
-
-// 8.0 default: every write requires a subtype. Exempt genuine catch-all types…
-const brain = new Brainy({ requireSubtype: { except: [NounType.Thing] } })
-
-// …or opt out while migrating pre-8.0 data, then audit and back-fill:
-const legacy = new Brainy({ requireSubtype: false })
-await legacy.audit()                                      // gaps, grouped by type
-await legacy.fillSubtypes({ [NounType.Person]: 'unspecified' })  // close them
-```
-
-For other facets you want counted (`status`, `source`, `role`), register them with `brain.trackField(name)`. Renaming an existing convention to `subtype`? Use `brain.migrateField({from, to, entityKind: 'both'})` to walk nouns AND verbs in one pass. Full guide: **[Subtypes & Facets](docs/guides/subtypes-and-facets.md)**.
-
-**[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** | **[Stage 3 Canonical Reference](docs/STAGE3-CANONICAL-TAXONOMY.md)**
-
----
-
-## Storage: Memory and Filesystem
-
-The same API at every scale. Change one config line to go from prototype to production.
-
-### Development — Zero Config
-
-```javascript
-const brain = new Brainy()
-```
-
-### Production — Filesystem (gzip compression on by default)
-
-```javascript
-const brain = new Brainy({
-  storage: { type: 'filesystem', path: './data' }
-})
-```
-
-### Backups and Portability — Snapshots
-
-```javascript
-const db = brain.now()
-await db.persist('/backups/2026-06-11')   // instant hard-link snapshot
-await db.release()
-
-const snapshot = await Brainy.load('/backups/2026-06-11')  // read-only Db
-const hits = await snapshot.find({ query: 'quarterly invoices' })
-await snapshot.release()
-```
-
-A snapshot directory is self-contained — copy it to another machine, open it with `Brainy.load()`, or restore it wholesale with `brain.restore(path, { confirm: true })`.
-
-Performance benchmarks and capacity planning in **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)**.
-
-**[Capacity Planning](docs/operations/capacity-planning.md)**
-
----
-
-## Use Cases
-
-- **AI agents** — Persistent memory with semantic recall and relationship tracking
-- **Knowledge bases** — Auto-linking, semantic search, relationship-aware navigation
-- **Semantic search** — Find by meaning across codebases, documents, or media
-- **Enterprise knowledge** — CRM, product catalogs, institutional memory
-- **Interactive experiences** — Game worlds, NPCs, and characters that remember
-- **Content platforms** — Similarity-based discovery, intelligent tagging
-
----
+**AI agent memory** — persistent semantic recall with relationship tracking · **Knowledge bases** — auto-linking and meaning-aware navigation · **Semantic search** over codebases, documents, media · **Enterprise data** — CRM, catalogs, institutional memory · **Games & simulations** — worlds and characters that remember.
 
 ## Documentation
 
-### Start Here
-
-- **[Brainy explained simply](docs/eli5.md)** — Plain-language overview, no jargon, no code
-
-### Core
-
-- **[API Reference](docs/api/README.md)** — Every method with parameters, returns, and examples
-- **[Data Model](docs/DATA_MODEL.md)** — Entity structure, data vs metadata
-- **[Query Operators](docs/QUERY_OPERATORS.md)** — All BFO operators with examples
-- **[Find System](docs/FIND_SYSTEM.md)** — Natural language find() and hybrid search
-
-### Architecture
-
-- **[Architecture Overview](docs/architecture/overview.md)** — System design and components
-- **[Triple Intelligence](docs/architecture/triple-intelligence.md)** — Vector + graph + metadata unified query
-- **[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** — Universal type system
-- **[Data Storage Architecture](docs/architecture/data-storage-architecture.md)** — Type-aware indexing and HNSW
-
-### Virtual Filesystem
-
-- **[VFS Quick Start](docs/vfs/QUICK_START.md)** — Build file explorers that never crash
-- **[VFS Core](docs/vfs/VFS_CORE.md)** — Full VFS API reference
-- **[Semantic VFS](docs/vfs/SEMANTIC_VFS.md)** — AI-powered file navigation
-
-### Guides
-
-- **[Import Anything](docs/guides/import-anything.md)** — CSV, Excel, PDF, URLs
-- **[Framework Integration](docs/guides/framework-integration.md)** — React, Vue, Angular, Svelte
-- **[Natural Language Queries](docs/guides/natural-language.md)** — Master the find() method
-
-### Operations
-
-- **[Capacity Planning](docs/operations/capacity-planning.md)** — Memory, storage, and scaling
-- **[Performance](docs/PERFORMANCE.md)** — Benchmarks and architecture details
-
----
+| Start | Core | Going deeper |
+|---|---|---|
+| [Brainy explained simply](docs/eli5.md) | [API reference](docs/api/README.md) | [Architecture overview](docs/architecture/overview.md) |
+| [Installation](docs/guides/installation.md) | [Data model](docs/DATA_MODEL.md) | [Consistency model](docs/concepts/consistency-model.md) |
+| [Natural-language queries](docs/guides/natural-language.md) | [Query operators](docs/QUERY_OPERATORS.md) | [Multi-process model](docs/concepts/multi-process.md) |
+| | [Find system](docs/FIND_SYSTEM.md) | [Scaling](docs/SCALING.md) |
 
 ## Requirements
 
-**Bun 1.1+** (recommended) or **Node.js 22 LTS**
+**Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use.
 
-```bash
-bun add @soulcraft/brainy        # Bun — best performance
-npm install @soulcraft/brainy    # Node.js — fully supported
-```
+## Contributing & license
 
-> Brainy 8.0 is server-only. Browser support (OPFS storage, Web Workers, in-browser WASM embeddings) was removed in 8.0 — the 7.x line remains available on npm if you need it.
-
-## Single-Writer Model
-
-Brainy is **single-writer, many-reader** on filesystem storage. One writer
-holds an exclusive lock on the data directory; any number of readers can
-inspect it concurrently. Opening a second writer throws with the PID of the
-existing one.
-
-```typescript
-// Live application — writer mode is the default
-const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/brain' } })
-await brain.init()
-
-// Out-of-band diagnostics from a separate process — safe to run while the
-// writer is live
-const reader = await Brainy.openReadOnly({
-  storage: { type: 'filesystem', path: '/data/brain' }
-})
-await reader.requestFlush({ timeoutMs: 5000 })
-const stats = await reader.stats()
-```
-
-For incident debugging, use the `brainy inspect` CLI:
-
-```bash
-brainy inspect stats   /data/brain
-brainy inspect find    /data/brain --where '{"entityType":"booking"}'
-brainy inspect explain /data/brain --where '{"entityType":"booking"}'
-brainy inspect health  /data/brain
-```
-
-See [the multi-process model](docs/concepts/multi-process.md) and the
-[inspection guide](docs/guides/inspection.md) for the full story, including
-stale-lock detection and the cross-process flush RPC.
-
-## Contributing
-
-We welcome contributions! See **[CONTRIBUTING.md](CONTRIBUTING.md)** for guidelines.
-
-## License
-
-MIT © Brainy Contributors
+Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors.

docs/PERFORMANCE.md

@@ -417,7 +417,7 @@ const brain = new Brainy({
 })
 ```
 
-The default JS index is `JsHnswVectorIndex`. An optional native acceleration package (`@soulcraft/cortex`) can replace it with a higher-performing implementation; the public knobs stay the same.
+The default JS index is `JsHnswVectorIndex`. An optional native acceleration package (`@soulcraft/cor`) can replace it with a higher-performing implementation; the public knobs stay the same.
 
 ### Scale Scenarios
 

docs/PLUGINS.md

@@ -12,7 +12,7 @@ next:
 
 # Plugin Development Guide
 
-Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cortex` provides optional native acceleration, and it's the same system available to any developer.
+Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cor` provides optional native acceleration, and it's the same system available to any developer.
 
 ## Architecture Overview
 
@@ -23,20 +23,19 @@ Brainy's plugin system uses **named providers** — string keys mapped to implem
 3. The plugin calls `context.registerProvider(key, implementation)` for each subsystem it provides
 4. Brainy checks each provider key and wires the implementation into its internal pipeline
 
-Plugins are **opt-in** — brainy never auto-imports packages. You must explicitly list plugins in the config:
+Installing the first-party accelerator is the opt-in: with the default config, brainy probes for `@soulcraft/cor` and loads it when present. Everything except "not installed" fails **loud** — a present-but-broken accelerator makes `init()` throw rather than silently degrading to the JS engines.
 
 ```typescript
-const brain = new Brainy({
-  plugins: ['@soulcraft/cortex']  // explicitly load the native acceleration package
-})
+const brain = new Brainy()                                  // @soulcraft/cor auto-detected when installed
+const pinned = new Brainy({ plugins: ['@soulcraft/cor'] })  // or pin exactly what loads
+const plain = new Brainy({ plugins: [] })                   // or opt out of detection entirely
 ```
 
 | `plugins` value | Behavior |
 |---|---|
-| `undefined` (default) | No plugins loaded |
-| `false` | No plugins loaded |
-| `[]` | No plugins loaded |
-| `['@soulcraft/cortex']` | Load only the listed packages |
+| `undefined` (default) | Guarded auto-detection of `@soulcraft/cor`: not installed → no plugins, silently; installed → loads + announces; installed-but-broken → `init()` throws |
+| `false` / `[]` | No plugins, no detection (explicit opt-out) |
+| `['@soulcraft/cor']` | Load only the listed packages; a listed plugin that fails to load throws |
 
 Plugins registered programmatically via `brain.use(plugin)` are always activated regardless of the `plugins` config.
 
@@ -70,7 +69,7 @@ export default myPlugin
 
 ### 2. Package exports
 
-Your package must export the plugin as the default export so brainy's auto-detection works:
+Your package must export the plugin as the default export so brainy's plugin loader can resolve it:
 
 ```typescript
 // index.ts
@@ -150,6 +149,17 @@ context.registerProvider('embedBatch', async (texts: string[]) => {
 
 ### Index Providers
 
+> **Write-path invariant (the change-feed contract).** Every canonical
+> mutation flows through Brainy's generation-store commit points — index
+> providers are invoked *inside* that commit and never originate canonical
+> writes of their own. The `brain.onChange` change feed is emitted from those
+> commit points and relies on this: **a plugin must never introduce a write
+> path that bypasses the generation-store commit.** If a future provider ever
+> needs a direct native ingest path, it must either route through the commit
+> or emit equivalent change events — otherwise every `onChange` consumer
+> (live UIs, cache invalidation, realtime sync) silently develops a blind
+> spot.
+
 #### `vector`
 **Type:** `(config: object, distanceFunction: Function, options: object) => VectorIndexProvider-compatible`
 
@@ -182,6 +192,27 @@ context.registerProvider('vector', (config, distanceFn, options) => {
 })
 ```
 
+#### The readiness contract (all three index providers)
+
+A provider that **persists its derived index** should implement the optional readiness
+members so a warm reopen never pays a redundant rebuild-from-canonical:
+
+- **`init?(): Promise<void>`** — eager cold-load. Brainy awaits it once during
+  `brain.init()`, after the metadata provider's `init()` (the id-mapper hydrates first)
+  and **before the rebuild gate**.
+- **`isReady?(): boolean`** — honest durability signal. `true` ⇔ the persisted index is
+  loaded (or cheaply demand-loadable) and consistent with what was last persisted. When
+  exposed, the rebuild gate defers to this signal **instead of** the `size() === 0` /
+  `totalEntries === 0` heuristics — a disk-native index may report 0 resident entries
+  while fully durable. Never return `true` if the durable state failed to load: the
+  signal is honest in both directions, and a not-ready provider gets its rebuild even
+  when `size() > 0`.
+- **`isMigrating?(): boolean`** — while `true`, the provider owns its index (background
+  migration); brainy skips its rebuild entirely.
+
+Providers that implement none of these keep the size/count heuristics — correct for
+engines whose `rebuild()` *is* their load path (like brainy's built-in JS vector index).
+
 #### `metadataIndex`
 **Type:** `(storage: StorageAdapter) => MetadataIndexManager-compatible`
 
@@ -215,7 +246,7 @@ context.registerProvider('aggregation', (storage) => {
 })
 ```
 
-When provided by an optional native acceleration plugin (such as `@soulcraft/cortex`), this enables:
+When provided by an optional native acceleration plugin (such as `@soulcraft/cor`), this enables:
 - Compiled source filters (vs per-entity JS object traversal)
 - Precise MIN/MAX via sorted data structures (vs lazy recompute)
 - Parallel aggregate rebuild across CPU cores
@@ -251,7 +282,7 @@ Native msgpack encode/decode for SSTable serialization.
 
 ### Analytics Providers (Native-Only)
 
-These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when an optional native acceleration plugin (such as `@soulcraft/cortex`) is installed.
+These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when an optional native acceleration plugin (such as `@soulcraft/cor`) is installed.
 
 Use `brain.getProvider('analytics:hyperloglog')` to check availability. Returns `undefined` if no plugin provides it.
 
@@ -359,8 +390,8 @@ brainy diagnostics
 When a plugin is active, brainy automatically logs a provider summary after `init()`:
 
 ```
-[brainy] Plugin activated: @soulcraft/cortex
-[brainy] Providers: 8/10 native (@soulcraft/cortex) | default: vector, cache
+[brainy] Plugin activated: @soulcraft/cor
+[brainy] Providers: 8/10 native (@soulcraft/cor) | default: vector, cache
 ```
 
 This tells you at a glance how many subsystems are accelerated and which ones are falling back to JavaScript. The log respects `config.silent`.
@@ -381,7 +412,7 @@ If a required provider is missing, the error message tells you exactly what's wr
 
 ```
 [brainy] Required providers using JS fallback: graphIndex.
-Active plugins: @soulcraft/cortex.
+Active plugins: @soulcraft/cor.
 These providers must be supplied by a plugin for this deployment.
 Check plugin installation, license, and native module availability.
 ```

docs/QUERY_OPERATORS.md

@@ -234,7 +234,7 @@ const all = await brain.related({
 })
 
 // Graph traversal — subtype filters traversal edges (depth-1 in 7.30 JS path;
-// multi-hop subtype filtering lands on Cortex native)
+// multi-hop subtype filtering lands on Cor native)
 const reports = await brain.find({
   connected: {
     from: ceoId,

docs/RELEASE-GUIDE.md

@@ -120,3 +120,12 @@ gh release create vY.Y.Y --generate-notes
 - **Major versions should be RARE**
 - **When in doubt, it's probably MINOR**
 - **NEVER use "BREAKING CHANGE" for internal changes**
+## Hard Ordering Constraints (check before EVERY release)
+
+- **Embedding model changes are SEQUENCED, not free.** No release may change the
+  embedding model (or its quantization/dimensions) before **vector model-version
+  stamping + hard-error-on-mismatch** ships. Stored vectors carry no model version
+  today; mixing vectors from two models silently corrupts every similarity
+  comparison. If a model bump is ever proposed, the stamping work moves ahead of
+  it in the schedule — coordinate with the native provider so both engines stamp
+  and enforce identically. (Registered with the native-provider team 2026-07-07.)

docs/SCALING.md

@@ -37,7 +37,7 @@ The three knobs that matter most:
 1. **`config.vector.recall`** — `'fast'`, `'balanced'`, or `'accurate'` (default `'balanced'`)
 2. **`config.vector.persistMode`** — `'immediate'` for durability, `'deferred'` for throughput
 
-The native vector provider (via the optional `@soulcraft/cortex` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.
+The native vector provider (via the optional `@soulcraft/cor` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.
 
 ## Measured Performance
 
@@ -77,7 +77,7 @@ graph, metadata index, and 100k edges).
 **Scale ceiling (open-core).** The pure-JS HNSW *build* cost (~100 inserts/s at 384-dim on
 one core) makes the in-process open-core path most appropriate up to ~10⁵–10⁶ entities.
 *Query* latency stays low well beyond that, but for the 10⁸–10¹⁰ regime install the native
-provider (`@soulcraft/cortex`, on-disk DiskANN) — same API, no code change. _Projected from
+provider (`@soulcraft/cor`, on-disk DiskANN) — same API, no code change. _Projected from
 the two measured points, vector p50 at 1M is ~2 ms; metadata-heavy composition grows with
 match-set size and is the path to move onto the native provider first._
 
@@ -213,12 +213,12 @@ const stats = await brain.stats()
 ### Issue: Slow queries
 1. Switch to `vector.recall: 'fast'`
 2. Increase the read cache (`storage.cache.maxSize`)
-3. Consider the optional native vector provider via `@soulcraft/cortex`
+3. Consider the optional native vector provider via `@soulcraft/cor`
 
 ### Issue: Memory pressure
 1. Reduce `storage.cache.maxSize`
 2. Move to `vector.persistMode: 'deferred'` to batch writes
-3. Consider the optional native vector provider via `@soulcraft/cortex` for at-scale index acceleration
+3. Consider the optional native vector provider via `@soulcraft/cor` for at-scale index acceleration
 
 ### Issue: Slow startup after a crash
 1. Use `vector.persistMode: 'immediate'` so the index file stays in sync with storage

docs/architecture/aggregation.md

@@ -33,7 +33,7 @@
 │             ▼                                            │
 │  ┌──────────────────────┐                                │
 │  │  AggregationProvider │  (optional, registered by      │
-│  │  ├─ incrementalUpdate│   plugin like @soulcraft/cortex)│
+│  │  ├─ incrementalUpdate│   plugin like @soulcraft/cor)   │
 │  │  ├─ rebuildAggregate │                                │
 │  │  ├─ queryAggregate   │                                │
 │  │  └─ serialize/restore│                                │
@@ -135,7 +135,7 @@ M2 is clamped to zero on remove to prevent floating-point drift from producing n
 
 The TypeScript engine uses simple comparison for add operations and marks MIN/MAX as potentially stale on delete (since removing the current min/max value requires a rescan). Stale values are lazily recomputed on the next query.
 
-The Cortex native engine uses a `BTreeMap<OrderedFloat<f64>, u64>` that tracks the exact frequency of every value, providing precise MIN/MAX after any sequence of operations without rescanning.
+The Cor native engine uses a `BTreeMap<OrderedFloat<f64>, u64>` that tracks the exact frequency of every value, providing precise MIN/MAX after any sequence of operations without rescanning.
 
 ### Time Window Bucketing
 

docs/architecture/data-storage-architecture.md

@@ -369,7 +369,10 @@ await brain.restore('/backups/today', { confirm: true })   // replace store stat
   through the metadata index, not the filesystem.
 - **`_system/`** holds singletons plus 256 hash buckets of index state.
 - **`_generations/` + `manifest.json` + `tx-log.jsonl`** implement
-  generational MVCC; `transact()` is the unit of history.
+  generational MVCC. History is per-write: every `add()`/`update()`/`remove()`/
+  `relate()` gets its own generation, and `transact()` groups several ops into
+  one atomic generation. (Single-op retention has been the model since 8.0;
+  you never need to route a write through `transact()` just to keep its history.)
 - **`_column_index/` + `_blobs/`** hold the columnar metadata runs and binary
   blobs (VFS content included).
 - **`locks/`** coordinates the single writer and reader flush requests, and

docs/architecture/index-architecture.md

@@ -402,7 +402,7 @@ const DEFAULT_EXCLUDE_FIELDS = [
 
 **Purpose**: O(log n) semantic similarity search using vector embeddings.
 
-The default JS implementation is `JsHnswVectorIndex`; an optional native acceleration package (`@soulcraft/cortex`) can register a higher-performing `VectorIndexProvider` through the plugin system. The public API stays the same either way.
+The default JS implementation is `JsHnswVectorIndex`; an optional native acceleration package (`@soulcraft/cor`) can register a higher-performing `VectorIndexProvider` through the plugin system. The public API stays the same either way.
 
 ### Internal Architecture
 

docs/architecture/multiprocess-storage-mixin.md

@@ -22,7 +22,7 @@ requestFlushOverFilesystem(timeoutMs)
 
 They live on `BaseStorage` as no-op defaults and are overridden on
 `FileSystemStorage` with real implementations. Any adapter extending
-`FileSystemStorage` (e.g. Cortex's `MmapFileSystemStorage`) inherits the
+`FileSystemStorage` (e.g. Cor's `MmapFileSystemStorage`) inherits the
 real ones for free.
 
 This works correctly today. The question is whether the methods *belong*
@@ -83,7 +83,7 @@ Benefits:
 ## The case against doing it now
 
 - Breaking change for any adapter that already overrides these methods.
-  `FileSystemStorage` is the only one in-tree, but Cortex's
+  `FileSystemStorage` is the only one in-tree, but Cor's
   `MmapFileSystemStorage` inherits from it — interface relocation would
   ripple through the plugin ecosystem.
 - The current state works. The real failure modes seen in the field

docs/architecture/overview.md

@@ -48,7 +48,7 @@ Pluggable vector index (`VectorIndexProvider`) for efficient nearest-neighbor se
 - **Configurable recall**: `fast` / `balanced` / `accurate` presets trade recall for latency
 - **Scalable**: Handles millions of vectors per process
 - **Persistent**: Serializable to storage
-- **Swappable**: Replace with a native implementation (such as `@soulcraft/cortex`) via the plugin system without changing application code
+- **Swappable**: Replace with a native implementation (such as `@soulcraft/cor`) via the plugin system without changing application code
 
 ### Metadata Index Manager
 High-performance field indexing system:

docs/architecture/zero-config.md

@@ -84,7 +84,7 @@ const brain = new Brainy({
 ```
 
 The default JS index is `JsHnswVectorIndex`. An optional native acceleration
-provider (the `@soulcraft/cortex` package) can replace it with a
+provider (the `@soulcraft/cor` package) can replace it with a
 higher-performing implementation; the public knobs stay the same. Quantization
 and other index-internal acceleration are the native provider's concern, not a
 Brainy configuration option.

docs/concepts/generation-fact-log.md

@@ -0,0 +1,117 @@
+---
+title: The Generation Fact Log
+slug: concepts/generation-fact-log
+public: true
+category: concepts
+template: concept
+order: 6
+description: Every committed write also appends a self-verifying "fact" — an after-image commit record — to an append-only log. What facts are, the crash-safety model, the scanFacts() streaming surface, family stamps, and how index providers consume the log for sequential heals.
+next:
+  - concepts/consistency-model
+  - guides/snapshots-and-time-travel
+---
+
+# The Generation Fact Log
+
+Since 8.4.0, every committed generation also appends a **fact** — a compact record of what each
+touched entity or relationship *became* — to an append-only, checksummed log under
+`_generations/facts/`. Where the generational history answers *"what did things look like
+before?"* (before-images, powering `asOf()` and rollback), the fact log answers *"what happened,
+in order?"* — one sequential, self-verifying stream of the store's present being written.
+
+Nothing about querying changes. The fact log exists for three consumers:
+
+1. **Index heals and rebuilds** — one sequential read in commit order replaces a per-entity
+   directory walk over millions of files.
+2. **Incremental catch-up** — a derived index that knows which generation it reflects reads *just
+   the gap*, instead of rebuilding from scratch.
+3. **Replay and audit tooling** — anything that wants the store's committed timeline as a stream.
+
+## What a fact is
+
+One fact per committed generation:
+
+- **`generation`** and **`timestamp`** — which commit, when.
+- **`ops`** — every write in that commit: `{ kind: 'noun' | 'verb', id, record }` where `record`
+  holds the entity's full after-image (both stored legs), or **`null` for a tombstone** — a
+  removal carries no body, by design.
+- **`meta`** — the transaction metadata `transact()` was submitted with, when present.
+- **`blobHashes`** — content-blob references, for exact reclamation accounting.
+
+Facts accumulate **from the first write after upgrading** — pre-existing history is not
+retroactively converted, and consumers fall back to the enumeration walk when no log exists.
+
+## Crash safety, in one paragraph
+
+Facts are appended and fsynced **inside the same durability window as the commit itself**, before
+the commit point — so after a crash, the log can only ever be *ahead* of committed truth, never
+behind it with a hole. On open, the store reconciles the log back to the committed watermark:
+torn tails are detected by per-record checksums and cut; whole records beyond the watermark are
+truncated. The invariant every reader can rely on: **an absent generation was never committed; a
+present fact was.** `transact()` facts are durable the moment `transact()` returns; single-op
+facts share the same group-commit flush as the rest of their generation, so a hard kill loses the
+fact and the generation *together* — never a torn state.
+
+## Reading the log
+
+```typescript
+const scan = brain.scanFacts({ fromGeneration: 1 })
+if (scan) {
+  // Telemetry up front — progress bars get a denominator from second zero.
+  console.log(scan.headGeneration, scan.segmentCount, scan.approxFactCount)
+
+  for await (const batch of scan.batches()) {
+    // Each batch: { facts, firstGeneration, lastGeneration, factCount, byteSize, segmentId }
+    for (const fact of batch.facts) {
+      for (const op of fact.ops) {
+        if (op.record === null) {
+          // a tombstone: op.id was removed in this generation
+        }
+      }
+    }
+  }
+
+  console.log(scan.summary()) // { factsYielded, segmentsRead } — the cross-check
+}
+```
+
+- `scanFacts()` returns `null` when the store hosts no fact log (older store, or a storage adapter
+  without binary append support) — fall back to enumerating entities.
+- Scans run against a **snapshot**: facts appended after the scan opens never bleed in, each fact
+  is yielded exactly once, and a detected gap aborts loudly — never a silent skip.
+- `brain.factSegmentPaths()` returns the immutable, *sealed* segment files for zero-copy consumers
+  (the append-mutable tail is excluded — read it through `scanFacts()`).
+
+## Family stamps: how a projection proves it's current
+
+Anything derived from the store — an index, the entity file tree itself — carries a **family
+stamp**: a small JSON record of *which committed generation the projection reflects*
+(`sourceGeneration`) plus the invariants that verify it whole (exact per-file byte sizes for
+bounded families; rollup invariants like entity counts for unbounded ones). At open, coherence is
+a **comparison**, not a walk:
+
+- stamp equals the committed watermark and invariants hold → serve;
+- stamp is behind → the projection reads just the gap from the fact log;
+- invariants fail → loud, named divergence — `brain.repairIndex()` rebuilds from canonical and
+  re-stamps.
+
+The verifier is exported (`verifyFamilyStamp`) so every projection — TypeScript or native — runs
+literally the same check.
+
+## For plugin authors: the storage capability
+
+Index providers receive the storage adapter, not the brain — so the host wires the log onto it.
+Feature-detect and prefer the stream; fall back to enumeration:
+
+```typescript
+const scan = storage.scanFacts?.({ fromGeneration: stamp.sourceGeneration + 1 })
+if (scan) {
+  // sequential catch-up from the log
+} else {
+  // enumeration walk (older store or adapter)
+}
+const committed = storage.committedGeneration?.() // the watermark stamps compare against
+```
+
+Providers must never construct their own reader over the log's files — the open path belongs to
+the single writer (it reconciles the log at open); the capability is the sanctioned seam.

docs/concepts/multi-process.md

@@ -55,20 +55,20 @@ process opening the same directory:
 The fix is the lock: refuse to open a second writer. SQLite has done the same
 since the late 1990s (`SQLITE_BUSY`).
 
-## What about Cortex?
+## What about Cor?
 
-Brainy + Cortex compose cleanly under this model:
+Brainy + Cor compose cleanly under this model:
 
-- Cortex stores its column-index segments inside the same `rootDir` (under
+- Cor stores its column-index segments inside the same `rootDir` (under
   `indexes/_column_index/{field}/`).
-- Segments (`*.cidx` files) are **immutable** once written. Cortex mmaps them
+- Segments (`*.cidx` files) are **immutable** once written. Cor mmaps them
   read-only.
 - The `MANIFEST.json` per field is updated via atomic rename — readers see
   either the old or new manifest, never a torn file.
 
-A reader process can safely mmap Cortex segments alongside a live writer
+A reader process can safely mmap Cor segments alongside a live writer
 without coordination. The single Brainy writer lock at
-`<rootDir>/locks/_writer.lock` covers Cortex too, because Cortex segment
+`<rootDir>/locks/_writer.lock` covers Cor too, because Cor segment
 writes happen on the writer's side.
 
 ## Stale-lock detection
@@ -141,7 +141,7 @@ The CLI `brainy inspect` subcommands all do this for you by default
   processes can both succeed at `init()` in writer mode and clobber each
   other's writes. A best-effort warning is logged in writer mode against a
   non-filesystem backend.
-- **Long-running readers** do not automatically pick up new Cortex segments
+- **Long-running readers** do not automatically pick up new Cor segments
   the writer publishes. One-shot inspector calls re-open the store and see
   fresh segments; a reader that stays open for hours sees its column store
   as-of the time it opened.
@@ -150,4 +150,4 @@ The CLI `brainy inspect` subcommands all do this for you by default
 
 - `Brainy.openReadOnly()` — [API reference](../api/brainy.md)
 - `brainy inspect` — [inspection guide](../guides/inspection.md)
-- Cortex columnar storage — see `node_modules/@soulcraft/cortex/README.md`
+- Cor columnar storage — see `node_modules/@soulcraft/cor/README.md`

docs/concepts/storage-adapters.md

@@ -33,7 +33,7 @@ BaseStorage                (type-statistics, lifecycle helpers, generation
 FileSystemStorage          (real filesystem I/O, writer-lock implementation,
                             flush-request watcher, atomic writes)
         ↑
-<your plugin's storage>    (Cortex's MmapFileSystemStorage, etc.)
+<your plugin's storage>    (Cor's MmapFileSystemStorage, etc.)
 ```
 
 When you `extend FileSystemStorage`, your adapter inherits every method on

docs/eli5.md

@@ -11,7 +11,7 @@ next:
   - getting-started/quick-start
 ---
 
-# Brainy and Cortex — Explained Simply
+# Brainy and Cor — Explained Simply
 
 *A plain-language guide for anyone who wants to understand what this thing actually does.*
 
@@ -65,13 +65,13 @@ Brainy can narrow any result set down by exact labels or ranges in the same brea
 
 ---
 
-## What is Cortex?
+## What is Cor?
 
-Cortex is a turbocharger for Brainy.
+Cor is a turbocharger for Brainy.
 
 Same car. Same controls. Same fuel. You just swap in a faster engine under the hood, and everything that used to take a moment now happens instantly.
 
-Technically, Cortex is an optional plugin written in Rust — a lower-level language that runs much closer to the raw metal of your processor. It plugs into Brainy and takes over the most compute-intensive work: the distance calculations that power meaning search, the number-crunching behind live aggregates, and the set operations that drive label filtering.
+Technically, Cor is an optional plugin written in Rust — a lower-level language that runs much closer to the raw metal of your processor. It plugs into Brainy and takes over the most compute-intensive work: the distance calculations that power meaning search, the number-crunching behind live aggregates, and the set operations that drive label filtering.
 
 You install it with one line, register it with one call, and Brainy automatically uses it everywhere it can help.
 
@@ -83,9 +83,9 @@ Plain language:
 
 - **Searches** go from "the blink of an eye" to "faster than a blink." The overall speedup is **5.2× on average** across all operations.
 - **Live aggregates** are rebuilt using all CPU cores in parallel, so re-indexing large datasets takes a fraction of the time.
-- **Analytics** that aren't even possible in pure JavaScript — real-time anomaly detection, streaming percentile estimates, approximate unique counts — become available because Cortex brings the native capabilities required to run them efficiently.
+- **Analytics** that aren't even possible in pure JavaScript — real-time anomaly detection, streaming percentile estimates, approximate unique counts — become available because Cor brings the native capabilities required to run them efficiently.
 
-If Brainy is what makes knowledge fast, Cortex is what makes Brainy feel instant.
+If Brainy is what makes knowledge fast, Cor is what makes Brainy feel instant.
 
 ---
 
@@ -135,7 +135,7 @@ Brainy is the only row with every box checked. And it runs all of them in a sing
 
 Brainy scales from a quick experiment to serious production datasets without changing a line of code. Small datasets live entirely in memory. Larger ones spill to disk, where Brainy shards and compresses everything automatically. Need a backup or a copy? Snapshots are instant — the same API the whole way.
 
-Add Cortex and you also unlock memory-mapped storage — aggregate state lives directly in the operating system's memory with zero serialization overhead, as fast as the hardware allows.
+Add Cor and you also unlock memory-mapped storage — aggregate state lives directly in the operating system's memory with zero serialization overhead, as fast as the hardware allows.
 
 ---
 

docs/guides/aggregation.md

@@ -11,7 +11,13 @@ No batch jobs. No scheduled recalculations. Aggregates stay current with every w
 **Defining over existing data:** if you define an aggregate on a store that already holds
 matching entities, Brainy backfills it from those entities on the first query (a one-time scan,
 then purely incremental). So `defineAggregate()` behaves the same whether you define it before
-or after the data exists — including when a persisted brain reopens already populated.
+or after the data exists.
+
+**Reopening a persisted brain:** aggregate state persists across restarts. Re-defining the
+same aggregate at boot (the normal declarative pattern) adopts the persisted state directly —
+no rescan. A backfill scan runs only when the definition actually changed, when no persisted
+state exists, or when the state failed to load; and however many aggregates need backfilling,
+they share a single scan.
 
 ## Quick Start
 
@@ -487,7 +493,7 @@ Aggregate definitions and running state are automatically persisted:
 
 ## Native Acceleration
 
-When [Cortex](https://github.com/soulcraftlabs/cortex) is installed as a plugin, the aggregation engine automatically uses Rust-accelerated computation:
+When [Cor](https://www.npmjs.com/package/@soulcraft/cor) is installed as a plugin, the aggregation engine automatically uses Rust-accelerated computation:
 
 - Incremental updates run in Rust with BTreeMap-backed precise MIN/MAX
 - Welford's online stddev/variance computed natively
@@ -496,7 +502,7 @@ When [Cortex](https://github.com/soulcraftlabs/cortex) is installed as a plugin,
 
 ```typescript
 const brain = new Brainy({
-  plugins: ['@soulcraft/cortex']
+  plugins: ['@soulcraft/cor']
 })
 await brain.init()
 
@@ -577,7 +583,7 @@ brain.defineAggregate({
 
 Aggregation complexity per write is O(A x G x M) where A = matching aggregates, G = groupBy dimensions, M = metrics. For typical configurations (2-5 aggregates, 1-3 dimensions, 3-5 metrics), this is effectively O(1).
 
-With Cortex native acceleration:
+With Cor native acceleration:
 
 | Operation | Throughput | Latency |
 |-----------|-----------|---------|

docs/guides/enterprise-for-everyone.md

@@ -174,13 +174,13 @@ await brain.syncWith({
 const brain = new Brainy()
 
 // 1 → ~1M vectors: pure-JS HNSW, zero extra setup
-// 1M → 10B+ vectors: install @soulcraft/cortex for the native DiskANN provider
+// 1M → 10B+ vectors: install @soulcraft/cor for the native DiskANN provider
 ```
 
 **Scaling model:**
 - **Single process, no cluster**: Brainy runs in one process — no coordinator,
   no peer discovery, no consensus to operate
-- **Optional native provider**: install `@soulcraft/cortex` to back the index with
+- **Optional native provider**: install `@soulcraft/cor` to back the index with
   on-disk DiskANN that scales to 10B+ vectors on one machine
 - **Per-tenant pools**: isolate tenants by giving each its own Brainy instance and
   storage directory

docs/guides/external-backups-and-sparse-storage.md

@@ -0,0 +1,99 @@
+---
+title: External Backups & Sparse Storage
+slug: guides/external-backups
+public: true
+category: guides
+template: guide
+order: 10
+description: How to back up a brain directory with external tools (tar, rsync, cp) without exploding sparse files — why a store can show 100+ GB "apparent" size on a small disk, which files are sparse, and how persist()/restore() handle it for you.
+next:
+  - guides/snapshots-and-time-travel
+  - concepts/storage-adapters
+---
+
+# External Backups & Sparse Storage
+
+The built-in snapshot path — [`db.persist()` and `brain.restore()`](/docs/guides/snapshots-and-time-travel) —
+already handles everything on this page for you. Read this when you back up a brain directory with
+**external tools**: `tar`, `rsync`, `cp`, `scp`, or a filesystem-level backup agent.
+
+## The one-sentence rule
+
+> **Always use the sparse-aware flag**: `tar czSf` (capital `S`), `rsync --sparse`,
+> `cp --sparse=always`. A naive copy can turn a 2 GB store into a 100+ GB one — or fail
+> the disk entirely.
+
+## Why: some files are sparse
+
+When a native accelerator plugin is active, parts of the index live in **memory-mapped files**
+created at a large fixed virtual size — the file's *apparent* size — while the filesystem only
+allocates blocks that were actually written. A brand-new id-mapper file can report tens of
+gigabytes in `ls -l` while occupying a few megabytes on disk.
+
+Check the difference yourself:
+
+```bash
+ls -lh  brain-data/_id_mapper/    # APPARENT size (can be huge)
+du -sh  brain-data/               # ALLOCATED size (the real footprint)
+```
+
+The sparse candidates in a brain directory:
+
+| Path | What it is |
+|---|---|
+| `_id_mapper/` | The native id-mapper's mmap files (large fixed virtual size) |
+| `_blobs/` | Native index files (vector base, segments) — may be mmap-backed |
+
+Everything else (entities, `_system`, `_generations`, `_cas` content blobs) is ordinary dense data.
+
+## Doing it right
+
+**tar** — the `S` flag detects holes and stores only real data:
+
+```bash
+tar czSf brain-backup.tgz /data/brain
+# restore preserves the holes:
+tar xzSf brain-backup.tgz -C /data/
+```
+
+**rsync**:
+
+```bash
+rsync -a --sparse /data/brain/ backup-host:/backups/brain/
+```
+
+**cp**:
+
+```bash
+cp -a --sparse=always /data/brain /backups/brain
+```
+
+**What goes wrong without the flag:** the copy *materializes* every hole as real zero bytes.
+A store whose apparent size exceeds the target disk fails with `ENOSPC` partway through — and a
+copy that *does* fit silently costs the full apparent size in storage and transfer time.
+
+## What the built-in paths do (so you don't have to)
+
+- **`db.persist(path)`** snapshots via **hard links** — instant and space-shared, since every data
+  file is immutable-by-rename. The handful of append-in-place files (the transaction log, the
+  commit fact log's tail segment) and mmap-mutated directories (`_id_mapper/`) are **byte-copied**
+  instead, so a post-snapshot write can never reach through a shared inode into your backup.
+- **`brain.restore(path, { confirm: true })`** is **non-destructive and sparse-aware**: the snapshot
+  is copied into a staging area *before* any live data is touched (all-zero blocks stay holes), and
+  only after the copy fully succeeds does an atomic swap move it into place. A failed copy —
+  including `ENOSPC` — leaves the live store exactly as it was. A crash mid-swap completes forward
+  on the next open.
+
+## Live-store caveats for external tools
+
+1. **Prefer snapshotting a `persist()` output, not the live directory.** `persist()` produces a
+   crash-consistent, immutable snapshot; running `tar` against a live, actively-written directory
+   can capture a torn mid-write state. If you must archive live, stop writes first (or accept that
+   the archive is only as consistent as the moment's flush state).
+2. **Never prune or "clean up" files inside a brain directory.** Index files that look stale or
+   redundant are load-bearing; the store protects its declared index families from in-process
+   deletion, but an external `rm` bypasses that fence. If space is the concern, `du -sh` first —
+   the allocated size is usually far smaller than it looks.
+3. **Verify restores by opening them.** `Brainy.load(path)` opens any snapshot or restored
+   directory read-only — the store verifies its own coherence at open and reports loudly if
+   anything is missing or torn.

docs/guides/find-limits.md

@@ -40,6 +40,10 @@ Brainy picks `maxLimit` from the first of these that's available:
 
 Worked example: a 4 GB Cloud Run container picks priority 3 → `floor(4 GB × 0.25 / 25 KB) = floor(40 960) = 40 000` results. A 900 MB free-memory box on priority 4 gets `floor(900 MB / 25 KB) = ~36 000`.
 
+The cap is fixed at construction and never changes at runtime. Query timing is recorded
+for diagnostics only — a burst of slow queries cannot silently shrink the cap, and the
+auto-detected tiers (3 and 4) never go below a floor of 10 000.
+
 > **Calibration note.** Pre-7.30.2 used 100 KB per result instead of 25 KB, which produced caps that were 4× too tight for typical workloads (an 8 KB / result reality). 7.30.2 recalibrated to match observed entity sizes; existing `limit: 10_000` safety patterns now pass silently on any reasonably-sized box.
 
 ## What happens when you exceed the cap

docs/guides/import-anything.md

@@ -300,7 +300,10 @@ await brain.import(data, {
  // Deduplication
  enableDeduplication: true, // Check for duplicate entities (default: true)
  deduplicationThreshold: 0.85, // Similarity threshold for duplicates (0-1, default: 0.85)
- // Note: Auto-disabled for imports >100 entities
+ // Notes: false disables BOTH the inline merge and the background pass that
+ // runs ~5 min after the last import (merged duplicates are deleted).
+ // The inline pass auto-disables for imports >100 entities (O(n²) cost);
+ // the background pass still covers those unless the flag is false.
 
  // Performance
  chunkSize: 100, // Batch size for processing (default: varies by operation)

docs/guides/import-quick-reference.md

@@ -86,11 +86,22 @@ await brain.import(file, {
 
 ```typescript
 await brain.import(file, {
-  enableDeduplication: true,         // Check for duplicates (default: false)
+  enableDeduplication: true,         // Check for duplicates (default: true)
   deduplicationThreshold: 0.85       // Similarity threshold (default: 0.85)
 })
 ```
 
+Deduplication merges entities judged duplicates — the non-primary records are
+**deleted**. Set `enableDeduplication: false` to disable it entirely: the flag
+gates both the inline merge during import and the background pass that runs
+about 5 minutes after the last import.
+
+```typescript
+await brain.import(file, {
+  enableDeduplication: false         // No merging, inline or background
+})
+```
+
 ### Import Tracking
 
 Track and organize imports by project:

docs/guides/inspection.md

@@ -166,6 +166,34 @@ brainy inspect diff /data/brain-prod /data/brain-staging
 Sample-based — for a full diff, dump both with `inspect dump` and compare
 the JSONL.
 
+## Auditing graph-read truth
+
+`brain.auditGraph()` (8.6.0+) proves — or disproves — that relationship reads
+return canonical truth on a given brain, without mutating anything. It walks
+every stored relationship record, asks the same read path your application
+uses (`related()`, VFS `readdir`) with every visibility tier included, and
+classifies every discrepancy:
+
+```typescript
+const report = await brain.auditGraph()
+
+report.coherent              // true = related()/readdir can be trusted on this brain
+report.missingFromReadsCount // records the read path omits — stale index
+report.danglingEndpointsCount // relationships whose endpoint entity is gone
+report.readOnlyCount         // read-path edges with NO stored record — ghosts
+report.visibilityHiddenCount // internal/system edges hidden by design (not a fault)
+```
+
+Counts are always exact; the example lists (`missingFromReads`,
+`danglingEndpoints`, `readOnlyVerbIds`) are capped at `maxExamples`
+(default 100) and `truncatedExamples` says so when they are.
+
+Run it after any engine upgrade, restore, or migration. If it reports
+discrepancies, run `brain.repairIndex()` and audit again — a `coherent`
+report after the repair is the verified statement that the heal worked.
+Cost: one relationship-record walk plus one indexed read per distinct
+source entity — safe on a live brain.
+
 ## Repairing a corrupted store
 
 If invariants fail and you suspect index corruption, `inspect repair`

docs/guides/installation.md

@@ -45,20 +45,20 @@ console.log('Brainy ready.')
 
 ## Native Acceleration (Optional)
 
-For production workloads, add Cortex for Rust-accelerated SIMD distance calculations and native embeddings:
+For production workloads, add Cor for Rust-accelerated SIMD distance calculations and native embeddings:
 
 ```bash
-npm install @soulcraft/cortex
+npm install @soulcraft/cor
 ```
 
 ```typescript
 import { Brainy } from '@soulcraft/brainy'
 
-const brain = new Brainy({ plugins: ['@soulcraft/cortex'] })
+const brain = new Brainy({ plugins: ['@soulcraft/cor'] })
 await brain.init()  // native providers registered during init
 ```
 
-Cortex registers native (Rust/SIMD) vector, metadata, and graph engines behind the same Brainy `find()` API — no code changes, an optional dependency for production-scale workloads.
+Cor registers native (Rust/SIMD) vector, metadata, and graph engines behind the same Brainy `find()` API — no code changes, an optional dependency for production-scale workloads.
 
 ## Server-only since 8.0
 

docs/guides/optimistic-concurrency.md

@@ -180,3 +180,35 @@ Brainy 8.0 has exactly two write-coordination counters, at two granularities:
 They compose: a `transact()` batch can carry per-entity `ifRev` checks *and* a whole-store `ifAtGeneration`; any failed check rejects the entire batch before anything is staged. Generations also power snapshots and time travel (`brain.now()`, `brain.asOf()`, `db.persist()`) — see the [consistency model](../concepts/consistency-model.md) and [Snapshots & Time Travel](./snapshots-and-time-travel.md).
 
 A snapshot or historical view captures each entity *including* its `_rev` at that moment, so reading the past and writing back with `ifRev` against the live state works exactly as you'd hope: the write fails if the entity moved since the state you copied from.
+
+## The transact envelope: batch size, budget, and bulk imports
+
+`transact()` applies its batch atomically under one commit — which means the whole batch
+shares one **apply budget**. Since 8.7.0 the budget scales with the batch:
+`max(30 s, opCount × 2 s)`, or exactly what you pass as `timeoutMs`. A tripped budget rolls
+the entire batch back (nothing partial survives) and throws a retryable
+`TransactionTimeoutError` that names the operation it stopped at, the batch size, and the
+elapsed vs budgeted time — a diagnosis, not just a failure:
+
+```
+Transaction timed out at operation 41/120 ('add') — 246012ms elapsed, budget 240000ms.
+The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.
+```
+
+Practical envelope guidance for bulk work:
+
+1. **Precompute embeddings outside the commit path.** Embedding inside `transact()` spends
+   the budget on model inference. Use `brain.embedBatch(texts)` and pass each vector via
+   the op's `vector` field — the commit then pays only storage costs, and a retried batch
+   never re-pays inference. (The win is *where* the inference happens, not raw embedding
+   throughput: on the default WASM engine, batch and sequential embedding measure
+   comparably, ~160 ms/text; native embedding providers may batch faster.)
+2. **Chunk very large imports** into batches of a few hundred ops with one `transact()`
+   each. You lose whole-import atomicity but keep per-chunk atomicity, bounded memory, and
+   resumability — pair with `ifAbsent` upserts so a retried chunk is idempotent.
+3. **Slow disks change the math, not the contract.** On network-attached storage a single
+   op can cost ~2 s (canonical write + fsync + index maintenance). The scaled default
+   absorbs that; pass an explicit `timeoutMs` only when you know better than the scale.
+4. **`addMany`/`relateMany` are the convenience tier** — they chunk and batch-embed for
+   you, with per-item error reporting instead of batch atomicity. Choose by what you need:
+   atomic-all-or-nothing → `transact()`; resilient bulk load → `addMany`.

docs/guides/reacting-to-changes.md

@@ -0,0 +1,117 @@
+---
+title: Reacting to Changes
+slug: guides/reacting-to-changes
+public: true
+category: guides
+template: guide
+order: 11
+description: Subscribe to every committed mutation with `brain.onChange` — the in-process change feed behind live UIs, cache invalidation, and realtime sync. Covers the event shape, delivery guarantees, and catch-up patterns.
+next:
+  - guides/optimistic-concurrency
+  - guides/snapshots-and-time-travel
+---
+
+# Reacting to Changes
+
+`brain.onChange(cb)` is Brainy's in-process change feed: subscribe once and
+receive one event per committed mutation — **every** mutation, regardless of
+how it happened. Direct calls, batch methods, `transact()`, imports, and
+Virtual Filesystem writes all funnel through the same commit point the feed is
+emitted from, so nothing slips past it.
+
+```ts
+const off = brain.onChange((e) => {
+  if (e.kind === 'entity') {
+    console.log(`${e.op} ${e.entity?.type} ${e.id} @ generation ${e.generation}`)
+  }
+})
+
+await brain.add({ data: 'Ada Lovelace', type: 'person' })
+// → "add person 0198... @ generation 42"
+
+off() // unsubscribe when done
+```
+
+## The event
+
+```ts
+interface BrainyChangeEvent {
+  kind: 'entity' | 'relation' | 'store'
+  op: 'add' | 'update' | 'remove' | 'relate' | 'unrelate' | 'updateRelation'
+    | 'clear' | 'restore'
+  id?: string
+  entity?:   { id: string; type: string; subtype?: string;
+               metadata: Record<string, unknown>; service?: string }
+  relation?: { id: string; from: string; to: string; type: string;
+               metadata?: Record<string, unknown> }
+  generation?: number
+  timestamp: number
+}
+```
+
+- **Entity events** (`add` / `update` / `remove`) carry the post-commit indexed
+  view — `type`, `subtype`, and the full custom `metadata`, so you can match
+  your own `where`-style filters against events without a read.
+- **Deletes are fully described.** A `remove` or `unrelate` event carries the
+  record's *last committed state* (sourced from the commit's own history
+  record), not just an id.
+- **Batches emit per item.** `addMany` / `updateMany` / `relateMany` /
+  `removeMany` emit one event per affected record; a `transact()` batch emits
+  one event per item, all sharing the batch's single `generation`.
+- **Cascades are visible.** Removing an entity also emits `unrelate` for each
+  relationship the delete cascaded to.
+- **Store-level events** (`kind: 'store'`) fire for the two wholesale
+  operations — `clear()` and `restore()` — and mean *"everything may have
+  changed; refetch what you care about."*
+
+## Delivery guarantees
+
+- **Post-commit only.** An aborted write — a losing
+  [`ifRev` compare-and-swap](optimistic-concurrency.md), a rejected
+  transaction — never emits. If you received the event, the write is durable.
+- **Commit-ordered.** Events arrive in the order writes committed;
+  `generation` is monotonic.
+- **Asynchronous, never blocking.** Delivery happens in a microtask after the
+  write completes. A slow listener cannot delay a write; a throwing listener
+  is logged and isolated from other listeners.
+- **Zero overhead when unused.** With no subscribers, the write path does no
+  event work at all.
+- **Fire-and-forget.** There is no replay or backpressure. For catch-up after
+  a disconnect, use the `generation` on each event together with
+  [`asOf()` / the transaction log](snapshots-and-time-travel.md): record the
+  last generation you processed, and on reconnect diff from there. For file
+  content specifically, `vfs.readFile(path, { asOf })` and
+  `vfs.history(path)` are the temporal read — see
+  [Snapshots & Time Travel](snapshots-and-time-travel.md).
+
+## Patterns
+
+**Cache invalidation** — drop cached reads for whatever changed:
+
+```ts
+brain.onChange((e) => {
+  if (e.kind === 'store') return cache.clear()
+  if (e.id) cache.delete(e.id)
+})
+```
+
+**Live queries (notify-and-refetch)** — re-run a query when a relevant change
+lands, rather than diffing incrementally:
+
+```ts
+brain.onChange((e) => {
+  if (e.kind === 'entity' && e.entity?.type === 'order') {
+    refreshOpenOrdersView() // debounce as needed
+  }
+})
+```
+
+**Forwarding to other processes** — the feed is in-process by design. To push
+changes to browsers or other services, forward events through your own
+transport (WebSocket, SSE) from the process that owns the brain.
+
+## Lifecycle
+
+`onChange` returns an unsubscribe function — call it when tearing down a
+subscriber (for example, when evicting a pooled instance). `brain.close()`
+drops all listeners; no events are delivered for or after `close()`.

docs/guides/snapshots-and-time-travel.md

@@ -9,6 +9,7 @@ description: Recipes for the Db API — instant backups with persist(), restore,
 next:
   - concepts/consistency-model
   - guides/optimistic-concurrency
+  - guides/external-backups
 ---
 
 # Snapshots & Time Travel
@@ -41,6 +42,10 @@ bytes. Cross-device targets fall back to per-file byte copies, and
 persisting an in-memory brain serializes it to the same directory layout —
 a real, durable store.
 
+> Archiving a brain directory with **external tools** (`tar`, `rsync`, `cp`)?
+> Some index files are sparse and can explode to their apparent size under a
+> naive copy — see [External Backups & Sparse Storage](/docs/guides/external-backups).
+
 Two things to know:
 
 - `persist()` requires the view to still be the store's **latest**
@@ -339,8 +344,12 @@ For per-entity write coordination (rather than whole-store history), the
 ## Keeping history bounded
 
 Under Model-B every write is a generation, so history can grow quickly —
-Brainy auto-compacts on every `flush()`/`close()` under the **`retention`**
-knob (configured on the constructor):
+Brainy auto-compacts at `close()` (time-bounded per pass) under the
+**`retention`** knob (configured on the constructor). Since 8.9.0, `flush()`
+never compacts: flushing is durability work and costs only what the current
+window's writes cost, regardless of history backlog. A long-lived writer that
+never closes keeps its history until its next explicit `compactHistory()` —
+schedule one in your maintenance window if you run bounded retention:
 
 ```typescript
 // Zero-config: ADAPTIVE — keep as much history as free disk/RAM allows,
@@ -354,10 +363,13 @@ new Brainy({ retention: 'all' })
 new Brainy({ retention: { maxGenerations: 1000, maxAge: 7 * 86_400_000, maxBytes: 512 * 1024 ** 2 } })
 ```
 
-Reclaim manually at any time (the same caps):
+Reclaim manually at any time (the same caps, plus an optional per-pass time
+budget for maintenance windows — an early stop is a consistent prefix and the
+next pass resumes):
 
 ```typescript
 await brain.compactHistory({ maxGenerations: 100, maxAge: 7 * 24 * 60 * 60 * 1000 })
+await brain.compactHistory({ maxBytes: 512 * 1024 ** 2, timeBudgetMs: 10_000 })
 ```
 
 Compaction never breaks a pinned read — record-sets are reclaimed only when
@@ -366,6 +378,61 @@ are done with (including the ones `transact()` returns), and `persist()` any
 generation you want to keep beyond the retention window: snapshots are
 self-contained and unaffected by compaction.
 
+## Time travel for files (the VFS)
+
+Since 8.2.0, time travel covers Virtual Filesystem **content**, not just
+entity records. File bytes are retention-protected: a content blob referenced
+by any generation inside the retention window is never reclaimed, so reading
+the past always returns the exact bytes — never a stale field or a
+dangling hash.
+
+**`vfs.readFile(path, { asOf })`** takes a generation number or a `Date` and
+returns the file's exact bytes as they stood then. It resolves the path's
+current entity, then materializes its state at the target generation — so it
+answers *"what did the file at this path hold at that point?"* It bypasses
+the content cache; the `encoding` option still applies. Asking about a
+generation before the file existed throws the usual not-found error, and
+asking past the retention window's compaction horizon throws a
+compacted-generation error.
+
+**`vfs.history(path)`** returns the file's versions inside the retention
+window, oldest first — one `FileVersion` per generation that wrote the file,
+the newest entry being the current state:
+
+```typescript
+// A CMS page evolves…
+await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch"}')
+await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch v2"}')
+await brain.vfs.writeFile('/pages/home.json', '{"title":""}') // bad deploy!
+
+// Every version is listed and readable:
+const versions = await brain.vfs.history('/pages/home.json')
+// → [{ generation, timestamp, hash, size, mimeType? }, …] ascending
+
+const good = versions[versions.length - 2]
+const bytes = await brain.vfs.readFile('/pages/home.json', {
+  asOf: good.generation
+})
+
+// Restore = write the old bytes back. This is a NEW write (a new
+// generation) — history is never rewritten, so the bad version stays
+// visible in the audit trail.
+await brain.vfs.writeFile('/pages/home.json', bytes)
+```
+
+Two lifecycle consequences worth stating plainly:
+
+- **Deleting or overwriting a file no longer frees its bytes immediately.**
+  Old content lives until history compaction reclaims the generations that
+  reference it — the same `retention` budget that bounds all Model-B history
+  (and pinned views are exempt, exactly as above). Size your `retention` for
+  the file-version depth you want; `retention: 'all'` keeps every version of
+  every file forever.
+- **After `compactHistory()` reclaims a generation, its file versions are
+  gone** and their bytes are physically reclaimed. (This also fixed a
+  pre-8.2.0 defect where overwritten content was never reclaimed at all — an
+  unbounded silent leak.)
+
 ## From branches to values
 
 If you used the pre-8.0 `fork`/`checkout`/`commit`/`versions` surface, every

docs/guides/subtypes-and-facets.md

@@ -332,7 +332,7 @@ await brain.updateRelation({ id: relationId, weight: 0.5, confidence: 0.9 })
 `find({ connected, subtype })` filters traversal edges by their subtype. Composes with `via` (verb-type filter):
 
 ```typescript
-// All direct reports two hops deep (will be supported in Cortex; for now depth-1)
+// All direct reports two hops deep (will be supported in Cor; for now depth-1)
 const directChain = await brain.find({
   connected: {
     from: ceoId,
@@ -343,7 +343,7 @@ const directChain = await brain.find({
 })
 ```
 
-Multi-hop subtype filtering (`depth > 1`) lights up on the Cortex native path; the JS path throws today rather than return incorrect partial results.
+Multi-hop subtype filtering (`depth > 1`) lights up on the Cor native path; the JS path throws today rather than return incorrect partial results.
 
 ### Counts
 

docs/guides/upgrading-7-to-8.md

@@ -0,0 +1,186 @@
+---
+title: Upgrading from 7.x to 8.0
+slug: guides/upgrading-7-to-8
+public: true
+category: guides
+template: guide
+order: 10
+description: What the one-time 7→8 on-disk migration does, how 8.0 automatically recovers Virtual Filesystem content that older layouts stored in the removed copy-on-write area, and how to verify (or force) that recovery.
+next:
+  - guides/storage-adapters
+  - guides/inspection
+---
+
+# Upgrading from 7.x to 8.0
+
+Opening a 7.x on-disk store with Brainy 8.0 runs a **one-time, in-place layout
+migration** the first time the store is opened. It is automatic (`autoMigrate`
+defaults to `true`), it runs once, and it stamps a marker so every later open is
+a no-op.
+
+Almost everything about the upgrade is transparent: your entities, relationships,
+metadata, and indexes migrate and rebuild without any action on your part. This
+guide covers the **one case that needs attention** — Virtual Filesystem (VFS)
+content — and how 8.0 recovers it for you.
+
+## TL;DR
+
+- **Just upgrade to `@soulcraft/brainy@8.0.12` (or later) and open the store.**
+  If a previous upgrade left VFS content stranded, 8.0.12 **heals it on open**,
+  with no operator action.
+- Want to force or script it? Call **`await brain.vfs.adoptOrphanedBlobs()`**.
+- The recovery is **non-destructive and idempotent** — it copies, never moves,
+  and running it twice is a no-op.
+
+## What the migration does
+
+The 7.x on-disk layout stored entities under a per-branch path
+(`branches/<head>/entities/...`). 8.0 uses a flat layout (`entities/...`). On
+first open, Brainy:
+
+1. Collapses `branches/<head>/entities/*` into the flat `entities/*` layout.
+2. Rebuilds the derived indexes (vector, graph, metadata) and count rollups from
+   the canonical entities.
+3. Stamps `_system/migration-layout.json` so re-opening is a no-op.
+4. Takes an automatic **pre-upgrade backup** of the directory before it starts,
+   and removes it once the upgrade is verified complete (see
+   [The safety net](#the-safety-net-pre-upgrade-backup) below).
+
+The log line to expect (once per store):
+
+```
+[brainy] Migrating a 7.x branch layout (branches/main) to the 8.0 flat layout
+  in place — 13175 entity files. This runs once; back up the directory first if
+  you need a rollback (8.0 does not keep the old layout).
+```
+
+## The one thing that needs recovery: VFS content
+
+If your application uses the Virtual Filesystem (VFS) to store file content (for
+example, a CMS that keeps page documents at paths like
+`/pages/homepage/page.json`), that content is held as **content blobs**.
+
+7.x kept those blobs in the branch system's **copy-on-write area** (`_cow/`).
+8.0 removed the branch/copy-on-write system, and its content blobs live in the
+content-addressed store (`_cas/`). The layout migration moves entities — it does
+**not** move the VFS content blobs. Left alone, a 7.x store's VFS blobs would
+remain in `_cow/`, and a read of one would throw:
+
+```
+VFS: Cannot read blob for /pages/homepage/page.json:
+  Blob metadata not found: 6b87cfb71ad2f04602a5c157214dc42000...
+```
+
+### 8.0.12 recovers them automatically
+
+Brainy 8.0.12 adds an **on-open recovery pass** that runs right after the layout
+migration. It scans `_cow/`, and for every content blob whose `blob:` +
+`blob-meta:` pair is not already in `_cas/`, it copies **both** across into the
+8.0 store. It:
+
+- **Heals a fresh 7→8 upgrade** (where `_cow/` still holds the blobs), **and**
+- **Heals a store already upgraded by an earlier 8.0.x** that stranded them —
+  the recovery is gated on the presence of `_cow/` and its own marker, not on
+  the layout-migration marker, so an already-migrated store is still healed.
+- Is a **cheap no-op** on a native-8.0 or fresh store (there is no `_cow/`), and
+  on a store already healed (a marker records completion so later opens skip the
+  scan).
+
+So the operator action for a stranded store is simply: **upgrade to 8.0.12 and
+open it.**
+
+```ts
+import { Brainy } from '@soulcraft/brainy'
+
+// Opening the store is all that is required — recovery runs during init().
+const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/my-store' } })
+await brain.init()
+
+// The previously-failing read now succeeds.
+const page = await brain.vfs.readFile('/pages/homepage/page.json')
+```
+
+On a store that needed recovery you will see:
+
+```
+[brainy] Recovered 337 VFS content blob(s) stranded by a 7→8 upgrade
+  (adopted _cow/ → _cas/ in place).
+```
+
+### Forcing recovery explicitly
+
+If you would rather run the recovery deliberately (for example, in an upgrade
+script that asserts a clean result before flipping traffic), call it directly:
+
+```ts
+const result = await brain.vfs.adoptOrphanedBlobs()
+// → { cowBlobs, adopted, alreadyPresent, incomplete }
+console.log(`adopted ${result.adopted}, already present ${result.alreadyPresent}`)
+if (result.incomplete > 0) {
+  // One or more _cow/ blobs are missing their bytes or metadata — investigate
+  // _cow/ before discarding your own backup. See "The safety net" below.
+}
+```
+
+`adoptOrphanedBlobs()` self-initializes the brain, so it is safe to call
+immediately after construction. It returns:
+
+| Field            | Meaning                                                          |
+| ---------------- | ---------------------------------------------------------------- |
+| `cowBlobs`       | Distinct content-blob hashes found in `_cow/`.                   |
+| `adopted`        | Newly copied into `_cas/` on this call.                          |
+| `alreadyPresent` | Already in `_cas/` (a prior open or run adopted them).           |
+| `incomplete`     | A `_cow/` blob missing its bytes *or* its metadata — **skipped** rather than half-adopted. |
+
+## Verifying recovery
+
+After opening under 8.0.12, confirm a previously-failing path reads:
+
+```ts
+const content = await brain.vfs.readFile('/pages/homepage/page.json')
+console.log(content.toString().slice(0, 80))
+```
+
+If you scripted it with `adoptOrphanedBlobs()`, a clean result is
+`incomplete === 0` and `adopted + alreadyPresent === cowBlobs`.
+
+## The safety net: pre-upgrade backup
+
+Brainy takes an automatic pre-upgrade backup and removes it once the upgrade is
+**verified complete**. In 8.0.12 that verification includes VFS content: if the
+blob recovery reports `incomplete > 0`, the completion marker is **not** stamped
+(the next open retries) and the **pre-upgrade backup is retained** so you still
+have a rollback while blobs remain unaccounted for. You will see:
+
+```
+[brainy] VFS blob recovery adopted N blob(s) but M orphaned _cow/ blob(s) are
+  missing their bytes or metadata and were left in place. The pre-upgrade backup
+  is being retained; inspect _cow/ before discarding it.
+```
+
+The recovery never deletes anything from `_cow/`, so the original blobs stay in
+place for inspection or a manual rollback regardless.
+
+## Who is affected
+
+**Any 7.x store that used the VFS to store file content** (so it has a `_cow/`
+area) is a candidate for stranded blobs on a 7→8 upgrade. Stores that never used
+the VFS have no `_cow/` content blobs and are unaffected.
+
+Because the recovery is **gated on the presence of `_cow/`**, it is
+self-selecting and safe to roll out everywhere:
+
+- On a store with stranded blobs → it adopts them.
+- On a native-8.0, fresh, or non-VFS store → it is a no-op on the existence
+  check.
+
+There is no configuration to set and nothing to opt into. Upgrading to 8.0.12
+and opening each store is sufficient.
+
+## Rollback
+
+The recovery is copy-only, so no rollback of the recovery itself is ever needed.
+If you need to roll back the **whole** 7→8 upgrade, restore the directory from
+your pre-upgrade backup (retained automatically while recovery is incomplete, or
+your own snapshot) and pin `@soulcraft/brainy@7.x`. 8.0 does not keep the old
+branch layout in place, so a directory-level restore is the rollback path.

docs/performance-envelopes.md

@@ -0,0 +1,83 @@
+---
+title: Performance Envelopes
+slug: guides/performance-envelopes
+public: true
+category: guides
+template: guide
+order: 40
+description: Measured per-operation latency envelopes at stated scales — what to expect, on what hardware, and exactly how each number was produced.
+next:
+  - guides/find-limits
+---
+
+# Performance Envelopes
+
+Every number on this page is **measured, never projected** — produced by the script
+cited at the bottom, against the built package (the artifact you install), on the stated
+hardware. Each entry says what was measured, at what scale, on which storage backend.
+When a release touches a measured path, that operation is re-measured and this page
+updates in the same release.
+
+Two scopes to keep straight:
+
+- **These envelopes are the pure-JS engine** (no native accelerator registered) on
+  filesystem storage. This is the floor every deployment gets from `npm install` alone.
+- **Accelerated deployments** (the optional native provider) publish their own numbers —
+  this page never claims them.
+
+## Read operations
+
+Reads are where the architecture pays off: after the write path has done its indexing
+work, queries answer from purpose-built indexes without scanning.
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `get(id)` (warm) | p50 < 0.1ms | p50 < 0.1ms | served from cache/metadata index |
+| `find` (metadata: indexed equality + range, limit 100) | p50 1.0ms · p95 1.8ms | p50 7.0ms · p95 8.9ms | column-store bitmap paths |
+| `related(id)` (per-node adjacency) | p50 < 0.1ms · p95 0.2ms | p50 < 0.1ms | LSM adjacency index — O(degree), scale-independent |
+| `find` (semantic: embed + HNSW, 1k docs) | p50 178ms · p95 393ms | — | dominated by WASM query embedding (measured on a machine under concurrent load — treat the p95 as an upper bound); the vector search itself is single-digit ms |
+
+## Write operations
+
+Under Model-B **every write is its own durable generation** — a single-op `add` pays
+serialization, before-image staging, and fsync before it acks. That durability is priced
+into the write path visibly, by design:
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `add` (single-op) | p50 167ms · p95 171ms | p50 165ms · p95 172ms | full durable generation per write — flat across scale |
+| `addMany` (bulk) | ~163ms/entity | ~187ms/entity | **currently per-item commits** — see the honest note below |
+| `relateMany` | ~0.8ms/edge | ~0.9ms/edge | edges batch efficiently today |
+| `flush` (steady-state, 1 pending write) | p50 8ms · p95 10ms | p50 45ms · p95 52ms | durability-only since 8.9.0 — cost no longer depends on history backlog or retention mode |
+
+**The honest note on bulk writes:** `addMany` today commits each item as its own
+generation (the same durability as single-op `add`, serialized by the single-writer
+lock), so bulk-load cost is N × single-op cost. Batched chunk commits (one generation
+and one fsync window per chunk, as `removeMany` already does) are designed into the
+unified-commit work on the current roadmap. Until that ships, size bulk imports
+accordingly — 10k entities is minutes, not seconds, on filesystem storage.
+
+## Open / close
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `open` (empty store) | ~560ms | ~190ms | includes embedder initialization |
+| `open` (warm, populated, clean shutdown) | 763ms | 4.9s | pure-JS vector index load dominates and grows with entity count; the native accelerator exists precisely to remove this |
+| `close` | bounded | bounded | auto-compaction pass is time-bounded (~5s max) since 8.9.0 |
+
+A store that was NOT cleanly closed pays index rebuilds on top of the warm-open
+number (tens of seconds at 10k) — clean shutdown is worth engineering for.
+
+## How these were produced
+
+- **Hardware**: Intel Core i9-14900HX (32 threads), 62GB RAM, NVMe, Linux, Node v22.
+- **Backend**: `storage: { type: 'filesystem' }`, pure JS (no native providers).
+- **Embeddings**: deterministic stub for non-semantic ops (isolates engine cost);
+  the real WASM embedder for the semantic row (that's what you'll run).
+- **Method**: p50/p95 over 50–200 samples per op against the built `dist/`;
+  the measuring script ships in the repo history and re-runs per release.
+
+Numbers on different hardware will differ; the *shape* (sub-2ms indexed reads,
+~160ms embedding-bound semantic queries, durability-priced writes) is the envelope
+you should hold your deployment against. If your measurements diverge from these
+shapes by an order of magnitude, something is wrong — file it.

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.0.0-rc.9",
+  "version": "8.9.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "@soulcraft/brainy",
-      "version": "8.0.0-rc.9",
+      "version": "8.9.0",
       "license": "MIT",
       "dependencies": {
         "@msgpack/msgpack": "^3.1.2",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.0.0-rc.9",
+  "version": "8.9.0",
   "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
   "main": "dist/index.js",
   "module": "dist/index.js",

scripts/push-docs.js

@@ -0,0 +1,116 @@
+#!/usr/bin/env node
+/**
+ * @module scripts/push-docs
+ * @description Push this repo's PUBLIC docs to the soulcraft.com docs ingest
+ * door after an npm publish (VENUE-DOCS-RELEASE-PUSH — retires the old
+ * build-time docs sync).
+ *
+ * Contract (mirrors the reference implementation on the serving side):
+ *   POST {base}/api/docs/ingest
+ *   headers: x-service-secret: $DOCS_INGEST_SECRET, Content-Type: application/json
+ *   body: { docs: [{ slug, title, markdown, nav: { order, section } }] }
+ *   batches of 10, idempotent per slug.
+ *
+ * A doc is public iff its frontmatter has `public: true` AND a `slug`. The
+ * frontmatter is stripped; `category` → nav.section, `order` → nav.order.
+ *
+ * Deliberately NOT pushed: the combined /docs landing index. It spans BOTH
+ * engine corpora (this repo's and the native accelerator's), so a per-repo
+ * push would clobber the union — the index is authored on the serving side.
+ *
+ * Env: DOCS_INGEST_SECRET (required), DOCS_INGEST_BASE (default
+ * https://soulcraft.com). Exits 0 with a LOUD warning when the secret is
+ * absent (the npm publish has already happened; the serving side runs its
+ * interim sync on request) and exits 1 when a push actually fails — the docs
+ * site would silently trail npm otherwise, and that must be visible.
+ */
+import * as fs from 'node:fs'
+import * as path from 'node:path'
+
+const BASE = (process.env.DOCS_INGEST_BASE || 'https://soulcraft.com').replace(/\/+$/, '')
+const SECRET = process.env.DOCS_INGEST_SECRET
+const DOCS_DIR = path.join(path.dirname(new URL(import.meta.url).pathname), '..', 'docs')
+const BATCH = 10
+
+if (!SECRET) {
+  console.warn(
+    '⚠️  DOCS PUSH SKIPPED: DOCS_INGEST_SECRET is not set.\n' +
+      '    soulcraft.com/docs now TRAILS this npm release until docs are pushed.\n' +
+      '    Either export DOCS_INGEST_SECRET and re-run `node scripts/push-docs.js`,\n' +
+      '    or ping venue on VENUE-DOCS-RELEASE-PUSH for the interim sync.'
+  )
+  process.exit(0)
+}
+
+/** Minimal frontmatter split — returns [meta, body] or [null, raw]. */
+function parseFrontmatter(raw) {
+  const m = raw.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/)
+  if (!m) return [null, raw]
+  const meta = {}
+  for (const line of m[1].split('\n')) {
+    const kv = line.match(/^(\w[\w-]*):\s*(.*)$/)
+    if (kv) meta[kv[1]] = kv[2].trim().replace(/^["']|["']$/g, '')
+  }
+  return [meta, m[2]]
+}
+
+const docs = []
+;(function walk(dir) {
+  for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+    const full = path.join(dir, entry.name)
+    if (entry.isDirectory()) walk(full)
+    else if (entry.name.endsWith('.md')) {
+      const [meta, body] = parseFrontmatter(fs.readFileSync(full, 'utf-8'))
+      if (!meta || meta.public !== 'true' || !meta.slug) continue
+      docs.push({
+        slug: meta.slug,
+        title: meta.title || meta.slug,
+        markdown: body.trim(),
+        nav: {
+          order: Number.parseInt(meta.order || '99', 10) || 99,
+          section: meta.category || 'guides'
+        }
+      })
+    }
+  }
+})(DOCS_DIR)
+
+if (docs.length === 0) {
+  console.error('❌ DOCS PUSH FAILED: zero public docs collected — refusing to push an empty corpus.')
+  process.exit(1)
+}
+docs.sort((a, b) => a.slug.localeCompare(b.slug))
+console.log(`Pushing ${docs.length} public docs to ${BASE}/api/docs/ingest …`)
+
+let failed = false
+for (let i = 0; i < docs.length; i += BATCH) {
+  const batch = docs.slice(i, i + BATCH)
+  try {
+    const res = await fetch(`${BASE}/api/docs/ingest`, {
+      method: 'POST',
+      headers: {
+        'x-service-secret': SECRET,
+        'Content-Type': 'application/json',
+        'User-Agent': 'brainy-docs-push/1.0'
+      },
+      body: JSON.stringify({ docs: batch }),
+      signal: AbortSignal.timeout(120_000)
+    })
+    if (!res.ok) {
+      throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`)
+    }
+    console.log(`  batch ${i / BATCH + 1}: ${batch.map((d) => d.slug).join(', ')} → ok`)
+  } catch (err) {
+    failed = true
+    console.error(`  batch ${i / BATCH + 1} FAILED: ${err instanceof Error ? err.message : err}`)
+  }
+}
+
+if (failed) {
+  console.error(
+    '❌ DOCS PUSH INCOMPLETE — soulcraft.com/docs may trail npm. ' +
+      'Re-run `node scripts/push-docs.js` or ping venue on VENUE-DOCS-RELEASE-PUSH.'
+  )
+  process.exit(1)
+}
+console.log('✅ Docs pushed.')

scripts/release.sh

@@ -196,6 +196,18 @@ else
 fi
 echo -e "${GREEN}✅ GitHub release created${NC}\n"
 
+# Step 12: Push public docs to the soulcraft.com docs ingest door
+# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
+# DOCS_INGEST_SECRET is unset; fails loudly (without undoing the publish —
+# that already happened) when a push errors, so the docs site never
+# silently trails npm.
+echo -e "${BLUE}1️⃣2️⃣ Pushing public docs to soulcraft.com/docs...${NC}"
+if node scripts/push-docs.js; then
+  echo -e "${GREEN}✅ Docs push step done${NC}\n"
+else
+  echo -e "${RED}❌ Docs push FAILED — soulcraft.com/docs trails npm until re-run or interim sync${NC}\n"
+fi
+
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
 echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}"
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"

src/aggregation/AggregationIndex.ts

@@ -29,6 +29,7 @@ import { matchesMetadataFilter } from '../utils/metadataFilter.js'
 import { compareCodePoints } from '../utils/collation.js'
 import { bucketTimestamp } from './timeWindows.js'
 import { NounType } from '../types/graphTypes.js'
+import { prodLog } from '../utils/logger.js'
 
 /** Persistence key for aggregate definitions */
 const DEFINITIONS_KEY = '__aggregation_definitions__'
@@ -87,10 +88,18 @@ function matchesSource(entity: Record<string, unknown>, source: AggregateDefinit
     if (entity.service !== source.service) return false
   }
 
-  // Metadata where filter — match against the entity's metadata sub-object
+  // Where filter — resolve each filtered field through resolveEntityField,
+  // the SAME single source of truth groupBy uses (top-level standard fields
+  // + custom metadata). Matching only the metadata sub-object made
+  // where:{subtype}/{visibility}/… a silent no-op: reserved fields never
+  // live in the custom bag, so those filters could never match anything.
   if (source.where && Object.keys(source.where).length > 0) {
-    const metadata = (entity.metadata ?? entity) as Record<string, unknown>
-    if (!matchesMetadataFilter(metadata, source.where)) return false
+    const e = entity as unknown as HNSWNounWithMetadata
+    const resolved: Record<string, unknown> = {}
+    for (const key of Object.keys(source.where)) {
+      resolved[key] = resolveEntityField(e, key)
+    }
+    if (!matchesMetadataFilter(resolved, source.where)) return false
   }
 
   return true
@@ -269,7 +278,7 @@ function recomputeMinMaxFromCounts(state: MetricState): void {
 /**
  * Exact percentile over a value multiset, using linear interpolation between closest ranks
  * (numpy 'linear' / type-7): `rank = p·(n−1)`, interpolating between the floor and ceil
- * positions of the sorted expansion. Mirrors Cortex's `MetricState::percentile` bit-for-bit.
+ * positions of the sorted expansion. Mirrors Cor's `MetricState::percentile` bit-for-bit.
  */
 function computePercentile(valueCounts: Record<string, number>, count: number, p: number): number {
   if (count === 0) return 0
@@ -327,6 +336,30 @@ export class AggregationIndex {
   /** Track aggregates with stale MIN/MAX (need lazy recompute) */
   private staleMinMax = new Map<string, Set<string>>()
 
+  /** Resolves when init() has finished loading persisted definitions/state. */
+  private initPromise: Promise<void> | null = null
+
+  /** True once init() has settled (success or failure). */
+  private initDone = false
+
+  /**
+   * Aggregates registered by the app before init() finished loading persisted
+   * state, awaiting reconciliation: init() adopts the persisted state when the
+   * definition hash matches; anything left unadopted when init settles resolves
+   * to a backfill. Deciding backfill eagerly at define time was the boot-order
+   * bug that wiped valid persisted state on every restart — the synchronous
+   * defineAggregate() always beats the async init().
+   */
+  private pendingAdopt = new Set<string>()
+
+  /**
+   * In-flight rescan targets. While a name has a staging map, ALL
+   * contributions (the walk's and concurrent write hooks') land there instead
+   * of the live map; the live map keeps serving until {@link finishBackfill}
+   * swaps the staging map in atomically.
+   */
+  private backfillStaging = new Map<string, Map<string, AggregateGroupState>>()
+
   constructor(storage: StorageAdapter, nativeProvider?: AggregationProvider) {
     this.storage = storage
     this.nativeProvider = nativeProvider
@@ -336,21 +369,128 @@ export class AggregationIndex {
 
   /**
    * Initialize: load persisted definitions and state, detect changes, rebuild stale.
+   *
+   * Idempotent — repeated calls return the same promise. Definitions registered
+   * *before* this completes (the normal boot order: `defineAggregate()` is
+   * synchronous and always beats this async load) are reconciled rather than
+   * clobbered: the app's definition wins, and its persisted state is adopted
+   * when the definition hash matches — backfill happens only on a real change.
    */
-  async init(): Promise<void> {
+  init(): Promise<void> {
+    if (!this.initPromise) {
+      this.initPromise = this.loadPersisted().finally(() => {
+        this.resolvePendingAdoptToBackfill()
+        this.initDone = true
+      })
+    }
+    return this.initPromise
+  }
+
+  /**
+   * Await the persisted-state load (if one was started) and settle every
+   * pending adoption decision. After this resolves, `getPendingBackfills()`
+   * is authoritative: a name is listed iff it genuinely needs a rescan.
+   * Query paths must await this before consulting backfill state.
+   */
+  async ready(): Promise<void> {
+    if (this.initPromise) {
+      try {
+        await this.initPromise
+      } catch {
+        // The owner already surfaced the load failure loudly; backfill covers.
+      }
+    }
+    this.resolvePendingAdoptToBackfill()
+  }
+
+  /**
+   * Any definition still awaiting state adoption has no persisted state to
+   * adopt (or init never ran / failed) — it must backfill.
+   */
+  private resolvePendingAdoptToBackfill(): void {
+    if (this.pendingAdopt.size > 0) {
+      prodLog.info(
+        `[Aggregation] no adoptable persisted state for: ${Array.from(this.pendingAdopt).join(', ')} — flagged for backfill`
+      )
+    }
+    for (const name of this.pendingAdopt) this.needsBackfill.add(name)
+    this.pendingAdopt.clear()
+  }
+
+  /**
+   * May this persisted state be ADOPTED? When the store exposes its committed
+   * watermark, the state's `sourceGeneration` must EQUAL it: behind means
+   * later writes are missing from the state (unclean shutdown); ahead means
+   * it counts writes that no longer exist (e.g. a fact-log truncation on a
+   * copied store pulled the watermark back). Either way: one exact rescan,
+   * said out loud — never a silent adopt. Stores without the capability (and
+   * pre-stamp state on them) fall back to hash-only adoption.
+   */
+  private stateGenerationAdoptable(name: string, stateData: unknown): boolean {
+    const committed = this.storage.committedGeneration?.() ?? null
+    if (committed === null) return true
+    const raw = (stateData as Record<string, unknown>).sourceGeneration
+    const stamped = typeof raw === 'number' ? raw : null
+    if (stamped === committed) return true
+    prodLog.warn(
+      `[Aggregation] '${name}': persisted state is at generation ${stamped ?? 'unstamped'} ` +
+        `but the store's committed generation is ${committed} — rescanning instead of adopting`
+    )
+    return false
+  }
+
+  private async loadPersisted(): Promise<void> {
     // Load persisted definitions
     const savedDefs = await this.storage.getMetadata(DEFINITIONS_KEY)
     if (savedDefs && typeof savedDefs === 'object' && savedDefs.definitions) {
       const defs = savedDefs.definitions as Array<AggregateDefinition & { _hash?: string }>
 
       for (const def of defs) {
-        this.definitions.set(def.name, def)
-        const currentHash = hashDefinition(def)
         const savedHash = def._hash || ''
 
-        // Load persisted state
+        if (this.definitions.has(def.name)) {
+          // The app re-registered this aggregate before the load finished.
+          // The app's definition wins — never clobber it with the persisted
+          // copy. Adopt the persisted state when the definition is unchanged
+          // AND no write has landed for it yet (a landed write would be lost
+          // by adoption; the hook flips such names to backfill).
+          const appHash = this.definitionHashes.get(def.name) || ''
+          if (appHash === savedHash && this.pendingAdopt.has(def.name)) {
             const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
-        if (stateData && stateData.groups && savedHash === currentHash) {
+            if (
+              stateData &&
+              stateData.groups &&
+              this.stateGenerationAdoptable(def.name, stateData)
+            ) {
+              const groupMap = new Map<string, AggregateGroupState>()
+              for (const group of stateData.groups as AggregateGroupState[]) {
+                groupMap.set(serializeGroupKey(group.groupKey), group)
+              }
+              this.states.set(def.name, groupMap)
+              this.pendingAdopt.delete(def.name)
+              this.needsBackfill.delete(def.name)
+              prodLog.info(
+                `[Aggregation] '${def.name}': adopted persisted state (${groupMap.size} groups) — no rescan`
+              )
+            }
+            // No/invalid persisted state: stays in pendingAdopt and resolves
+            // to backfill when init settles.
+          }
+          continue
+        }
+
+        // Not registered this session — restore definition + state from
+        // persistence.
+        this.definitions.set(def.name, def)
+        const currentHash = hashDefinition(def)
+
+        const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
+        if (
+          stateData &&
+          stateData.groups &&
+          savedHash === currentHash &&
+          this.stateGenerationAdoptable(def.name, stateData)
+        ) {
           // Definition unchanged — load state
           const groupMap = new Map<string, AggregateGroupState>()
           for (const group of stateData.groups as AggregateGroupState[]) {
@@ -358,6 +498,10 @@ export class AggregationIndex {
             groupMap.set(serialized, group)
           }
           this.states.set(def.name, groupMap)
+          this.needsBackfill.delete(def.name)
+          prodLog.info(
+            `[Aggregation] '${def.name}': restored definition + adopted persisted state (${groupMap.size} groups)`
+          )
         } else {
           // Definition changed or no saved state — start fresh and backfill from
           // existing entities (the owner drains needsBackfill on first query).
@@ -398,14 +542,22 @@ export class AggregationIndex {
     }))
     await this.storage.saveMetadata(DEFINITIONS_KEY, { definitions: defsToSave })
 
-    // Persist dirty states
+    // Persist dirty states, stamped with the committed generation they
+    // reflect. The stamp is what makes reopen-adoption verifiable: state at a
+    // different generation than the store's committed watermark is stale (an
+    // unclean shutdown after later writes) or over-counts (a fact-log
+    // truncation on a copied store pulled the watermark BACK below the
+    // stamp) — either way the answer is one exact rescan, never a silent
+    // adopt. Read the generation after collecting groups so any racing
+    // commit resolves toward rescan, not wrong-adopt.
     for (const name of this.dirty) {
       const stateMap = this.states.get(name)
       if (stateMap) {
         const groups = Array.from(stateMap.values())
+        const sourceGeneration = this.storage.committedGeneration?.() ?? null
         await this.storage.saveMetadata(
           `${STATE_KEY_PREFIX}${name}__`,
-          { groups }
+          sourceGeneration === null ? { groups } : { groups, sourceGeneration }
         )
       }
     }
@@ -452,10 +604,19 @@ export class AggregationIndex {
     this.definitions.set(def.name, def)
     this.definitionHashes.set(def.name, newHash)
 
+    // First sight this session, before init() settled: defer the backfill
+    // decision — init() adopts the persisted state on hash match, and anything
+    // left unadopted resolves to backfill. Deciding eagerly here wiped valid
+    // persisted state on every restart.
+    if (!this.states.has(def.name) && !this.initDone) {
+      this.states.set(def.name, new Map())
+      this.pendingAdopt.add(def.name)
+    }
     // Reset state if definition changed or doesn't exist yet, and flag it for
     // backfill so already-stored entities are counted (write-time hooks only see
     // future writes). The owner drains this on the next query via getPendingBackfills().
-    if (!this.states.has(def.name) || (oldHash && oldHash !== newHash)) {
+    else if (!this.states.has(def.name) || (oldHash && oldHash !== newHash)) {
+      this.pendingAdopt.delete(def.name)
       this.states.set(def.name, new Map())
       this.needsBackfill.add(def.name)
     }
@@ -476,6 +637,8 @@ export class AggregationIndex {
     this.definitionHashes.delete(name)
     this.states.delete(name)
     this.staleMinMax.delete(name)
+    this.pendingAdopt.delete(name)
+    this.needsBackfill.delete(name)
 
     // Notify native provider
     if (this.nativeProvider?.removeAggregate) {
@@ -513,9 +676,17 @@ export class AggregationIndex {
     return Array.from(this.needsBackfill)
   }
 
-  /** Clear an aggregate's state so a full rescan cannot double-count. */
+  /**
+   * Begin a rescan into a STAGING map. The live state is not touched — it
+   * keeps serving (possibly stale, but flagged pending) until the rescan
+   * completes and swaps in atomically. A mid-walk failure drops the staging
+   * map via {@link abortBackfill} and loses nothing: wiping live state before
+   * a scan that could throw was the destructive-before-durable defect.
+   * Contributions (walk + concurrent write hooks) land in staging while it
+   * exists, so the swapped-in result reflects writes that raced the walk.
+   */
   beginBackfill(name: string): void {
-    this.states.set(name, new Map())
+    this.backfillStaging.set(name, new Map())
     // Reset native provider state for this aggregate too, if present.
     const def = this.definitions.get(name)
     if (def && this.nativeProvider?.removeAggregate && this.nativeProvider?.defineAggregate) {
@@ -524,6 +695,15 @@ export class AggregationIndex {
     }
   }
 
+  /**
+   * Abandon an in-flight rescan after a failure: drop the staging map, keep
+   * the live state serving, leave the aggregate flagged as pending so a later
+   * attempt rescans. The failure itself must be surfaced loudly by the owner.
+   */
+  abortBackfill(name: string): void {
+    this.backfillStaging.delete(name)
+  }
+
   /** Feed one already-stored entity into a single aggregate during backfill. */
   backfillEntity(name: string, entity: Record<string, unknown>): void {
     if (isAggregateEntity(entity)) return
@@ -537,14 +717,33 @@ export class AggregationIndex {
     }
   }
 
-  /** Mark an aggregate's backfill complete; rebuilt state persists on next flush(). */
+  /** Swap the rebuilt staging state in atomically; persists on next flush(). */
   finishBackfill(name: string): void {
+    const staged = this.backfillStaging.get(name)
+    if (staged) {
+      this.states.set(name, staged)
+      this.backfillStaging.delete(name)
+    }
     this.needsBackfill.delete(name)
     this.dirty.add(name)
   }
 
   // ============= Write-Time Hooks =============
 
+  /**
+   * A write is landing for an aggregate whose persisted-state adoption is still
+   * pending — adopting after this write would lose its contribution. Settle the
+   * decision now: an exact rescan instead of adoption. The window is the few
+   * milliseconds between a boot-time defineAggregate() and init() completing,
+   * so this rarely fires; when it does, correctness wins over the walk.
+   */
+  private resolveAdoptOnWrite(name: string): void {
+    if (this.pendingAdopt.has(name)) {
+      this.pendingAdopt.delete(name)
+      this.needsBackfill.add(name)
+    }
+  }
+
   /**
    * Called when an entity is added. Updates all matching aggregates.
    */
@@ -553,6 +752,7 @@ export class AggregationIndex {
 
     for (const [name, def] of this.definitions) {
       if (!matchesSource(entity, def.source)) continue
+      this.resolveAdoptOnWrite(name)
 
       if (this.nativeProvider) {
         const results = this.nativeProvider.incrementalUpdate(name, def, entity, 'add')
@@ -579,6 +779,10 @@ export class AggregationIndex {
       const oldMatches = matchesSource(oldEntity, def.source)
       const newMatches = matchesSource(newEntity, def.source)
 
+      if (oldMatches || newMatches) {
+        this.resolveAdoptOnWrite(name)
+      }
+
       if (this.nativeProvider && (oldMatches || newMatches)) {
         const results = this.nativeProvider.incrementalUpdate(name, def, newEntity, 'update', oldEntity)
         this.applyNativeResults(name, results)
@@ -605,6 +809,7 @@ export class AggregationIndex {
 
     for (const [name, def] of this.definitions) {
       if (!matchesSource(entity, def.source)) continue
+      this.resolveAdoptOnWrite(name)
 
       if (this.nativeProvider) {
         const results = this.nativeProvider.incrementalUpdate(name, def, entity, 'delete')
@@ -757,7 +962,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: an unnest dimension makes one entity contribute to several groups.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -815,7 +1020,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: reverse the entity's contribution from every group it joined.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -871,7 +1076,7 @@ export class AggregationIndex {
    * Apply results from native provider back into the state maps.
    */
   private applyNativeResults(aggName: string, results: AggregateGroupState[]): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
     for (const group of results) {
       const serialized = serializeGroupKey(group.groupKey)
       stateMap.set(serialized, group)

src/aggregation/materializer.ts

@@ -14,6 +14,7 @@ import type {
   AggregateMetricDef
 } from '../types/brainy.types.js'
 import { serializeGroupKey } from './AggregationIndex.js'
+import { prodLog } from '../utils/logger.js'
 
 /**
  * Callback interface for the materializer to create/update Brainy entities.
@@ -86,8 +87,15 @@ export class AggregateMaterializer {
     if (existing) clearTimeout(existing)
 
     this.debounceTimers.set(key, setTimeout(() => {
-      this.materializeOne(key).catch(() => {
-        // Non-fatal — materialization is derived data
+      this.materializeOne(key).catch((err) => {
+        // Materialization is derived data (rebuildable via backfill-on-query), so
+        // a failure is non-fatal — but it leaves the materialized Measurement
+        // entity STALE. Surface it loudly rather than swallow it silently.
+        prodLog.warn(
+          `[Aggregation] Failed to materialize aggregate group '${key}': ` +
+          `${(err as Error).message}. The materialized value is stale until the ` +
+          `next successful materialization or a backfill-on-query.`
+        )
       })
     }, debounceMs))
   }

src/coreTypes.ts

@@ -754,6 +754,39 @@ export interface Change {
   data?: HNSWNounWithMetadata | HNSWVerbWithMetadata
 }
 
+/**
+ * @description A declared derived-index blob FAMILY (the
+ * registered-blob contract). A family names the set of on-disk blobs that a
+ * derived index needs AS A SET (e.g. the vector base = `main.dkann` +
+ * `main.slotmap` + `main.slotrev`): losing ANY member corrupts the index. Once a
+ * family is declared:
+ * - its members are UNDELETABLE through the storage layer — `deleteBinaryBlob` /
+ *   `removeRawPrefix` refuse with a `ProtectedArtifactError`, so an in-process
+ *   GC/sweeper cannot remove a load-bearing file (intentional retirement =
+ *   `unregisterDerivedFamily` first);
+ * - a member missing on open is a loud `DerivedArtifactMissingError` → rebuild.
+ * `COLD ≠ DEAD`: a write-once segment or a recovery-critical archive is
+ * load-bearing even when it has not been touched in a long time.
+ */
+export interface DerivedFamilyDeclaration {
+  /** Stable family id, e.g. `'vector-base'` / `'metadata-sstables'`. */
+  name: string
+  /**
+   * The logical blob keys that make up the family. When {@link namespace} is
+   * set, each entry is a PREFIX protecting every key beneath it (for growing
+   * sets like `seg-*`); otherwise each entry is an exact member key.
+   */
+  members: string[]
+  /** When true, {@link members} are prefixes (protect all keys beneath each). */
+  namespace?: boolean
+  /**
+   * Whether a missing family can be rebuilt from the canonical records (default
+   * `true`). `false` marks an irreplaceable family (a missing member is data
+   * loss, not a rebuild).
+   */
+  rebuildable?: boolean
+}
+
 export interface StorageAdapter {
   init(): Promise<void>
 
@@ -815,7 +848,17 @@ export interface StorageAdapter {
    */
   getNounsByNounType(nounType: string): Promise<HNSWNounWithMetadata[]>
 
-  deleteNoun(id: string): Promise<void>
+  /**
+   * Delete a noun — FULL canonical removal (both legs + the entity container).
+   *
+   * @param id The entity id.
+   * @param priorMetadata OPTIONAL already-known metadata of the entity being
+   *   removed (the caller's pre-delete read). The count decrement must never
+   *   REQUIRE re-reading the record being removed: when the internal read
+   *   returns `null` (replace race, or a ghost left by an earlier version) the
+   *   decrement falls back to this record instead of being silently skipped.
+   */
+  deleteNoun(id: string, priorMetadata?: NounMetadata | null): Promise<void>
 
   /**
    * Save verb - Pure HNSW verb with core fields only
@@ -872,7 +915,15 @@ export interface StorageAdapter {
    */
   getVerbsByType(type: string): Promise<HNSWVerbWithMetadata[]>
 
-  deleteVerb(id: string): Promise<void>
+  /**
+   * Delete a verb — FULL canonical removal (both legs + the container).
+   *
+   * @param id The relationship id.
+   * @param priorMetadata OPTIONAL already-known metadata of the edge being
+   *   removed (the caller's pre-delete read); keeps the count decrement honest
+   *   when the internal read returns `null` (see `deleteNoun`).
+   */
+  deleteVerb(id: string, priorMetadata?: VerbMetadata | null): Promise<void>
 
   /**
    * Save metadata
@@ -1049,6 +1100,104 @@ export interface StorageAdapter {
    */
   getBinaryBlobPath(key: string): string | null
 
+  /**
+   * @description OPTIONAL (registered-blob contract). Declare a
+   * derived-index blob {@link DerivedFamilyDeclaration | family} whose members
+   * become UNDELETABLE through this adapter — a subsequent `deleteBinaryBlob` /
+   * `removeRawPrefix` that would remove a declared member throws a
+   * `ProtectedArtifactError`. Providers declare their families on create; the
+   * declaration is persisted so protection survives a reopen. Idempotent per
+   * `name` (re-declaring replaces).
+   * @param family - The family to protect.
+   */
+  registerDerivedFamily?(family: DerivedFamilyDeclaration): Promise<void>
+
+  /**
+   * @description OPTIONAL. Remove a family's protection so its members can be
+   * deleted again — the explicit, auditable step for intentional retirement of a
+   * derived index (the ONLY way a declared member becomes deletable).
+   * @param name - The {@link DerivedFamilyDeclaration.name} to unregister.
+   */
+  unregisterDerivedFamily?(name: string): Promise<void>
+
+  /**
+   * @description OPTIONAL. List the currently-declared derived-index families —
+   * the source of truth for what `clear()` must wipe and what a
+   * missing-on-open check verifies.
+   */
+  listDerivedFamilies?(): Promise<DerivedFamilyDeclaration[]>
+
+  /**
+   * @description OPTIONAL binary raw-byte primitives — the substrate for
+   * append-only log-structured files (the generation fact log's CRC-framed
+   * segments). Feature-detected: an adapter that omits them simply hosts no
+   * fact log (readers fall back to canonical enumeration). Paths are
+   * storage-root-relative and used VERBATIM (no `.gz`/`.bin` suffixing —
+   * unlike the JSON object and blob primitives).
+   *
+   * Append to a raw binary file, creating it (and parent directories) when
+   * absent. Append durability is the CALLER's job via `syncRawObjects` —
+   * matching the commit protocol, which batches fsyncs at its barrier.
+   */
+  appendRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+
+  /**
+   * Read a raw binary file whole. Absent → `null`; a real IO fault throws
+   * (never masked as absence).
+   */
+  readRawBytes?(path: string): Promise<Uint8Array | null>
+
+  /**
+   * Replace a raw binary file atomically (write-new → fsync → rename) — the
+   * reconcile primitive (e.g. truncating a fact-log tail back to committed
+   * truth after a crash).
+   */
+  writeRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+
+  /**
+   * Byte size of a raw binary file, or `null` when absent.
+   */
+  rawByteSize?(path: string): Promise<number | null>
+
+  /**
+   * @description OPTIONAL fact-scan capability — how an index provider that
+   * holds only `storage` reaches the generation fact log (the host brain
+   * wires it at init; providers must never construct their own fact-log
+   * reader — the log's open path is writer-side). Present ⟺ this store hosts
+   * a fact log AND the host wired the capability. Returns a scan handle over
+   * committed facts (heal-grade telemetry included), or `null` when no fact
+   * log exists — callers fall back to the canonical enumeration walk.
+   */
+  scanFacts?(options?: {
+    fromGeneration?: number
+    toGeneration?: number
+    kinds?: Array<'noun' | 'verb'>
+    batchSize?: number
+  }): import('./db/factLog.js').FactScanHandle | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the fact log's
+   * head generation — the replay target for `stamp.sourceGeneration + 1`
+   * catch-ups. `null` when no fact log exists.
+   */
+  factLogHeadGeneration?(): number | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the COMMITTED
+   * generation watermark — the manifest truth a projection's
+   * `sourceGeneration` compares against. Exposed as a capability so a
+   * provider never parses the store's private manifest format. `null` when
+   * the capability is unwired.
+   */
+  committedGeneration?(): number | null
+
+  /**
+   * @description OPTIONAL (rides the fact-scan capability): the immutable,
+   * sealed fact-segment file paths covering `fromGeneration` — the zero-copy
+   * handoff. The mutable tail is excluded (read it via `scanFacts`).
+   */
+  factSegmentPaths?(options?: { fromGeneration?: number }): string[]
+
   /**
    * Save statistics data
    * @param statistics The statistics data to save
@@ -1138,4 +1287,20 @@ export interface StorageAdapter {
    * @returns Promise that resolves to the total number of verbs
    */
   getVerbCount(): Promise<number>
+
+  /**
+   * OPTIONAL — create a pre-upgrade backup of the whole store and return its
+   * location, or `null` when there is nothing to back up (empty store). On the
+   * filesystem adapter this is a zero-copy hard-link snapshot into a sibling
+   * directory; other backends may omit it. Callers feature-detect. Paired with
+   * {@link removeMigrationBackup}. Used by the 7.x → 8.0 migration lifecycle.
+   */
+  createMigrationBackup?(): Promise<string | null>
+
+  /**
+   * OPTIONAL — remove a backup created by {@link createMigrationBackup}.
+   * Best-effort: a missing `location` is a no-op. Removing a hard-link snapshot
+   * never affects the live store (shared inodes; only the extra links are gone).
+   */
+  removeMigrationBackup?(location: string): Promise<void>
 }

src/db/db.ts

@@ -945,6 +945,13 @@ export class Db<T = any> {
    * {@link SpeculativeOverlayError} (commit them with `brain.transact()`
    * first).
    *
+   * SPARSE FILES: a native accelerator's mmap index files can be sparse —
+   * huge apparent size, small allocated size. `persist()` handles them
+   * correctly (hard links share the allocation). But if you then archive the
+   * snapshot with EXTERNAL tools, use the sparse-aware flags (`tar czSf`,
+   * `rsync --sparse`, `cp --sparse=always`) or the copy materializes every
+   * hole — see docs/guides/external-backups-and-sparse-storage.md.
+   *
    * @param path - Absolute directory for the snapshot (created; must be
    *   empty or absent).
    * @throws GenerationConflictError when this view is no longer the latest

src/db/errors.ts

@@ -159,3 +159,133 @@ export class GenerationCompactedError extends Error {
     this.horizon = horizon
   }
 }
+
+/** One entity/relationship left in an unreconciled state by a failed rollback. */
+export interface UnreconciledRecord {
+  /** The entity or relationship id. */
+  id: string
+  /** `'noun'` (entity) or `'verb'` (relationship). */
+  kind: 'noun' | 'verb'
+  /**
+   * `'orphan'` — the record is durably PRESENT but the transaction aborted
+   * (an add whose delete-undo failed); `'loss'` — the record is durably GONE
+   * or wrong when it should have been restored (a remove/update whose
+   * restore-undo failed — actual data loss).
+   */
+  disposition: 'orphan' | 'loss'
+}
+
+/**
+ * @description Thrown when a transaction's rollback could not be fully applied
+ * and the resulting inconsistency cannot be safely adopted forward — i.e. a
+ * multi-operation batch, or ANY case where a record was lost (a remove/update
+ * whose restore-undo failed). The store is left in a known-inconsistent state:
+ * the {@link records} name every entity/relationship whose canonical state no
+ * longer matches what the aborted transaction should have produced.
+ *
+ * On throw, the brain enters **write-quarantine** — reads continue to work, but
+ * further writes are refused until {@link } `repairIndex()` reconciles the
+ * derived indexes against canonical storage and lifts the quarantine. This is
+ * the honest, loud failure: a visible inconsistency the operator must repair,
+ * never a silent partial write. The generation counter is NOT advanced.
+ *
+ * (A SINGLE-op add whose only damage is a durably-present orphan is NOT this
+ * error: it is adopted forward as a committed generation and returned as a
+ * degraded-but-successful write — the record the caller asked for exists.)
+ *
+ * @example
+ * try {
+ *   await brain.transact(ops)
+ * } catch (err) {
+ *   if (err instanceof StoreInconsistentError) {
+ *     console.error('store inconsistent:', err.records) // ids + dispositions
+ *     await brain.repairIndex() // reconcile + lift the write-quarantine
+ *   }
+ * }
+ */
+export class StoreInconsistentError extends Error {
+  /** Every record left in an unreconciled state by the failed rollback. */
+  public readonly records: UnreconciledRecord[]
+  /** The original error that triggered the (then-failed) rollback. */
+  public override readonly cause: Error
+
+  /**
+   * @param records - The unreconciled entities/relationships (ids + disposition).
+   * @param cause - The error that triggered the rollback.
+   */
+  constructor(records: UnreconciledRecord[], cause: Error) {
+    const orphans = records.filter((r) => r.disposition === 'orphan').length
+    const losses = records.filter((r) => r.disposition === 'loss').length
+    super(
+      `Store left inconsistent by a failed transaction rollback: ` +
+        `${records.length} record(s) could not be reconciled ` +
+        `(${orphans} orphaned, ${losses} lost). ` +
+        `The brain is now WRITE-QUARANTINED (reads still work) — run repairIndex() ` +
+        `to reconcile the derived indexes against canonical storage and lift the ` +
+        `quarantine. Ids: ${records.map((r) => `${r.id}:${r.disposition}`).join(', ')}. ` +
+        `Cause: ${cause.message}`
+    )
+    this.name = 'StoreInconsistentError'
+    this.records = records
+    this.cause = cause
+  }
+}
+
+/**
+ * @description Thrown by a write when the store cannot make single-op
+ * generation **history** durable: the asynchronous group-commit flush that
+ * persists buffered before-images to disk has failed repeatedly (a real
+ * storage fault — a full disk, an EIO, a permission loss — not a transient
+ * blip). Rather than keep accepting writes whose history silently piles up in
+ * memory and is never persisted — an invisible durability loss, and an
+ * unbounded leak — the store LATCHES this failure and refuses further writes
+ * until the pending tier drains.
+ *
+ * This is the loud, honest response to a broken history-durability path:
+ * - Live canonical data is unaffected (single-op live bytes are written before
+ *   the history flush; only the immutable before-image history is stuck).
+ * - The background flush keeps retrying with backoff; when the underlying fault
+ *   clears and a flush succeeds, the latch lifts and writes resume
+ *   automatically. An explicit `flush()` also clears it on success.
+ * - {@link cause} is the underlying storage error from the last failed flush.
+ *
+ * A caller seeing this should treat it exactly like a full disk: stop writing,
+ * resolve the storage fault, and retry. It is NOT a data-corruption error — no
+ * committed generation is lost — it is a refusal to *promise* durability the
+ * store currently cannot deliver.
+ *
+ * @example
+ * try {
+ *   await brain.add({ ... })
+ * } catch (err) {
+ *   if (err instanceof PendingFlushDurabilityError) {
+ *     // History can't be persisted right now (disk fault). Resolve storage,
+ *     // then retry — the store self-heals once a flush succeeds.
+ *     console.error('history durability stalled:', err.cause)
+ *   }
+ * }
+ */
+export class PendingFlushDurabilityError extends Error {
+  /** The underlying storage error from the most recent failed flush. */
+  public override readonly cause: Error
+  /** How many consecutive flush attempts had failed when the latch tripped. */
+  public readonly failedAttempts: number
+
+  /**
+   * @param cause - The storage error from the last failed pending-tier flush.
+   * @param failedAttempts - Consecutive failed flush attempts at latch time.
+   */
+  constructor(cause: Error, failedAttempts: number) {
+    super(
+      `Write refused: single-op generation history could not be made durable ` +
+        `after ${failedAttempts} consecutive flush attempts (${cause.message}). ` +
+        `Live data is intact, but buffered history is not yet persisted — the ` +
+        `store refuses further writes rather than silently accumulate ` +
+        `un-durable history. Resolve the underlying storage fault; the store ` +
+        `self-heals and resumes writes once a flush succeeds. Cause: ${cause.message}`
+    )
+    this.name = 'PendingFlushDurabilityError'
+    this.cause = cause
+    this.failedAttempts = failedAttempts
+  }
+}

src/db/factLog.ts

@@ -0,0 +1,721 @@
+/**
+ * @module db/factLog
+ * @description The generation FACT LOG — an append-only, CRC-framed record of
+ * every committed generation as an AFTER-IMAGE "fact": what each touched
+ * entity/relationship BECAME (or a body-less tombstone when it was removed).
+ * This is the dual-write half of the log-canonical transition: today the
+ * before-image history + canonical tree remain authoritative; the fact log is
+ * appended at the same commit points and reconciled to committed truth at
+ * open, so consumers (index heals, replays, scans) can read one sequential,
+ * self-verifying stream instead of walking the entity tree.
+ *
+ * ## Wire format (frozen; additive-only within a major)
+ *
+ * Fact (msgpack, POSITIONAL array — the segment header's formatVersion
+ * governs the schema):
+ *
+ *   fact := [ generation:u64, timestamp:u64, ops, meta|nil, blobHashes|nil ]
+ *   op   := [ kind:u8 (0=noun, 1=verb), id:bin16 (raw uuid bytes),
+ *             record:[metaLeg, vecLeg] | nil ]        // nil = TOMBSTONE
+ *
+ * Segment file (`_generations/facts/seg-<firstGeneration, zero-padded 20>.bfl`):
+ *
+ *   header := magic "BFACTS\0\0" (8B) | formatVersion:u32 LE |
+ *             firstGeneration:u64 LE | reserved 12B (ZEROED, verified)
+ *   frame  := length:u32 LE | crc32c:u32 LE (of payload) | payload
+ *
+ * A fact is never split across segments; a torn tail (length overrun or CRC
+ * mismatch) terminates that segment's scan — everything before it is intact.
+ * Zero-padded names make lexicographic order == generation order.
+ *
+ * ## Invariant
+ *
+ * After {@link FactLog.open}, the log contains EXACTLY the committed prefix:
+ * facts are appended BEFORE the commit point (inside the same durability
+ * window), so a crash can only leave the log AHEAD of committed truth — open
+ * truncates any fact beyond the committed generation. Absent generation =
+ * never committed; present = committed. A scan can never see an uncommitted
+ * fact.
+ *
+ * The manifest (`_generations/facts/manifest.json`, JSON — forensics stay
+ * terminal-readable) is the single source of truth for the segment SET;
+ * rotation flips it atomically (write-new → fsync → rename) BEFORE the new
+ * tail's first byte exists, so no segment file is ever unaccounted for.
+ */
+import { encode as defaultEncode, decode as defaultDecode } from '@msgpack/msgpack'
+import { crc32c } from '../utils/crc32c.js'
+import { prodLog } from '../utils/logger.js'
+
+// Swappable msgpack implementation — defaults to the JS codec; a native
+// provider (registered via the plugin registry's 'msgpack' key) may replace
+// it. Byte-compatibility is the contract (positional arrays, bin16 ids).
+let msgpackEncode: (value: unknown) => Uint8Array = defaultEncode
+let msgpackDecode: (bytes: Uint8Array) => unknown = defaultDecode
+
+/** Replace the msgpack encode/decode implementation at runtime. */
+export function setFactCodec(impl: {
+  encode: (value: unknown) => Uint8Array
+  decode: (bytes: Uint8Array) => unknown
+}): void {
+  msgpackEncode = impl.encode
+  msgpackDecode = impl.decode
+}
+
+/** Storage-root-relative home of the fact log. */
+export const FACTS_PREFIX = '_generations/facts'
+/** The facts manifest path (JSON). */
+export const FACTS_MANIFEST_PATH = `${FACTS_PREFIX}/manifest.json`
+/** Current segment format version (header field; additive-only within a major). */
+export const FACTS_FORMAT_VERSION = 1
+/** Rotation threshold: seal the tail segment once it exceeds this many bytes. */
+const SEGMENT_ROTATE_BYTES = 8 * 1024 * 1024
+/** Segment header: magic(8) + formatVersion(4) + firstGeneration(8) + reserved(12). */
+const HEADER_BYTES = 32
+const MAGIC = new Uint8Array([0x42, 0x46, 0x41, 0x43, 0x54, 0x53, 0x00, 0x00]) // "BFACTS\0\0"
+/** Frame prefix: length(4) + crc32c(4). */
+const FRAME_PREFIX_BYTES = 8
+
+/** One write inside a fact: what the id became (or a tombstone). */
+export interface FactOp {
+  kind: 'noun' | 'verb'
+  id: string
+  /** The AFTER-IMAGE legs, or `null` for a tombstone (the id was removed). */
+  record: { metadata: unknown | null; vector: unknown | null } | null
+}
+
+/** One committed generation, as scanned back out of the log. */
+export interface CommitFact {
+  generation: number
+  timestamp: number
+  ops: FactOp[]
+  meta?: Record<string, unknown>
+  blobHashes?: string[]
+}
+
+/** The telemetry a scan batch carries (frozen shape). */
+export interface FactScanBatch {
+  facts: CommitFact[]
+  firstGeneration: number
+  lastGeneration: number
+  factCount: number
+  byteSize: number
+  segmentId: string
+}
+
+/**
+ * Liveness bound on a scan's FIRST batch (Stage-2 co-freeze, D1 contract):
+ * `batches()` must yield its first batch — or fail loudly — within this many
+ * ms of the first pull. A backlogged or damaged store may be SLOW, but it may
+ * never be SILENT: a consumer awaiting the first batch is otherwise
+ * indistinguishable from a wedge (the exact failure shape a production heal
+ * hit against a generations-backlogged brain).
+ */
+export const SCANFACTS_FIRST_BATCH_MS = 10_000
+
+/** The telemetry a scan OPEN returns (frozen shape). */
+export interface FactScanHandle {
+  headGeneration: number
+  segmentCount: number
+  approxFactCount: number
+  /**
+   * Ordered batches; a detected gap aborts LOUDLY, never a silent skip.
+   * Liveness contract: the FIRST batch resolves or rejects within
+   * {@link SCANFACTS_FIRST_BATCH_MS} of the first pull — never a silent hang.
+   */
+  batches: () => AsyncGenerator<FactScanBatch>
+  /** Close telemetry — the invariant cross-check, valid after iteration ends. */
+  summary: () => { factsYielded: number; segmentsRead: number }
+}
+
+/** Manifest entry for a sealed segment. */
+interface SegmentEntry {
+  file: string
+  firstGeneration: number
+  lastGeneration: number
+  facts: number
+  bytes: number
+}
+
+/** The facts manifest (JSON on disk). */
+interface FactsManifest {
+  formatVersion: number
+  segments: SegmentEntry[]
+  /** The append target. Its true content is established by scanning (crash tolerance). */
+  tailSegment: string | null
+  updatedAt: string
+}
+
+/** The narrow byte-level storage surface the fact log rides. */
+export interface FactLogStorage {
+  appendRawBytes(path: string, bytes: Uint8Array): Promise<void>
+  readRawBytes(path: string): Promise<Uint8Array | null>
+  writeRawBytes(path: string, bytes: Uint8Array): Promise<void>
+  rawByteSize(path: string): Promise<number | null>
+  readRawObject(path: string): Promise<any | null>
+  writeRawObject(path: string, data: any): Promise<void>
+  syncRawObjects(paths: string[]): Promise<void>
+  deleteRawObject(path: string): Promise<void>
+}
+
+/** True when the storage adapter exposes every primitive the fact log needs. */
+export function storageSupportsFactLog(storage: unknown): storage is FactLogStorage {
+  const s = storage as Record<string, unknown>
+  return (
+    typeof s.appendRawBytes === 'function' &&
+    typeof s.readRawBytes === 'function' &&
+    typeof s.writeRawBytes === 'function' &&
+    typeof s.rawByteSize === 'function'
+  )
+}
+
+/** uuid string → 16 raw bytes (bin16 on the wire). */
+function uuidToBytes(id: string): Uint8Array {
+  const hex = id.replace(/-/g, '')
+  if (hex.length !== 32) {
+    // Non-uuid ids (legacy/natural keys) ride as UTF-8 with a length prefix
+    // marker impossible for uuids: we refuse instead — the write API has
+    // guaranteed uuid ids since 8.0, so anything else is a corruption signal.
+    throw new Error(`fact log: id is not a uuid: ${id}`)
+  }
+  const bytes = new Uint8Array(16)
+  for (let i = 0; i < 16; i++) {
+    bytes[i] = parseInt(hex.slice(i * 2, i * 2 + 2), 16)
+  }
+  return bytes
+}
+
+/** 16 raw bytes → canonical lowercase uuid string. */
+function bytesToUuid(bytes: Uint8Array): string {
+  let hex = ''
+  for (let i = 0; i < 16; i++) hex += bytes[i].toString(16).padStart(2, '0')
+  return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`
+}
+
+/** Zero-padded segment filename: lexicographic order == generation order. */
+function segmentFileName(firstGeneration: number): string {
+  return `seg-${String(firstGeneration).padStart(20, '0')}.bfl`
+}
+
+/** Build a segment header. Reserved bytes are ZEROED (and verified on open). */
+function buildHeader(firstGeneration: number): Uint8Array {
+  const header = new Uint8Array(HEADER_BYTES)
+  header.set(MAGIC, 0)
+  const view = new DataView(header.buffer)
+  view.setUint32(8, FACTS_FORMAT_VERSION, true)
+  view.setBigUint64(12, BigInt(firstGeneration), true)
+  // bytes 20..31 stay zero (reserved)
+  return header
+}
+
+/** Encode one fact into a framed record (length + crc32c + msgpack payload). */
+function encodeFrame(fact: CommitFact): Uint8Array {
+  const payload = msgpackEncode([
+    fact.generation,
+    fact.timestamp,
+    fact.ops.map((op) => [
+      op.kind === 'noun' ? 0 : 1,
+      uuidToBytes(op.id),
+      op.record === null ? null : [op.record.metadata, op.record.vector]
+    ]),
+    fact.meta ?? null,
+    fact.blobHashes && fact.blobHashes.length > 0 ? fact.blobHashes : null
+  ])
+  const frame = new Uint8Array(FRAME_PREFIX_BYTES + payload.length)
+  const view = new DataView(frame.buffer)
+  view.setUint32(0, payload.length, true)
+  view.setUint32(4, crc32c(payload), true)
+  frame.set(payload, FRAME_PREFIX_BYTES)
+  return frame
+}
+
+/** Decode one msgpack payload back into a CommitFact. */
+function decodeFact(payload: Uint8Array): CommitFact {
+  const raw = msgpackDecode(payload) as unknown[]
+  const [generation, timestamp, ops, meta, blobHashes] = raw as [
+    number,
+    number,
+    Array<[number, Uint8Array, [unknown, unknown] | null]>,
+    Record<string, unknown> | null,
+    string[] | null
+  ]
+  return {
+    generation: Number(generation),
+    timestamp: Number(timestamp),
+    ops: ops.map(([kind, idBytes, record]) => ({
+      kind: kind === 0 ? ('noun' as const) : ('verb' as const),
+      id: bytesToUuid(idBytes),
+      record: record === null ? null : { metadata: record[0] ?? null, vector: record[1] ?? null }
+    })),
+    ...(meta ? { meta } : {}),
+    ...(blobHashes && blobHashes.length > 0 ? { blobHashes } : {})
+  }
+}
+
+/**
+ * Parse a segment's bytes: verify the header, then walk frames until the end
+ * or a torn tail (length overrun / CRC mismatch), which terminates the walk —
+ * everything before it is intact. Returns the decoded facts plus the byte
+ * length of the VALID prefix (header + intact frames), which reconciliation
+ * uses to cut a torn tail without re-encoding.
+ */
+function parseSegment(
+  file: string,
+  bytes: Uint8Array
+): { facts: CommitFact[]; validBytes: number } {
+  if (bytes.length < HEADER_BYTES) {
+    prodLog.warn(`[FactLog] segment ${file} shorter than its header — treating as empty`)
+    return { facts: [], validBytes: 0 }
+  }
+  for (let i = 0; i < MAGIC.length; i++) {
+    if (bytes[i] !== MAGIC[i]) {
+      throw new Error(`fact log: segment ${file} has a bad magic — not a fact segment`)
+    }
+  }
+  const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength)
+  const version = view.getUint32(8, true)
+  if (version !== FACTS_FORMAT_VERSION) {
+    throw new Error(
+      `fact log: segment ${file} has formatVersion ${version}; this build reads ${FACTS_FORMAT_VERSION}`
+    )
+  }
+  for (let i = 20; i < HEADER_BYTES; i++) {
+    if (bytes[i] !== 0) {
+      // Non-zero reserved bytes = a future format this build cannot verify.
+      throw new Error(`fact log: segment ${file} has non-zero reserved header bytes — unverifiable`)
+    }
+  }
+
+  const facts: CommitFact[] = []
+  let offset = HEADER_BYTES
+  while (offset + FRAME_PREFIX_BYTES <= bytes.length) {
+    const length = view.getUint32(offset, true)
+    const expectedCrc = view.getUint32(offset + 4, true)
+    const start = offset + FRAME_PREFIX_BYTES
+    const end = start + length
+    if (end > bytes.length) break // torn tail: frame length overruns the file
+    const payload = bytes.subarray(start, end)
+    if (crc32c(payload) !== expectedCrc) break // torn tail: payload CRC mismatch
+    facts.push(decodeFact(payload))
+    offset = end
+  }
+  return { facts, validBytes: offset }
+}
+
+/**
+ * The generation fact log. One instance per open store; every method assumes
+ * the single-writer discipline the generation store already enforces (calls
+ * arrive under its commit mutex).
+ */
+export class FactLog {
+  private readonly storage: FactLogStorage
+  /** Rotation threshold (bytes); tests may lower it to exercise rotation. */
+  private readonly rotateBytes: number
+  private manifest: FactsManifest = {
+    formatVersion: FACTS_FORMAT_VERSION,
+    segments: [],
+    tailSegment: null,
+    updatedAt: new Date(0).toISOString()
+  }
+  /** Decoded facts of the TAIL segment (bounded by the rotation threshold). */
+  private tailFacts: CommitFact[] = []
+  /** Byte size of the tail segment file (valid prefix). */
+  private tailBytes = 0
+  /** Highest generation in the log (0 = empty). */
+  private head = 0
+  /** Segment paths appended since the last sync (the fsync batch). */
+  private readonly dirtySegments = new Set<string>()
+
+  constructor(storage: FactLogStorage, options?: { rotateBytes?: number }) {
+    this.storage = storage
+    this.rotateBytes = options?.rotateBytes ?? SEGMENT_ROTATE_BYTES
+  }
+
+  /** The highest committed generation the log holds (0 = empty). */
+  headGeneration(): number {
+    return this.head
+  }
+
+  /**
+   * Open the log and reconcile it to committed truth: read the manifest,
+   * establish the tail's intact content (torn-tail scan), then TRUNCATE any
+   * fact with `generation > committedGeneration` — those never committed (a
+   * crash between fact-append and the commit point). After open, the log is
+   * exactly the committed prefix.
+   */
+  async open(committedGeneration: number): Promise<void> {
+    const stored = (await this.storage.readRawObject(FACTS_MANIFEST_PATH)) as FactsManifest | null
+    if (stored && typeof stored === 'object' && Array.isArray(stored.segments)) {
+      if (stored.formatVersion !== FACTS_FORMAT_VERSION) {
+        throw new Error(
+          `fact log: manifest formatVersion ${stored.formatVersion}; this build reads ${FACTS_FORMAT_VERSION}`
+        )
+      }
+      this.manifest = stored
+    }
+
+    // Drop sealed segments that sit ENTIRELY beyond committed truth (a crash
+    // right after a rotation whose facts never committed), newest first.
+    while (this.manifest.segments.length > 0) {
+      const last = this.manifest.segments[this.manifest.segments.length - 1]
+      if (last.firstGeneration > committedGeneration) {
+        prodLog.warn(
+          `[FactLog] dropping sealed segment ${last.file} (generations ${last.firstGeneration}..` +
+            `${last.lastGeneration} never committed)`
+        )
+        await this.storage.deleteRawObject(`${FACTS_PREFIX}/${last.file}`)
+        this.manifest.segments.pop()
+        await this.persistManifest()
+      } else if (last.lastGeneration > committedGeneration) {
+        // A sealed segment STRADDLING committed truth: cut it back.
+        await this.truncateSegmentTo(last.file, committedGeneration)
+        const cut = await this.reloadSegmentEntry(last.file)
+        this.manifest.segments[this.manifest.segments.length - 1] = cut
+        await this.persistManifest()
+        break
+      } else {
+        break
+      }
+    }
+
+    // Establish the tail: scan its intact prefix, then truncate beyond
+    // committed truth (the common crash shape: buffered single-op facts whose
+    // counter never went durable).
+    if (this.manifest.tailSegment) {
+      const tailPath = `${FACTS_PREFIX}/${this.manifest.tailSegment}`
+      const bytes = await this.storage.readRawBytes(tailPath)
+      if (bytes === null) {
+        // Manifest named a tail whose first byte never landed — an empty tail.
+        this.tailFacts = []
+        this.tailBytes = 0
+      } else {
+        const { facts, validBytes } = parseSegment(this.manifest.tailSegment, bytes)
+        const kept = facts.filter((f) => f.generation <= committedGeneration)
+        if (kept.length !== facts.length || validBytes !== bytes.length) {
+          const dropped = facts.length - kept.length
+          if (dropped > 0) {
+            prodLog.warn(
+              `[FactLog] truncating ${dropped} uncommitted fact(s) beyond generation ` +
+                `${committedGeneration} from the tail (never committed)`
+            )
+          }
+          await this.rewriteTail(kept)
+        } else {
+          this.tailFacts = facts
+          this.tailBytes = validBytes
+        }
+      }
+    }
+
+    this.head = this.computeHead()
+  }
+
+  /**
+   * Append one committed generation's fact. NOT durable until {@link sync} —
+   * the caller batches durability at its commit barrier (transact syncs in
+   * the same call; Model-B group-commit syncs at flush).
+   */
+  async append(fact: CommitFact): Promise<void> {
+    if (fact.generation <= this.head) {
+      throw new Error(
+        `fact log: non-monotonic append (generation ${fact.generation} ≤ head ${this.head})`
+      )
+    }
+    if (this.manifest.tailSegment === null) {
+      await this.startTail(fact.generation)
+    } else if (this.tailBytes >= this.rotateBytes) {
+      await this.rotate(fact.generation)
+    }
+    const frame = encodeFrame(fact)
+    const tailPath = `${FACTS_PREFIX}/${this.manifest.tailSegment}`
+    await this.storage.appendRawBytes(tailPath, frame)
+    this.tailFacts.push(fact)
+    this.tailBytes += frame.length
+    this.head = fact.generation
+    this.dirtySegments.add(tailPath)
+  }
+
+  /** Fsync every segment appended since the last sync. */
+  async sync(): Promise<void> {
+    if (this.dirtySegments.size === 0) return
+    const paths = [...this.dirtySegments]
+    this.dirtySegments.clear()
+    await this.storage.syncRawObjects(paths)
+  }
+
+  /**
+   * Open a scan over committed facts. The scan runs against a MANIFEST
+   * SNAPSHOT (sealed segments + the tail's decoded facts at open) — exactly-
+   * once per fact, inclusive bounds, stable under concurrent appends. Gaps
+   * abort LOUDLY: a missing generation inside a segment's declared range is
+   * corruption, never silently skipped.
+   */
+  scanFacts(options?: {
+    fromGeneration?: number
+    toGeneration?: number
+    kinds?: Array<'noun' | 'verb'>
+    batchSize?: number
+    /** Test override for the first-batch liveness bound (default {@link SCANFACTS_FIRST_BATCH_MS}). */
+    firstBatchTimeoutMs?: number
+  }): FactScanHandle {
+    const from = options?.fromGeneration ?? 1
+    const to = options?.toGeneration ?? this.head
+    const kinds = options?.kinds
+    const batchSize = Math.max(1, options?.batchSize ?? 256)
+
+    // Snapshot: the segment list + tail content as of NOW.
+    const segments = this.manifest.segments.filter(
+      (s) => s.lastGeneration >= from && s.firstGeneration <= to
+    )
+    const tailSnapshot = this.tailFacts.filter((f) => f.generation >= from && f.generation <= to)
+    const tailId = this.manifest.tailSegment ?? 'tail'
+    const approxFactCount =
+      segments.reduce((sum, s) => sum + s.facts, 0) + tailSnapshot.length
+
+    let factsYielded = 0
+    let segmentsRead = 0
+    const storage = this.storage
+
+    async function* batches(this: void): AsyncGenerator<FactScanBatch> {
+      let expectedNext = 0 // gap detection: generations are monotonic, not necessarily dense
+      const emit = (facts: CommitFact[], segmentId: string, byteSize: number): FactScanBatch => ({
+        facts,
+        firstGeneration: facts[0].generation,
+        lastGeneration: facts[facts.length - 1].generation,
+        factCount: facts.length,
+        byteSize,
+        segmentId
+      })
+      const filterOps = (fact: CommitFact): CommitFact =>
+        kinds
+          ? { ...fact, ops: fact.ops.filter((op) => kinds.includes(op.kind)) }
+          : fact
+
+      for (const entry of segments) {
+        const bytes = await storage.readRawBytes(`${FACTS_PREFIX}/${entry.file}`)
+        if (bytes === null) {
+          throw new Error(
+            `fact log: sealed segment ${entry.file} is MISSING — the log is damaged; aborting scan`
+          )
+        }
+        const { facts } = parseSegment(entry.file, bytes)
+        segmentsRead++
+        const inRange = facts.filter((f) => f.generation >= from && f.generation <= to)
+        for (const f of inRange) {
+          if (f.generation <= expectedNext - 1) {
+            throw new Error(`fact log: out-of-order fact ${f.generation} in ${entry.file} — aborting scan`)
+          }
+          expectedNext = f.generation + 1
+        }
+        for (let i = 0; i < inRange.length; i += batchSize) {
+          const slice = inRange.slice(i, i + batchSize).map(filterOps)
+          if (slice.length === 0) continue
+          factsYielded += slice.length
+          yield emit(slice, entry.file, slice.reduce((n, f) => n + encodeFrame(f).length, 0))
+        }
+      }
+
+      if (tailSnapshot.length > 0) {
+        segmentsRead++
+        for (const f of tailSnapshot) {
+          if (f.generation <= expectedNext - 1) {
+            throw new Error(`fact log: out-of-order fact ${f.generation} in the tail — aborting scan`)
+          }
+          expectedNext = f.generation + 1
+        }
+        for (let i = 0; i < tailSnapshot.length; i += batchSize) {
+          const slice = tailSnapshot.slice(i, i + batchSize).map(filterOps)
+          factsYielded += slice.length
+          yield emit(slice, tailId, slice.reduce((n, f) => n + encodeFrame(f).length, 0))
+        }
+      }
+    }
+
+    // Liveness wrapper: the FIRST pull races the contract deadline. Only the
+    // first — the bound is time-to-first-batch (proof the producer is alive),
+    // not per-batch pacing; and it runs only while a pull is actually pending,
+    // so consumer think-time between pulls never counts against the producer.
+    const firstBatchTimeoutMs = options?.firstBatchTimeoutMs ?? SCANFACTS_FIRST_BATCH_MS
+    async function* batchesWithLiveness(this: void): AsyncGenerator<FactScanBatch> {
+      const inner = batches()
+      let timer: NodeJS.Timeout | undefined
+      try {
+        const deadline = new Promise<never>((_, reject) => {
+          timer = setTimeout(
+            () =>
+              reject(
+                new Error(
+                  `fact log: scanFacts produced no first batch within ${firstBatchTimeoutMs}ms ` +
+                    `(liveness contract) — the store is wedged or unreadably slow; aborting scan LOUDLY ` +
+                    `instead of hanging the consumer.`
+                )
+              ),
+            firstBatchTimeoutMs
+          )
+          timer.unref?.()
+        })
+        const first = await Promise.race([inner.next(), deadline])
+        if (first.done) return
+        yield first.value
+      } finally {
+        clearTimeout(timer)
+      }
+      yield* inner
+    }
+
+    return {
+      headGeneration: this.head,
+      segmentCount: segments.length + (tailSnapshot.length > 0 ? 1 : 0),
+      approxFactCount,
+      batches: batchesWithLiveness,
+      summary: () => ({ factsYielded, segmentsRead })
+    }
+  }
+
+  /**
+   * The mmap fast path (capability handoff): the immutable sealed segment
+   * files covering `fromGeneration`, in order. The TAIL is deliberately NOT
+   * included — it is append-mutable; consumers read it via {@link scanFacts}.
+   */
+  segmentPaths(options?: { fromGeneration?: number }): string[] {
+    const from = options?.fromGeneration ?? 1
+    return this.manifest.segments
+      .filter((s) => s.lastGeneration >= from)
+      .map((s) => `${FACTS_PREFIX}/${s.file}`)
+  }
+
+  /**
+   * Drop every fact with `generation > keepThrough` — the in-session abort
+   * compensation: a transact appends its fact BEFORE the commit point, so a
+   * real (non-crash) abort after the append must take the fact back out. The
+   * dropped facts can only live in the TAIL (they were just appended); the
+   * rewrite is atomic and bounded by the rotation threshold.
+   */
+  async dropAbove(keepThrough: number): Promise<void> {
+    if (this.head <= keepThrough) return
+    const kept = this.tailFacts.filter((f) => f.generation <= keepThrough)
+    if (kept.length === this.tailFacts.length) {
+      throw new Error(
+        `fact log: dropAbove(${keepThrough}) found no droppable facts in the tail ` +
+          `(head ${this.head}) — the fact to drop was already sealed; the log needs reopen`
+      )
+    }
+    await this.rewriteTail(kept)
+    this.head = this.computeHead()
+  }
+
+  // -- internals -------------------------------------------------------------
+
+  private computeHead(): number {
+    if (this.tailFacts.length > 0) return this.tailFacts[this.tailFacts.length - 1].generation
+    const sealed = this.manifest.segments
+    if (sealed.length > 0) return sealed[sealed.length - 1].lastGeneration
+    return 0
+  }
+
+  /** Create the very first tail segment (manifest-first, then header bytes). */
+  private async startTail(firstGeneration: number): Promise<void> {
+    const file = segmentFileName(firstGeneration)
+    this.manifest.tailSegment = file
+    await this.persistManifest()
+    await this.storage.appendRawBytes(`${FACTS_PREFIX}/${file}`, buildHeader(firstGeneration))
+    this.tailFacts = []
+    this.tailBytes = HEADER_BYTES
+  }
+
+  /**
+   * Seal the tail into the manifest and start a new one. Manifest-first: the
+   * flip both seals the old tail AND names the new one atomically, so no
+   * segment file ever exists unaccounted for.
+   */
+  private async rotate(nextGeneration: number): Promise<void> {
+    const sealedFile = this.manifest.tailSegment
+    if (!sealedFile) return
+    // Seal what the tail actually holds.
+    await this.sync() // sealed segments are always fully durable
+    const entry: SegmentEntry = {
+      file: sealedFile,
+      firstGeneration: this.tailFacts[0]?.generation ?? nextGeneration,
+      lastGeneration: this.tailFacts[this.tailFacts.length - 1]?.generation ?? nextGeneration - 1,
+      facts: this.tailFacts.length,
+      bytes: this.tailBytes
+    }
+    const newFile = segmentFileName(nextGeneration)
+    this.manifest.segments.push(entry)
+    this.manifest.tailSegment = newFile
+    await this.persistManifest()
+    await this.storage.appendRawBytes(`${FACTS_PREFIX}/${newFile}`, buildHeader(nextGeneration))
+    this.tailFacts = []
+    this.tailBytes = HEADER_BYTES
+  }
+
+  /** Atomically persist the manifest (write-new → fsync → rename downstream). */
+  private async persistManifest(): Promise<void> {
+    this.manifest.updatedAt = new Date().toISOString()
+    await this.storage.writeRawObject(FACTS_MANIFEST_PATH, this.manifest)
+    await this.storage.syncRawObjects([FACTS_MANIFEST_PATH])
+  }
+
+  /** Rewrite the tail segment to hold exactly `facts` (atomic replace). */
+  private async rewriteTail(facts: CommitFact[]): Promise<void> {
+    const file = this.manifest.tailSegment
+    if (!file) return
+    const first = facts[0]?.generation ?? this.segmentFirstGenerationFromName(file)
+    const parts: Uint8Array[] = [buildHeader(first)]
+    for (const f of facts) parts.push(encodeFrame(f))
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const merged = new Uint8Array(total)
+    let offset = 0
+    for (const p of parts) {
+      merged.set(p, offset)
+      offset += p.length
+    }
+    await this.storage.writeRawBytes(`${FACTS_PREFIX}/${file}`, merged)
+    this.tailFacts = facts
+    this.tailBytes = total
+  }
+
+  /** Cut a SEALED segment back to `committedGeneration` (atomic replace). */
+  private async truncateSegmentTo(file: string, committedGeneration: number): Promise<void> {
+    const path = `${FACTS_PREFIX}/${file}`
+    const bytes = await this.storage.readRawBytes(path)
+    if (bytes === null) return
+    const { facts } = parseSegment(file, bytes)
+    const kept = facts.filter((f) => f.generation <= committedGeneration)
+    prodLog.warn(
+      `[FactLog] truncating sealed segment ${file} to generation ${committedGeneration} ` +
+        `(${facts.length - kept.length} uncommitted fact(s) dropped)`
+    )
+    const first = kept[0]?.generation ?? this.segmentFirstGenerationFromName(file)
+    const parts: Uint8Array[] = [buildHeader(first)]
+    for (const f of kept) parts.push(encodeFrame(f))
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const merged = new Uint8Array(total)
+    let offset = 0
+    for (const p of parts) {
+      merged.set(p, offset)
+      offset += p.length
+    }
+    await this.storage.writeRawBytes(path, merged)
+  }
+
+  /** Re-derive a sealed segment's manifest entry from its actual bytes. */
+  private async reloadSegmentEntry(file: string): Promise<SegmentEntry> {
+    const bytes = await this.storage.readRawBytes(`${FACTS_PREFIX}/${file}`)
+    const { facts, validBytes } = bytes
+      ? parseSegment(file, bytes)
+      : { facts: [] as CommitFact[], validBytes: 0 }
+    return {
+      file,
+      firstGeneration: facts[0]?.generation ?? this.segmentFirstGenerationFromName(file),
+      lastGeneration: facts[facts.length - 1]?.generation ?? 0,
+      facts: facts.length,
+      bytes: validBytes
+    }
+  }
+
+  /** Parse the zero-padded firstGeneration back out of a segment filename. */
+  private segmentFirstGenerationFromName(file: string): number {
+    const match = /^seg-(\d{20})\.bfl$/.exec(file)
+    return match ? Number(match[1]) : 0
+  }
+}

src/db/familyStamp.ts

@@ -0,0 +1,152 @@
+/**
+ * @module db/familyStamp
+ * @description The generalized FAMILY STAMP — one JSON shape that declares,
+ * for any derived projection, WHICH source state it reflects and HOW to verify
+ * it is whole. The entity tree (canonical current-state files) carries the
+ * first brainy-side stamp; native index families carry the same shape. One
+ * verifier reads both member modes:
+ *
+ * - `enumerated` — bounded families: exact byte size per member file,
+ *   verified at open.
+ * - `rollup`     — unbounded families (the entity tree: millions of files):
+ *   the verified surface is a small set of rollup invariants (entity/
+ *   relationship counts) plus `sourceGeneration`.
+ *
+ * `sourceGeneration` is the generation of the source-of-truth log this
+ * projection reflects — open-time coherence becomes a COMPARISON (stamp vs
+ * log head), not a walk:
+ *
+ * - equal + invariants hold → coherent, serve.
+ * - behind → the projection missed the tail (crash between commit and stamp);
+ *   for the Stage-1 tree this is benign by construction (the tree is written
+ *   BY the commit), so the stamp refreshes; a DERIVED projection would replay
+ *   the gap instead.
+ * - invariants FAIL at equal generation → genuine incoherence: loud, and the
+ *   repair ritual (`repairIndex()`, whose recount rebuilds the rollups from a
+ *   canonical walk) heals it.
+ *
+ * Stamps are JSON on purpose — every incident gets debugged by reading a
+ * stamp in a terminal.
+ */
+
+/** Storage-root-relative directory holding family stamps. */
+export const FAMILY_STAMPS_PREFIX = '_system/family-stamps'
+
+/** The entity tree's stamp path. */
+export const ENTITY_TREE_STAMP_PATH = `${FAMILY_STAMPS_PREFIX}/entity-tree.json`
+
+/** One enumerated member: a file and its exact expected byte size. */
+export interface EnumeratedMember {
+  path: string
+  bytes: number
+}
+
+/**
+ * The stamp's verified surface, in one of the two member modes. Rollup
+ * invariant values may be numbers (counts, byte sizes) or strings (content
+ * fingerprints, e.g. a per-tree SHA-256) — the verifier compares by strict
+ * equality either way, so a type mismatch reads as incoherence, never a pass.
+ */
+export type StampMembers =
+  | { mode: 'enumerated'; files: EnumeratedMember[] }
+  | { mode: 'rollup'; invariants: Record<string, number | string> }
+
+/** The generalized family stamp (one shape, one verifier, both engines). */
+export interface FamilyStamp {
+  /** Which projection this stamps (e.g. `'entity-tree'`). */
+  family: string
+  /** Monotonic per-family stamp generation — bumps on every committed stamp. */
+  generation: number
+  /** ISO timestamp of the stamp write. */
+  committedAt: string
+  /** The source-of-truth generation this projection reflects. */
+  sourceGeneration: number
+  /** The verified surface. */
+  members: StampMembers
+}
+
+/** The verdict of an open-time stamp verification. */
+export type StampVerdict =
+  | { state: 'coherent' }
+  | { state: 'absent' } // legacy store — first stamp writes at the next flush
+  | { state: 'behind'; stampSource: number; head: number }
+  | { state: 'incoherent'; failures: string[] }
+  | { state: 'unverifiable'; reason: string } // a FAULT reading the stamp — never conflated with absence
+
+/** The narrow storage surface stamps ride (JSON objects + fsync). */
+export interface StampStorage {
+  readRawObject(path: string): Promise<any | null>
+  writeRawObject(path: string, data: any): Promise<void>
+  syncRawObjects(paths: string[]): Promise<void>
+}
+
+/** Read a family's stamp; `null` when none was ever written. */
+export async function readFamilyStamp(
+  storage: StampStorage,
+  path: string
+): Promise<FamilyStamp | null> {
+  const stored = (await storage.readRawObject(path)) as FamilyStamp | null
+  if (!stored || typeof stored !== 'object' || typeof stored.family !== 'string') return null
+  return stored
+}
+
+/** Write a family's stamp durably (atomic object write + fsync). */
+export async function writeFamilyStamp(
+  storage: StampStorage,
+  path: string,
+  stamp: Omit<FamilyStamp, 'generation' | 'committedAt'> & { generation?: number }
+): Promise<void> {
+  const prior = await readFamilyStamp(storage, path)
+  const full: FamilyStamp = {
+    ...stamp,
+    generation: (prior?.generation ?? 0) + 1,
+    committedAt: new Date().toISOString()
+  }
+  await storage.writeRawObject(path, full)
+  await storage.syncRawObjects([path])
+}
+
+/**
+ * The ONE verifier, both member modes. `actual` supplies the observed rollup
+ * values (rollup mode) or file sizes (enumerated mode, keyed by path);
+ * `head` is the source-of-truth generation now.
+ */
+export function verifyFamilyStamp(
+  stamp: FamilyStamp | null,
+  head: number,
+  actual: Record<string, number | string>
+): StampVerdict {
+  if (stamp === null) return { state: 'absent' }
+  if (stamp.sourceGeneration > head) {
+    // A stamp AHEAD of the log claims state that never committed — the
+    // projection was stamped against truth that a crash rolled back.
+    return {
+      state: 'incoherent',
+      failures: [`sourceGeneration ${stamp.sourceGeneration} is ahead of the log head ${head}`]
+    }
+  }
+  if (stamp.sourceGeneration < head) {
+    return { state: 'behind', stampSource: stamp.sourceGeneration, head }
+  }
+  const failures: string[] = []
+  if (stamp.members.mode === 'rollup') {
+    for (const [name, expected] of Object.entries(stamp.members.invariants)) {
+      const observed = actual[name]
+      if (observed === undefined) {
+        failures.push(`rollup invariant '${name}' has no observed value`)
+      } else if (observed !== expected) {
+        failures.push(`rollup invariant '${name}': stamped ${expected}, observed ${observed}`)
+      }
+    }
+  } else {
+    for (const member of stamp.members.files) {
+      const observed = actual[member.path]
+      if (observed === undefined) {
+        failures.push(`member '${member.path}' is missing`)
+      } else if (observed !== member.bytes) {
+        failures.push(`member '${member.path}': stamped ${member.bytes} bytes, observed ${observed}`)
+      }
+    }
+  }
+  return failures.length > 0 ? { state: 'incoherent', failures } : { state: 'coherent' }
+}

src/db/generationSegments.ts

@@ -0,0 +1,459 @@
+/**
+ * @module db/generationSegments
+ * @description The generation-segment store — Stage-2 D1+D3+repacking's file
+ * format (co-frozen 2026-07-19; design: the d1-d3-repacking spec).
+ *
+ * Packs CONSECUTIVE cold generations' record-sets (before-images + delta)
+ * into append-once segment files with derived sidecar indexes, so history
+ * scales in SEGMENTS (tens) instead of FILES-PER-GENERATION (hundreds of
+ * thousands), and cold-open reads ONE manifest instead of listing the
+ * backlog. Layout under `_generations/segments/`:
+ *
+ * - `seg-<firstGen, zero-padded 20>.bgs` — magic "BGS1", then one frame per
+ *   generation: `u32 payloadLen | u32 crc32c | msgpack payload`. Payload is
+ *   POSITIONAL: `[generation, timestamp, delta, records[], flags]` with
+ *   records `[kindByte, id, record]`. `flags` reserves encoding evolution
+ *   (bit 0 = compressed payload — v1 always 0; a future writer upgrade,
+ *   never a format break). Sealed segments are IMMUTABLE — the fact log's
+ *   own law, generalized.
+ * - `seg-<firstGen>.idx` — DERIVED sidecar (msgpack): per-generation frame
+ *   offsets (point reads = one ranged read, never a listing) + per-id
+ *   generation postings (per-id chain rebuilds read only what they need).
+ *   Corrupt/missing → rebuilt from its segment in one sequential read,
+ *   loudly.
+ * - `manifest.json` — the segment catalogue + `compactedBelow` (D3's
+ *   horizon marker). Cold-open reads THIS; the packed backlog is never
+ *   listed.
+ *
+ * D3 semantics carried here: bounded-retention reclaim drops WHOLE segments
+ * at boundaries (O(1) per segment, no rewrite); under the archival profile
+ * (`retention: 'all'`) nothing here is ever dropped — folding is the only
+ * transform (re-representation, never deletion).
+ */
+
+import { encode as msgpackEncode, decode as msgpackDecode } from '@msgpack/msgpack'
+import { crc32c } from '../utils/crc32c.js'
+import type { FactLogStorage } from './factLog.js'
+import { prodLog } from '../utils/logger.js'
+
+/** Directory for segment files + manifest, under the generations prefix. */
+export const SEGMENTS_PREFIX = '_generations/segments'
+
+/** Target sealed-segment size (co-freeze proposal; tunable on evidence). */
+export const SEGMENT_TARGET_BYTES = 64 * 1024 * 1024
+
+const MAGIC = new TextEncoder().encode('BGS1')
+const FRAME_PREFIX_BYTES = 8 // u32 payloadLen + u32 crc32c
+const MANIFEST_PATH = `${SEGMENTS_PREFIX}/manifest.json`
+
+/** One generation's fold input — exactly what the live tier holds for it. */
+export interface FoldGeneration {
+  generation: number
+  timestamp: number
+  /** The tx.json delta object, carried verbatim. */
+  delta: unknown
+  /** The before-image record-set (empty for record-less generations). */
+  records: Array<{ kind: 'noun' | 'verb'; id: string; record: unknown }>
+}
+
+/** Manifest entry for one sealed segment. */
+export interface SegmentMeta {
+  file: string
+  firstGeneration: number
+  lastGeneration: number
+  frames: number
+  bytes: number
+  /** crc32c of the full segment byte stream — the digest chain's link. */
+  checksum: number
+}
+
+interface SegmentManifest {
+  version: 1
+  compactedBelow: number
+  segments: SegmentMeta[]
+}
+
+interface SidecarIndex {
+  version: 1
+  /** [generation, frameOffset, frameLen] ascending by generation. */
+  generations: Array<[number, number, number]>
+  /** `${kindByte}:${id}` → ascending generations holding a record for it. */
+  ids: Record<string, number[]>
+}
+
+const segmentFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.bgs`
+const sidecarFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.idx`
+
+/**
+ * The generation-segment store. Owns the packed tier ONLY — the live
+ * per-generation tier and the routing between tiers belong to
+ * `GenerationStore`. All mutating entry points here are called under the
+ * generation store's commit mutex.
+ */
+export class GenerationSegmentStore {
+  private readonly storage: FactLogStorage
+  private manifest: SegmentManifest = { version: 1, compactedBelow: 0, segments: [] }
+  /** Sidecar cache — segments are immutable, so entries never invalidate. */
+  private readonly sidecars = new Map<string, SidecarIndex>()
+
+  constructor(storage: FactLogStorage) {
+    this.storage = storage
+  }
+
+  /** Load the manifest (ONE read — never a directory listing). */
+  async open(): Promise<void> {
+    const raw = (await this.storage.readRawObject(MANIFEST_PATH)) as SegmentManifest | null
+    if (raw) {
+      if (raw.version !== 1) {
+        throw new Error(
+          `[GenerationSegments] manifest version ${String(raw.version)} is newer than this ` +
+            `engine understands — refusing to serve partial history. Upgrade the engine.`
+        )
+      }
+      this.manifest = raw
+    }
+  }
+
+  /** The packed tier's catalogue (ascending, immutable snapshot). */
+  segments(): readonly SegmentMeta[] {
+    return this.manifest.segments
+  }
+
+  /** D3's horizon marker: generations below this were reclaimed (bounded profiles only). */
+  compactedBelow(): number {
+    return this.manifest.compactedBelow
+  }
+
+  /** The covering sealed segment for `gen`, or null if it lives outside the packed tier. */
+  private coveringSegment(gen: number): SegmentMeta | null {
+    // Manifest is ascending and ranges never overlap — binary search.
+    const segs = this.manifest.segments
+    let lo = 0
+    let hi = segs.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      const s = segs[mid]
+      if (gen < s.firstGeneration) hi = mid - 1
+      else if (gen > s.lastGeneration) lo = mid + 1
+      else return s
+    }
+    return null
+  }
+
+  /** True when `gen` is packed (readable from this tier). */
+  hasGeneration(gen: number): boolean {
+    return this.coveringSegment(gen) !== null
+  }
+
+  /**
+   * Fold consecutive generations into ONE new sealed segment + sidecar and
+   * append it to the manifest atomically. Caller guarantees: `gens` is
+   * ascending, contiguous with the packed tier (first = last packed + 1 when
+   * segments exist), and already durable in the live tier. Crash between the
+   * segment write and the caller's live-tier delete leaves a DUPLICATE
+   * representation — resolved live-tier-wins by the reader; never a gap.
+   */
+  async fold(gens: FoldGeneration[]): Promise<SegmentMeta> {
+    if (gens.length === 0) {
+      throw new Error('[GenerationSegments] fold() requires at least one generation')
+    }
+    for (let i = 1; i < gens.length; i++) {
+      if (gens[i].generation <= gens[i - 1].generation) {
+        throw new Error('[GenerationSegments] fold() input must be strictly ascending')
+      }
+    }
+    const last = this.manifest.segments[this.manifest.segments.length - 1]
+    if (last && gens[0].generation <= last.lastGeneration) {
+      throw new Error(
+        `[GenerationSegments] fold() overlaps the packed tier: ${gens[0].generation} ≤ ` +
+          `sealed ${last.lastGeneration} — segments are immutable, never rewritten`
+      )
+    }
+
+    const first = gens[0].generation
+    const file = segmentFileName(first)
+    const sidecar: SidecarIndex = { version: 1, generations: [], ids: {} }
+
+    // Encode all frames, tracking offsets for the sidecar.
+    const parts: Uint8Array[] = [MAGIC]
+    let offset = MAGIC.length
+    for (const g of gens) {
+      const payload = msgpackEncode([
+        g.generation,
+        g.timestamp,
+        g.delta,
+        g.records.map((r) => [r.kind === 'noun' ? 0 : 1, r.id, r.record]),
+        0 // flags: v1 = uncompressed
+      ])
+      const frame = new Uint8Array(FRAME_PREFIX_BYTES + payload.length)
+      const view = new DataView(frame.buffer)
+      view.setUint32(0, payload.length, true)
+      view.setUint32(4, crc32c(payload), true)
+      frame.set(payload, FRAME_PREFIX_BYTES)
+      sidecar.generations.push([g.generation, offset, frame.length])
+      for (const r of g.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(sidecar.ids[key] ??= []).push(g.generation)
+      }
+      parts.push(frame)
+      offset += frame.length
+    }
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const bytes = new Uint8Array(total)
+    let at = 0
+    for (const p of parts) {
+      bytes.set(p, at)
+      at += p.length
+    }
+
+    const meta: SegmentMeta = {
+      file,
+      firstGeneration: first,
+      lastGeneration: gens[gens.length - 1].generation,
+      frames: gens.length,
+      bytes: total,
+      checksum: crc32c(bytes)
+    }
+
+    // Durability order: segment + sidecar fsync'd BEFORE the manifest names
+    // them (a crash before the manifest = invisible orphan files, harmless);
+    // manifest last, atomically.
+    const segPath = `${SEGMENTS_PREFIX}/${file}`
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(first)}`
+    await this.storage.writeRawBytes(segPath, bytes)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(sidecar))
+    await this.storage.syncRawObjects([segPath, idxPath])
+    const next: SegmentManifest = {
+      ...this.manifest,
+      segments: [...this.manifest.segments, meta]
+    }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    this.sidecars.set(file, sidecar)
+    return meta
+  }
+
+  /** Load (or rebuild, loudly) a segment's sidecar. */
+  private async sidecarFor(meta: SegmentMeta): Promise<SidecarIndex> {
+    const cached = this.sidecars.get(meta.file)
+    if (cached) return cached
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(meta.firstGeneration)}`
+    const raw = await this.storage.readRawBytes(idxPath)
+    if (raw) {
+      try {
+        const idx = msgpackDecode(raw) as SidecarIndex
+        if (idx.version === 1) {
+          this.sidecars.set(meta.file, idx)
+          return idx
+        }
+      } catch {
+        // fall through to rebuild
+      }
+    }
+    // Sidecars are DERIVED: rebuild from the segment, loudly — never serve
+    // wrong offsets silently.
+    prodLog.warn(
+      `[GenerationSegments] sidecar for ${meta.file} missing or unreadable — rebuilding from the segment`
+    )
+    const rebuilt = await this.rebuildSidecar(meta)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(rebuilt))
+    this.sidecars.set(meta.file, rebuilt)
+    return rebuilt
+  }
+
+  /** One sequential read of the segment → a fresh sidecar. Verifies every frame CRC. */
+  private async rebuildSidecar(meta: SegmentMeta): Promise<SidecarIndex> {
+    const frames = await this.readAllFrames(meta)
+    const idx: SidecarIndex = { version: 1, generations: [], ids: {} }
+    for (const f of frames) {
+      idx.generations.push([f.generation, f.offset, f.frameLen])
+      for (const r of f.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(idx.ids[key] ??= []).push(f.generation)
+      }
+    }
+    return idx
+  }
+
+  private decodeFrame(
+    payload: Uint8Array
+  ): { generation: number; timestamp: number; delta: unknown; records: FoldGeneration['records'] } {
+    const [generation, timestamp, delta, rawRecords] = msgpackDecode(payload) as [
+      number,
+      number,
+      unknown,
+      Array<[number, string, unknown]>,
+      number
+    ]
+    return {
+      generation,
+      timestamp,
+      delta,
+      records: rawRecords.map(([kindByte, id, record]) => ({
+        kind: kindByte === 0 ? ('noun' as const) : ('verb' as const),
+        id,
+        record
+      }))
+    }
+  }
+
+  private async readAllFrames(meta: SegmentMeta): Promise<
+    Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }>
+  > {
+    const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+    if (!bytes) {
+      throw new Error(
+        `[GenerationSegments] sealed segment ${meta.file} is MISSING — packed history is damaged; ` +
+          `refusing to continue silently`
+      )
+    }
+    const out: Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }> = []
+    let at = MAGIC.length
+    const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength)
+    while (at + FRAME_PREFIX_BYTES <= bytes.length) {
+      const payloadLen = view.getUint32(at, true)
+      const crc = view.getUint32(at + 4, true)
+      const payload = bytes.subarray(at + FRAME_PREFIX_BYTES, at + FRAME_PREFIX_BYTES + payloadLen)
+      if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+        throw new Error(
+          `[GenerationSegments] frame CRC mismatch in ${meta.file} at offset ${at} — ` +
+            `packed history is damaged; refusing to serve it`
+        )
+      }
+      out.push({ ...this.decodeFrame(payload), offset: at, frameLen: FRAME_PREFIX_BYTES + payloadLen })
+      at += FRAME_PREFIX_BYTES + payloadLen
+    }
+    return out
+  }
+
+  /** Read one packed generation's frame via its sidecar offset (one ranged read). */
+  private async readFrame(
+    gen: number
+  ): Promise<ReturnType<GenerationSegmentStore['decodeFrame']> | null> {
+    const meta = this.coveringSegment(gen)
+    if (!meta) return null
+    const idx = await this.sidecarFor(meta)
+    // generations ascending → binary search.
+    const gens = idx.generations
+    let lo = 0
+    let hi = gens.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      if (gens[mid][0] < gen) lo = mid + 1
+      else if (gens[mid][0] > gen) hi = mid - 1
+      else {
+        const [, offset, frameLen] = gens[mid]
+        const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+        if (!bytes) {
+          throw new Error(`[GenerationSegments] sealed segment ${meta.file} is MISSING`)
+        }
+        const frame = bytes.subarray(offset, offset + frameLen)
+        const view = new DataView(frame.buffer, frame.byteOffset, frame.byteLength)
+        const payloadLen = view.getUint32(0, true)
+        const crc = view.getUint32(4, true)
+        const payload = frame.subarray(FRAME_PREFIX_BYTES, FRAME_PREFIX_BYTES + payloadLen)
+        if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+          throw new Error(
+            `[GenerationSegments] frame CRC mismatch for generation ${gen} in ${meta.file} — ` +
+              `packed history is damaged; refusing to serve it`
+          )
+        }
+        return this.decodeFrame(payload)
+      }
+    }
+    // In the covering range but not present: the packed tier is dense by
+    // construction (fold packs every generation it is handed, including
+    // record-less ones) — absence inside a sealed range is damage.
+    throw new Error(
+      `[GenerationSegments] generation ${gen} is inside sealed segment ${meta.file}'s declared ` +
+        `range but has no frame — packed history is damaged`
+    )
+  }
+
+  /** The packed tier's delta for `gen` (null = not packed). */
+  async readDelta(gen: number): Promise<{ delta: unknown; timestamp: number } | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? { delta: frame.delta, timestamp: frame.timestamp } : null
+  }
+
+  /** The packed tier's full record-set for `gen` (null = not packed). */
+  async readRecords(gen: number): Promise<FoldGeneration['records'] | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? frame.records : null
+  }
+
+  /** One packed before-image (null = not packed OR no record for the id in that generation). */
+  async readRecord(gen: number, kind: 'noun' | 'verb', id: string): Promise<unknown | null> {
+    const frame = await this.readFrame(gen)
+    if (!frame) return null
+    const hit = frame.records.find((r) => r.kind === kind && r.id === id)
+    return hit ? hit.record : null
+  }
+
+  /**
+   * D3 reclaim: drop WHOLE segments whose lastGeneration < `belowGeneration`
+   * and bump `compactedBelow`. Partial segments are never dropped — the
+   * boundary waits. NEVER called under the archival profile (the caller
+   * enforces retention semantics; this method only executes boundary drops).
+   */
+  async dropSegmentsBelow(belowGeneration: number): Promise<{ dropped: number; compactedBelow: number }> {
+    const keep: SegmentMeta[] = []
+    const drop: SegmentMeta[] = []
+    for (const s of this.manifest.segments) {
+      ;(s.lastGeneration < belowGeneration ? drop : keep).push(s)
+    }
+    if (drop.length === 0) {
+      return { dropped: 0, compactedBelow: this.manifest.compactedBelow }
+    }
+    const compactedBelow = Math.max(
+      this.manifest.compactedBelow,
+      drop[drop.length - 1].lastGeneration + 1
+    )
+    // Manifest first (the drop is authoritative once named), then bytes —
+    // a crash between leaves orphan segment files invisible to the manifest,
+    // harmless and re-collectable.
+    const next: SegmentManifest = { ...this.manifest, compactedBelow, segments: keep }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    for (const s of drop) {
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${s.file}`)
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${sidecarFileName(s.firstGeneration)}`)
+      this.sidecars.delete(s.file)
+    }
+    return { dropped: drop.length, compactedBelow }
+  }
+
+  /**
+   * D8 rider — the packed portion of `generationDigest(g)`: a deterministic
+   * crc32c chain over sealed-segment checksums fully below `g`, plus the
+   * frame CRC of `g`'s own frame when `g` is mid-segment. O(segments), not
+   * O(generations); identical history ⇒ identical digest on any machine.
+   * The live-tier portion is composed by the caller.
+   */
+  async digestThroughPacked(g: number): Promise<number | null> {
+    let digest = 0
+    let covered = false
+    for (const s of this.manifest.segments) {
+      if (s.lastGeneration <= g) {
+        digest = crc32c(new TextEncoder().encode(`${digest}:${s.checksum}`))
+        if (s.lastGeneration === g) covered = true
+      } else if (s.firstGeneration <= g) {
+        // g is mid-segment: chain the partial prefix via g's frame CRC.
+        const frame = await this.readFrame(g)
+        if (frame === null) return null
+        const idx = await this.sidecarFor(s)
+        const upTo = idx.generations.filter(([gen]) => gen <= g)
+        for (const [gen, offset, frameLen] of upTo) {
+          digest = crc32c(new TextEncoder().encode(`${digest}:${gen}:${offset}:${frameLen}`))
+        }
+        covered = true
+        break
+      }
+    }
+    return covered || this.manifest.segments.length > 0 ? digest : null
+  }
+}

src/db/types.ts

@@ -116,6 +116,16 @@ export interface TransactOptions {
    * record is staged.
    */
   ifAtGeneration?: number
+  /**
+   * Budget (ms) for the atomic apply phase. When omitted, the budget SCALES
+   * with the batch: `max(30 000, opCount × 2 000)` — production imports on
+   * network-attached disks measure ~2 s per operation, so a flat 30 s budget
+   * silently capped honest bulk work at ~15 operations. A tripped budget
+   * rolls the whole batch back and throws a retryable
+   * `TransactionTimeoutError` naming the operation it stopped at, the batch
+   * size, and the elapsed/budget times.
+   */
+  timeoutMs?: number
 }
 
 /**
@@ -166,6 +176,15 @@ export interface CompactHistoryOptions {
    * of each surviving generation's serialized record set (`GenerationDelta.bytes`).
    */
   maxBytes?: number
+  /**
+   * Stop reclaiming after this many milliseconds even if caps are still
+   * exceeded (8.9.0). Compaction is maintenance — a bounded pass keeps
+   * `close()` (and any explicit maintenance window) from stalling on a large
+   * backlog; the next pass resumes where this one stopped (reclamation is
+   * oldest-first, so an early stop is always a consistent prefix). Unset =
+   * run to completion.
+   */
+  timeBudgetMs?: number
 }
 
 /**
@@ -183,6 +202,36 @@ export interface CompactHistoryResult {
   horizon: number
 }
 
+/**
+ * @description Result of `brain.historyStats()` — the read-only generational
+ * history footprint, for fleet audits and ops doors. A pool operator runs this
+ * per brain to size retention exposure (how much MVCC history each brain
+ * carries and under which policy) without touching any data.
+ */
+export interface HistoryStats {
+  /** Committed generation record-sets currently on disk. */
+  generations: number
+  /** Total on-disk history bytes across those record-sets. */
+  bytes: number
+  /** Oldest committed generation still on disk (null when history is empty). */
+  oldestGeneration: number | null
+  /** Newest committed generation (null when history is empty). */
+  newestGeneration: number | null
+  /** Commit timestamp (ms) of the oldest on-disk generation. */
+  oldestTimestamp: number | null
+  /** Commit timestamp (ms) of the newest on-disk generation. */
+  newestTimestamp: number | null
+  /** Compaction horizon — generations below it were reclaimed. */
+  horizon: number
+  /** The effective retention mode this brain runs under. */
+  retentionMode: 'all' | 'adaptive' | 'explicit'
+  /**
+   * The adaptive byte budget in force (coordinator-driven or the local
+   * free-memory probe); null under 'all' or explicit caps.
+   */
+  effectiveBudgetBytes: number | null
+}
+
 // ============================================================================
 // Db surfaces
 // ============================================================================
@@ -301,6 +350,16 @@ export interface GenerationDelta {
   nouns: string[]
   /** Relationship ids touched by this generation. */
   verbs: string[]
+  /**
+   * Content-blob hashes referenced by this generation's before-image records
+   * (a MULTISET — one entry per referencing record occurrence), captured at
+   * persist time so compaction can release the exact history references it
+   * reclaims without re-reading the records. Always present (possibly empty)
+   * on deltas written under the temporal-blob contract; absent only on
+   * pre-contract generations, for which compaction falls back to reading the
+   * record-set itself.
+   */
+  blobHashes?: string[]
   /**
    * `true` for a Model-B single-operation generation persisted by the async
    * group-commit flush (`GenerationStore.flushPendingSingleOps`). It marks the
@@ -381,6 +440,24 @@ export interface GenerationStorage {
   /** Durability barrier: fsync the given object paths (no-op in memory). */
   syncRawObjects(paths: string[]): Promise<void>
 
+  /**
+   * OPTIONAL transaction durability barrier. `commitTransaction` calls
+   * {@link beginWriteBarrier} immediately before running the planned operations
+   * and {@link flushWriteBarrier} immediately after — BEFORE the generation
+   * counter and manifest are advanced. An adapter whose canonical writes are
+   * not synchronously durable (the filesystem adapter's tmp+rename lands in the
+   * page cache) MUST implement these so a transaction reported "committed" is
+   * durable before its generation stamp: otherwise a hard kill can leave the
+   * fsync'd counter ahead of the still-buffered entity bytes (phantom progress
+   * for any generation-based consumer). Adapters whose writes are already
+   * durable per-call (cloud object PUT) may leave these undefined — the store
+   * treats them as no-ops. `beginWriteBarrier` also resets any tracking left by
+   * an aborted prior transaction.
+   */
+  beginWriteBarrier?(): void
+  /** @see beginWriteBarrier — fsync every canonical write since begin. */
+  flushWriteBarrier?(): Promise<void>
+
   /** Read an entity's raw stored metadata+vector objects. */
   readNounRaw(id: string): Promise<{ metadata: any | null; vector: any | null }>
   /** Restore an entity's raw stored objects (`null` part ⇒ delete that file). */
@@ -395,6 +472,44 @@ export interface GenerationStorage {
   /** Read all lines of `_system/tx-log.jsonl` (empty array if absent). */
   readTxLogLines(): Promise<string[]>
 
+  /**
+   * OPTIONAL binary raw-byte primitives — the substrate for the generation
+   * fact log's append-only CRC-framed segments. Feature-detected: a storage
+   * layer that omits them hosts no fact log (dual-write is skipped; readers
+   * fall back to canonical enumeration). Paths are used VERBATIM (no
+   * suffixing). Append durability rides `syncRawObjects` at the commit
+   * barrier, exactly like the staged history files.
+   */
+  appendRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+  /** Read a raw binary file whole; absent → null; a real fault throws. */
+  readRawBytes?(path: string): Promise<Uint8Array | null>
+  /** Replace a raw binary file atomically (tmp → fsync → rename). */
+  writeRawBytes?(path: string, bytes: Uint8Array): Promise<void>
+  /** Byte size of a raw binary file, or null when absent. */
+  rawByteSize?(path: string): Promise<number | null>
+
+  /**
+   * OPTIONAL temporal-blob contract (implemented by blob-aware storage; the
+   * generation store treats the hashes as opaque strings). Extract the
+   * content-blob hashes a record-set references — a pure multiset extraction,
+   * no side effects.
+   */
+  extractBlobHashesFromRecords?(records: GenerationRecord[]): string[]
+  /**
+   * OPTIONAL: record history references for the given hashes (one increment
+   * per occurrence). Called BEFORE the referencing record-set is persisted —
+   * a crash between the two can only over-count (a leak the scrub repairs),
+   * never under-count (which would risk reclaiming bytes history still needs).
+   */
+  recordHistoryBlobReferences?(hashes: string[]): Promise<void>
+  /**
+   * OPTIONAL: release history references for the given hashes (one decrement
+   * per occurrence) and physically reclaim any hash left with zero live AND
+   * zero history references. Called AFTER the referencing record-set is
+   * deleted by compaction — same over-count-only crash ordering.
+   */
+  releaseHistoryBlobReferences?(hashes: string[]): Promise<void>
+
   /**
    * Register the generation-bump hook invoked on every entity-visible
    * single-operation write (see `BaseStorage.setGenerationBumpHook`).

src/embeddings/EmbeddingManager.ts

@@ -60,7 +60,7 @@ export class EmbeddingManager {
   private constructor() {
     this.engine = WASMEmbeddingEngine.getInstance()
     // Log deferred to init() — at construction time we don't know if a plugin
-    // (like Cortex) will replace the WASM embedder with a native one.
+    // (like Cor) will replace the WASM embedder with a native one.
   }
 
   /**

src/embeddings/wasm/pkg/candle_embeddings.d.ts

@@ -0,0 +1,100 @@
+/* tslint:disable */
+/* eslint-disable */
+
+/**
+ * WASM-compatible embedding engine
+ */
+export class EmbeddingEngine {
+    free(): void;
+    [Symbol.dispose](): void;
+    /**
+     * Get the embedding dimension (384 for all-MiniLM-L6-v2)
+     */
+    dimension(): number;
+    /**
+     * Generate embedding for a single text
+     *
+     * Returns a Float32Array of 384 dimensions
+     */
+    embed(text: string): Float32Array;
+    /**
+     * Generate embeddings for multiple texts
+     *
+     * Takes a JavaScript Array of strings
+     * Returns a JavaScript Array of Float32Array
+     */
+    embed_batch(texts: Array<any>): Array<any>;
+    /**
+     * Check if the engine is ready for inference
+     */
+    is_ready(): boolean;
+    /**
+     * Load the model and tokenizer from bytes
+     *
+     * This is now the ONLY way to initialize the engine.
+     * Model weights are no longer embedded in WASM for faster initialization.
+     *
+     * # Arguments
+     * * `model_bytes` - SafeTensors format model weights
+     * * `tokenizer_bytes` - tokenizer.json contents
+     * * `config_bytes` - config.json contents
+     */
+    load(model_bytes: Uint8Array, tokenizer_bytes: Uint8Array, config_bytes: Uint8Array): void;
+    /**
+     * Get the maximum sequence length
+     */
+    max_sequence_length(): number;
+    /**
+     * Create a new embedding engine instance (not loaded)
+     */
+    constructor();
+}
+
+/**
+ * Calculate cosine similarity between two embeddings
+ */
+export function cosine_similarity(a: Float32Array, b: Float32Array): number;
+
+export type InitInput = RequestInfo | URL | Response | BufferSource | WebAssembly.Module;
+
+export interface InitOutput {
+    readonly memory: WebAssembly.Memory;
+    readonly __wbg_embeddingengine_free: (a: number, b: number) => void;
+    readonly cosine_similarity: (a: number, b: number, c: number, d: number) => number;
+    readonly embeddingengine_dimension: (a: number) => number;
+    readonly embeddingengine_embed: (a: number, b: number, c: number) => [number, number, number];
+    readonly embeddingengine_embed_batch: (a: number, b: any) => [number, number, number];
+    readonly embeddingengine_is_ready: (a: number) => number;
+    readonly embeddingengine_load: (a: number, b: number, c: number, d: number, e: number, f: number, g: number) => [number, number];
+    readonly embeddingengine_max_sequence_length: (a: number) => number;
+    readonly embeddingengine_new: () => number;
+    readonly __wbindgen_malloc: (a: number, b: number) => number;
+    readonly __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
+    readonly __wbindgen_exn_store: (a: number) => void;
+    readonly __externref_table_alloc: () => number;
+    readonly __wbindgen_externrefs: WebAssembly.Table;
+    readonly __externref_table_dealloc: (a: number) => void;
+    readonly __wbindgen_start: () => void;
+}
+
+export type SyncInitInput = BufferSource | WebAssembly.Module;
+
+/**
+ * Instantiates the given `module`, which can either be bytes or
+ * a precompiled `WebAssembly.Module`.
+ *
+ * @param {{ module: SyncInitInput }} module - Passing `SyncInitInput` directly is deprecated.
+ *
+ * @returns {InitOutput}
+ */
+export function initSync(module: { module: SyncInitInput } | SyncInitInput): InitOutput;
+
+/**
+ * If `module_or_path` is {RequestInfo} or {URL}, makes a request and
+ * for everything else, calls `WebAssembly.instantiate` directly.
+ *
+ * @param {{ module_or_path: InitInput | Promise<InitInput> }} module_or_path - Passing `InitInput` directly is deprecated.
+ *
+ * @returns {Promise<InitOutput>}
+ */
+export default function __wbg_init (module_or_path?: { module_or_path: InitInput | Promise<InitInput> } | InitInput | Promise<InitInput>): Promise<InitOutput>;

src/embeddings/wasm/pkg/candle_embeddings.js

@@ -0,0 +1,525 @@
+/* @ts-self-types="./candle_embeddings.d.ts" */
+
+/**
+ * WASM-compatible embedding engine
+ */
+export class EmbeddingEngine {
+    __destroy_into_raw() {
+        const ptr = this.__wbg_ptr;
+        this.__wbg_ptr = 0;
+        EmbeddingEngineFinalization.unregister(this);
+        return ptr;
+    }
+    free() {
+        const ptr = this.__destroy_into_raw();
+        wasm.__wbg_embeddingengine_free(ptr, 0);
+    }
+    /**
+     * Get the embedding dimension (384 for all-MiniLM-L6-v2)
+     * @returns {number}
+     */
+    dimension() {
+        const ret = wasm.embeddingengine_dimension(this.__wbg_ptr);
+        return ret >>> 0;
+    }
+    /**
+     * Generate embedding for a single text
+     *
+     * Returns a Float32Array of 384 dimensions
+     * @param {string} text
+     * @returns {Float32Array}
+     */
+    embed(text) {
+        const ptr0 = passStringToWasm0(text, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
+        const len0 = WASM_VECTOR_LEN;
+        const ret = wasm.embeddingengine_embed(this.__wbg_ptr, ptr0, len0);
+        if (ret[2]) {
+            throw takeFromExternrefTable0(ret[1]);
+        }
+        return takeFromExternrefTable0(ret[0]);
+    }
+    /**
+     * Generate embeddings for multiple texts
+     *
+     * Takes a JavaScript Array of strings
+     * Returns a JavaScript Array of Float32Array
+     * @param {Array<any>} texts
+     * @returns {Array<any>}
+     */
+    embed_batch(texts) {
+        const ret = wasm.embeddingengine_embed_batch(this.__wbg_ptr, texts);
+        if (ret[2]) {
+            throw takeFromExternrefTable0(ret[1]);
+        }
+        return takeFromExternrefTable0(ret[0]);
+    }
+    /**
+     * Check if the engine is ready for inference
+     * @returns {boolean}
+     */
+    is_ready() {
+        const ret = wasm.embeddingengine_is_ready(this.__wbg_ptr);
+        return ret !== 0;
+    }
+    /**
+     * Load the model and tokenizer from bytes
+     *
+     * This is now the ONLY way to initialize the engine.
+     * Model weights are no longer embedded in WASM for faster initialization.
+     *
+     * # Arguments
+     * * `model_bytes` - SafeTensors format model weights
+     * * `tokenizer_bytes` - tokenizer.json contents
+     * * `config_bytes` - config.json contents
+     * @param {Uint8Array} model_bytes
+     * @param {Uint8Array} tokenizer_bytes
+     * @param {Uint8Array} config_bytes
+     */
+    load(model_bytes, tokenizer_bytes, config_bytes) {
+        const ptr0 = passArray8ToWasm0(model_bytes, wasm.__wbindgen_malloc);
+        const len0 = WASM_VECTOR_LEN;
+        const ptr1 = passArray8ToWasm0(tokenizer_bytes, wasm.__wbindgen_malloc);
+        const len1 = WASM_VECTOR_LEN;
+        const ptr2 = passArray8ToWasm0(config_bytes, wasm.__wbindgen_malloc);
+        const len2 = WASM_VECTOR_LEN;
+        const ret = wasm.embeddingengine_load(this.__wbg_ptr, ptr0, len0, ptr1, len1, ptr2, len2);
+        if (ret[1]) {
+            throw takeFromExternrefTable0(ret[0]);
+        }
+    }
+    /**
+     * Get the maximum sequence length
+     * @returns {number}
+     */
+    max_sequence_length() {
+        const ret = wasm.embeddingengine_max_sequence_length(this.__wbg_ptr);
+        return ret >>> 0;
+    }
+    /**
+     * Create a new embedding engine instance (not loaded)
+     */
+    constructor() {
+        const ret = wasm.embeddingengine_new();
+        this.__wbg_ptr = ret >>> 0;
+        EmbeddingEngineFinalization.register(this, this.__wbg_ptr, this);
+        return this;
+    }
+}
+if (Symbol.dispose) EmbeddingEngine.prototype[Symbol.dispose] = EmbeddingEngine.prototype.free;
+
+/**
+ * Calculate cosine similarity between two embeddings
+ * @param {Float32Array} a
+ * @param {Float32Array} b
+ * @returns {number}
+ */
+export function cosine_similarity(a, b) {
+    const ptr0 = passArrayF32ToWasm0(a, wasm.__wbindgen_malloc);
+    const len0 = WASM_VECTOR_LEN;
+    const ptr1 = passArrayF32ToWasm0(b, wasm.__wbindgen_malloc);
+    const len1 = WASM_VECTOR_LEN;
+    const ret = wasm.cosine_similarity(ptr0, len0, ptr1, len1);
+    return ret;
+}
+
+function __wbg_get_imports() {
+    const import0 = {
+        __proto__: null,
+        __wbg___wbindgen_is_function_0095a73b8b156f76: function(arg0) {
+            const ret = typeof(arg0) === 'function';
+            return ret;
+        },
+        __wbg___wbindgen_is_object_5ae8e5880f2c1fbd: function(arg0) {
+            const val = arg0;
+            const ret = typeof(val) === 'object' && val !== null;
+            return ret;
+        },
+        __wbg___wbindgen_is_string_cd444516edc5b180: function(arg0) {
+            const ret = typeof(arg0) === 'string';
+            return ret;
+        },
+        __wbg___wbindgen_is_undefined_9e4d92534c42d778: function(arg0) {
+            const ret = arg0 === undefined;
+            return ret;
+        },
+        __wbg___wbindgen_string_get_72fb696202c56729: function(arg0, arg1) {
+            const obj = arg1;
+            const ret = typeof(obj) === 'string' ? obj : undefined;
+            var ptr1 = isLikeNone(ret) ? 0 : passStringToWasm0(ret, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
+            var len1 = WASM_VECTOR_LEN;
+            getDataViewMemory0().setInt32(arg0 + 4 * 1, len1, true);
+            getDataViewMemory0().setInt32(arg0 + 4 * 0, ptr1, true);
+        },
+        __wbg___wbindgen_throw_be289d5034ed271b: function(arg0, arg1) {
+            throw new Error(getStringFromWasm0(arg0, arg1));
+        },
+        __wbg_call_389efe28435a9388: function() { return handleError(function (arg0, arg1) {
+            const ret = arg0.call(arg1);
+            return ret;
+        }, arguments); },
+        __wbg_call_4708e0c13bdc8e95: function() { return handleError(function (arg0, arg1, arg2) {
+            const ret = arg0.call(arg1, arg2);
+            return ret;
+        }, arguments); },
+        __wbg_crypto_86f2631e91b51511: function(arg0) {
+            const ret = arg0.crypto;
+            return ret;
+        },
+        __wbg_getRandomValues_b3f15fcbfabb0f8b: function() { return handleError(function (arg0, arg1) {
+            arg0.getRandomValues(arg1);
+        }, arguments); },
+        __wbg_get_9b94d73e6221f75c: function(arg0, arg1) {
+            const ret = arg0[arg1 >>> 0];
+            return ret;
+        },
+        __wbg_length_32ed9a279acd054c: function(arg0) {
+            const ret = arg0.length;
+            return ret;
+        },
+        __wbg_length_35a7bace40f36eac: function(arg0) {
+            const ret = arg0.length;
+            return ret;
+        },
+        __wbg_length_9a7876c9728a0979: function(arg0) {
+            const ret = arg0.length;
+            return ret;
+        },
+        __wbg_msCrypto_d562bbe83e0d4b91: function(arg0) {
+            const ret = arg0.msCrypto;
+            return ret;
+        },
+        __wbg_new_3eb36ae241fe6f44: function() {
+            const ret = new Array();
+            return ret;
+        },
+        __wbg_new_no_args_1c7c842f08d00ebb: function(arg0, arg1) {
+            const ret = new Function(getStringFromWasm0(arg0, arg1));
+            return ret;
+        },
+        __wbg_new_with_length_1763c527b2923202: function(arg0) {
+            const ret = new Array(arg0 >>> 0);
+            return ret;
+        },
+        __wbg_new_with_length_63f2683cc2521026: function(arg0) {
+            const ret = new Float32Array(arg0 >>> 0);
+            return ret;
+        },
+        __wbg_new_with_length_a2c39cbe88fd8ff1: function(arg0) {
+            const ret = new Uint8Array(arg0 >>> 0);
+            return ret;
+        },
+        __wbg_node_e1f24f89a7336c2e: function(arg0) {
+            const ret = arg0.node;
+            return ret;
+        },
+        __wbg_process_3975fd6c72f520aa: function(arg0) {
+            const ret = arg0.process;
+            return ret;
+        },
+        __wbg_prototypesetcall_bdcdcc5842e4d77d: function(arg0, arg1, arg2) {
+            Uint8Array.prototype.set.call(getArrayU8FromWasm0(arg0, arg1), arg2);
+        },
+        __wbg_randomFillSync_f8c153b79f285817: function() { return handleError(function (arg0, arg1) {
+            arg0.randomFillSync(arg1);
+        }, arguments); },
+        __wbg_require_b74f47fc2d022fd6: function() { return handleError(function () {
+            const ret = module.require;
+            return ret;
+        }, arguments); },
+        __wbg_set_f43e577aea94465b: function(arg0, arg1, arg2) {
+            arg0[arg1 >>> 0] = arg2;
+        },
+        __wbg_set_f8edeec46569cc70: function(arg0, arg1, arg2) {
+            arg0.set(getArrayF32FromWasm0(arg1, arg2));
+        },
+        __wbg_static_accessor_GLOBAL_12837167ad935116: function() {
+            const ret = typeof global === 'undefined' ? null : global;
+            return isLikeNone(ret) ? 0 : addToExternrefTable0(ret);
+        },
+        __wbg_static_accessor_GLOBAL_THIS_e628e89ab3b1c95f: function() {
+            const ret = typeof globalThis === 'undefined' ? null : globalThis;
+            return isLikeNone(ret) ? 0 : addToExternrefTable0(ret);
+        },
+        __wbg_static_accessor_SELF_a621d3dfbb60d0ce: function() {
+            const ret = typeof self === 'undefined' ? null : self;
+            return isLikeNone(ret) ? 0 : addToExternrefTable0(ret);
+        },
+        __wbg_static_accessor_WINDOW_f8727f0cf888e0bd: function() {
+            const ret = typeof window === 'undefined' ? null : window;
+            return isLikeNone(ret) ? 0 : addToExternrefTable0(ret);
+        },
+        __wbg_subarray_a96e1fef17ed23cb: function(arg0, arg1, arg2) {
+            const ret = arg0.subarray(arg1 >>> 0, arg2 >>> 0);
+            return ret;
+        },
+        __wbg_versions_4e31226f5e8dc909: function(arg0) {
+            const ret = arg0.versions;
+            return ret;
+        },
+        __wbindgen_cast_0000000000000001: function(arg0, arg1) {
+            // Cast intrinsic for `Ref(Slice(U8)) -> NamedExternref("Uint8Array")`.
+            const ret = getArrayU8FromWasm0(arg0, arg1);
+            return ret;
+        },
+        __wbindgen_cast_0000000000000002: function(arg0, arg1) {
+            // Cast intrinsic for `Ref(String) -> Externref`.
+            const ret = getStringFromWasm0(arg0, arg1);
+            return ret;
+        },
+        __wbindgen_init_externref_table: function() {
+            const table = wasm.__wbindgen_externrefs;
+            const offset = table.grow(4);
+            table.set(0, undefined);
+            table.set(offset + 0, undefined);
+            table.set(offset + 1, null);
+            table.set(offset + 2, true);
+            table.set(offset + 3, false);
+        },
+    };
+    return {
+        __proto__: null,
+        "./candle_embeddings_bg.js": import0,
+    };
+}
+
+const EmbeddingEngineFinalization = (typeof FinalizationRegistry === 'undefined')
+    ? { register: () => {}, unregister: () => {} }
+    : new FinalizationRegistry(ptr => wasm.__wbg_embeddingengine_free(ptr >>> 0, 1));
+
+function addToExternrefTable0(obj) {
+    const idx = wasm.__externref_table_alloc();
+    wasm.__wbindgen_externrefs.set(idx, obj);
+    return idx;
+}
+
+function getArrayF32FromWasm0(ptr, len) {
+    ptr = ptr >>> 0;
+    return getFloat32ArrayMemory0().subarray(ptr / 4, ptr / 4 + len);
+}
+
+function getArrayU8FromWasm0(ptr, len) {
+    ptr = ptr >>> 0;
+    return getUint8ArrayMemory0().subarray(ptr / 1, ptr / 1 + len);
+}
+
+let cachedDataViewMemory0 = null;
+function getDataViewMemory0() {
+    if (cachedDataViewMemory0 === null || cachedDataViewMemory0.buffer.detached === true || (cachedDataViewMemory0.buffer.detached === undefined && cachedDataViewMemory0.buffer !== wasm.memory.buffer)) {
+        cachedDataViewMemory0 = new DataView(wasm.memory.buffer);
+    }
+    return cachedDataViewMemory0;
+}
+
+let cachedFloat32ArrayMemory0 = null;
+function getFloat32ArrayMemory0() {
+    if (cachedFloat32ArrayMemory0 === null || cachedFloat32ArrayMemory0.byteLength === 0) {
+        cachedFloat32ArrayMemory0 = new Float32Array(wasm.memory.buffer);
+    }
+    return cachedFloat32ArrayMemory0;
+}
+
+function getStringFromWasm0(ptr, len) {
+    ptr = ptr >>> 0;
+    return decodeText(ptr, len);
+}
+
+let cachedUint8ArrayMemory0 = null;
+function getUint8ArrayMemory0() {
+    if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
+        cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
+    }
+    return cachedUint8ArrayMemory0;
+}
+
+function handleError(f, args) {
+    try {
+        return f.apply(this, args);
+    } catch (e) {
+        const idx = addToExternrefTable0(e);
+        wasm.__wbindgen_exn_store(idx);
+    }
+}
+
+function isLikeNone(x) {
+    return x === undefined || x === null;
+}
+
+function passArray8ToWasm0(arg, malloc) {
+    const ptr = malloc(arg.length * 1, 1) >>> 0;
+    getUint8ArrayMemory0().set(arg, ptr / 1);
+    WASM_VECTOR_LEN = arg.length;
+    return ptr;
+}
+
+function passArrayF32ToWasm0(arg, malloc) {
+    const ptr = malloc(arg.length * 4, 4) >>> 0;
+    getFloat32ArrayMemory0().set(arg, ptr / 4);
+    WASM_VECTOR_LEN = arg.length;
+    return ptr;
+}
+
+function passStringToWasm0(arg, malloc, realloc) {
+    if (realloc === undefined) {
+        const buf = cachedTextEncoder.encode(arg);
+        const ptr = malloc(buf.length, 1) >>> 0;
+        getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
+        WASM_VECTOR_LEN = buf.length;
+        return ptr;
+    }
+
+    let len = arg.length;
+    let ptr = malloc(len, 1) >>> 0;
+
+    const mem = getUint8ArrayMemory0();
+
+    let offset = 0;
+
+    for (; offset < len; offset++) {
+        const code = arg.charCodeAt(offset);
+        if (code > 0x7F) break;
+        mem[ptr + offset] = code;
+    }
+    if (offset !== len) {
+        if (offset !== 0) {
+            arg = arg.slice(offset);
+        }
+        ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
+        const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
+        const ret = cachedTextEncoder.encodeInto(arg, view);
+
+        offset += ret.written;
+        ptr = realloc(ptr, len, offset, 1) >>> 0;
+    }
+
+    WASM_VECTOR_LEN = offset;
+    return ptr;
+}
+
+function takeFromExternrefTable0(idx) {
+    const value = wasm.__wbindgen_externrefs.get(idx);
+    wasm.__externref_table_dealloc(idx);
+    return value;
+}
+
+let cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });
+cachedTextDecoder.decode();
+const MAX_SAFARI_DECODE_BYTES = 2146435072;
+let numBytesDecoded = 0;
+function decodeText(ptr, len) {
+    numBytesDecoded += len;
+    if (numBytesDecoded >= MAX_SAFARI_DECODE_BYTES) {
+        cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });
+        cachedTextDecoder.decode();
+        numBytesDecoded = len;
+    }
+    return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
+}
+
+const cachedTextEncoder = new TextEncoder();
+
+if (!('encodeInto' in cachedTextEncoder)) {
+    cachedTextEncoder.encodeInto = function (arg, view) {
+        const buf = cachedTextEncoder.encode(arg);
+        view.set(buf);
+        return {
+            read: arg.length,
+            written: buf.length
+        };
+    };
+}
+
+let WASM_VECTOR_LEN = 0;
+
+let wasmModule, wasm;
+function __wbg_finalize_init(instance, module) {
+    wasm = instance.exports;
+    wasmModule = module;
+    cachedDataViewMemory0 = null;
+    cachedFloat32ArrayMemory0 = null;
+    cachedUint8ArrayMemory0 = null;
+    wasm.__wbindgen_start();
+    return wasm;
+}
+
+async function __wbg_load(module, imports) {
+    if (typeof Response === 'function' && module instanceof Response) {
+        if (typeof WebAssembly.instantiateStreaming === 'function') {
+            try {
+                return await WebAssembly.instantiateStreaming(module, imports);
+            } catch (e) {
+                const validResponse = module.ok && expectedResponseType(module.type);
+
+                if (validResponse && module.headers.get('Content-Type') !== 'application/wasm') {
+                    console.warn("`WebAssembly.instantiateStreaming` failed because your server does not serve Wasm with `application/wasm` MIME type. Falling back to `WebAssembly.instantiate` which is slower. Original error:\n", e);
+
+                } else { throw e; }
+            }
+        }
+
+        const bytes = await module.arrayBuffer();
+        return await WebAssembly.instantiate(bytes, imports);
+    } else {
+        const instance = await WebAssembly.instantiate(module, imports);
+
+        if (instance instanceof WebAssembly.Instance) {
+            return { instance, module };
+        } else {
+            return instance;
+        }
+    }
+
+    function expectedResponseType(type) {
+        switch (type) {
+            case 'basic': case 'cors': case 'default': return true;
+        }
+        return false;
+    }
+}
+
+function initSync(module) {
+    if (wasm !== undefined) return wasm;
+
+
+    if (module !== undefined) {
+        if (Object.getPrototypeOf(module) === Object.prototype) {
+            ({module} = module)
+        } else {
+            console.warn('using deprecated parameters for `initSync()`; pass a single object instead')
+        }
+    }
+
+    const imports = __wbg_get_imports();
+    if (!(module instanceof WebAssembly.Module)) {
+        module = new WebAssembly.Module(module);
+    }
+    const instance = new WebAssembly.Instance(module, imports);
+    return __wbg_finalize_init(instance, module);
+}
+
+async function __wbg_init(module_or_path) {
+    if (wasm !== undefined) return wasm;
+
+
+    if (module_or_path !== undefined) {
+        if (Object.getPrototypeOf(module_or_path) === Object.prototype) {
+            ({module_or_path} = module_or_path)
+        } else {
+            console.warn('using deprecated parameters for the initialization function; pass a single object instead')
+        }
+    }
+
+    if (module_or_path === undefined) {
+        module_or_path = new URL('candle_embeddings_bg.wasm', import.meta.url);
+    }
+    const imports = __wbg_get_imports();
+
+    if (typeof module_or_path === 'string' || (typeof Request === 'function' && module_or_path instanceof Request) || (typeof URL === 'function' && module_or_path instanceof URL)) {
+        module_or_path = fetch(module_or_path);
+    }
+
+    const { instance, module } = await __wbg_load(await module_or_path, imports);
+
+    return __wbg_finalize_init(instance, module);
+}
+
+export { initSync, __wbg_init as default };

src/embeddings/wasm/pkg/candle_embeddings_bg.wasm.d.ts

@@ -0,0 +1,19 @@
+/* tslint:disable */
+/* eslint-disable */
+export const memory: WebAssembly.Memory;
+export const __wbg_embeddingengine_free: (a: number, b: number) => void;
+export const cosine_similarity: (a: number, b: number, c: number, d: number) => number;
+export const embeddingengine_dimension: (a: number) => number;
+export const embeddingengine_embed: (a: number, b: number, c: number) => [number, number, number];
+export const embeddingengine_embed_batch: (a: number, b: any) => [number, number, number];
+export const embeddingengine_is_ready: (a: number) => number;
+export const embeddingengine_load: (a: number, b: number, c: number, d: number, e: number, f: number, g: number) => [number, number];
+export const embeddingengine_max_sequence_length: (a: number) => number;
+export const embeddingengine_new: () => number;
+export const __wbindgen_malloc: (a: number, b: number) => number;
+export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
+export const __wbindgen_exn_store: (a: number) => void;
+export const __externref_table_alloc: () => number;
+export const __wbindgen_externrefs: WebAssembly.Table;
+export const __externref_table_dealloc: (a: number) => void;
+export const __wbindgen_start: () => void;

src/errors/brainyError.ts

@@ -10,8 +10,13 @@ export type BrainyErrorType =
   | 'NOT_FOUND'
   | 'RETRY_EXHAUSTED'
   | 'VALIDATION'
+  | 'INVALID_QUERY'
   | 'FIELD_NOT_INDEXED'
   | 'GRAPH_INDEX_NOT_READY'
+  | 'METADATA_INDEX_NOT_READY'
+  | 'VECTOR_INDEX_NOT_READY'
+  | 'PROTECTED_ARTIFACT'
+  | 'DERIVED_ARTIFACT_MISSING'
   | 'MIGRATION_IN_PROGRESS'
 
 /**
@@ -255,6 +260,109 @@ export class GraphIndexNotReadyError extends BrainyError {
     }
 }
 
+/**
+ * Thrown when the metadata field index reports data but cannot serve a KNOWN
+ * persisted field value even after a rebuild — i.e. the `where` / filter
+ * postings did not load on a cold open and could not be restored. The
+ * field-index counterpart of {@link GraphIndexNotReadyError}: it replaces the
+ * silent-empty failure mode (a cold `find({ where })` returning `[]`
+ * indistinguishable from "no such data") with a loud, catchable error, so a
+ * consumer never renders "nothing found" over data that is simply not-yet-warm.
+ *
+ * Detected once per brain by a known-value serving probe on the first filtered
+ * `find()`; brainy self-heals (rebuilds the index from the canonical records)
+ * first and only raises this if the rebuild still cannot serve the known value.
+ */
+export class MetadataIndexNotReadyError extends BrainyError {
+    constructor(message: string, originalError?: Error) {
+        super(message, 'METADATA_INDEX_NOT_READY', false, originalError)
+        this.name = 'MetadataIndexNotReadyError'
+        if (Error.captureStackTrace) {
+            Error.captureStackTrace(this, MetadataIndexNotReadyError)
+        }
+    }
+}
+
+/**
+ * Thrown when the vector index reports vectors (`size() > 0` or, on a native
+ * provider, `isReady() === false`) but cannot return a KNOWN persisted vector
+ * even after a rebuild — i.e. the semantic serving structure did not load on a
+ * cold open and could not be restored. The vector-search counterpart of
+ * {@link GraphIndexNotReadyError} / {@link MetadataIndexNotReadyError}: it
+ * replaces the silent-empty failure mode (a cold `find({ query })` returning
+ * `[]` indistinguishable from "no similar data") with a loud, catchable error,
+ * so a consumer never renders "nothing found" over data that is simply
+ * not-yet-warm.
+ *
+ * Detected once per brain by a known-vector serving probe on the first
+ * semantic / proximity `find()`; brainy self-heals (rebuilds the index from the
+ * canonical records) first and only raises this if the rebuild still cannot
+ * serve the known vector.
+ */
+export class VectorIndexNotReadyError extends BrainyError {
+    constructor(message: string, originalError?: Error) {
+        super(message, 'VECTOR_INDEX_NOT_READY', false, originalError)
+        this.name = 'VectorIndexNotReadyError'
+        if (Error.captureStackTrace) {
+            Error.captureStackTrace(this, VectorIndexNotReadyError)
+        }
+    }
+}
+
+/**
+ * Thrown when a delete (`deleteBinaryBlob` / `removeRawPrefix`) would remove a
+ * blob that is a declared member of a protected derived-index FAMILY (the
+ * registered-blob contract). Declared derived artifacts are undeletable through
+ * the storage layer — this makes an in-process GC / sweeper INCAPABLE of removing
+ * a load-bearing index file (the lost-`main.dkann` class). Intentional retirement
+ * is the explicit `unregisterDerivedFamily(name)` step, then the delete.
+ */
+export class ProtectedArtifactError extends BrainyError {
+    /** The blob key the delete targeted. */
+    public readonly key: string
+    /** The protected family the key belongs to. */
+    public readonly family: string
+    constructor(key: string, family: string) {
+        super(
+            `Refused to delete '${key}': it is a declared member of the protected ` +
+                `derived-index family '${family}'. Declared derived artifacts are undeletable ` +
+                `through the storage layer (COLD ≠ DEAD) — unregisterDerivedFamily('${family}') ` +
+                `first if retirement is intentional.`,
+            'PROTECTED_ARTIFACT',
+            false
+        )
+        this.name = 'ProtectedArtifactError'
+        this.key = key
+        this.family = family
+    }
+}
+
+/**
+ * Raised (loudly) when a declared derived-index family is missing one or more of
+ * its members on open — i.e. a load-bearing blob was deleted OUTSIDE the write
+ * path (an external sweeper the in-process refusal cannot stop). The index must
+ * be rebuilt from canonical; "healthy-while-broken" is impossible because the
+ * missing member is named, not silently tolerated.
+ */
+export class DerivedArtifactMissingError extends BrainyError {
+    /** The family with missing members. */
+    public readonly family: string
+    /** The member keys that are absent. */
+    public readonly missing: string[]
+    constructor(family: string, missing: string[]) {
+        super(
+            `Derived-index family '${family}' is missing ${missing.length} declared ` +
+                `member(s) on open (${missing.join(', ')}) — deleted outside the write path. ` +
+                `The index must be rebuilt from canonical.`,
+            'DERIVED_ARTIFACT_MISSING',
+            false
+        )
+        this.name = 'DerivedArtifactMissingError'
+        this.family = family
+        this.missing = missing
+    }
+}
+
 /**
  * Thrown when a data-plane read or write is issued against a brain that is
  * running its one-time, automatic 7.x → 8.0 on-disk upgrade — the coordinated

src/events/changeFeed.ts

@@ -0,0 +1,171 @@
+/**
+ * @module events/changeFeed
+ * @description The in-process change feed behind {@link Brainy.onChange} — the
+ * authoritative "something committed" signal for every canonical mutation,
+ * regardless of origin (direct API calls, batches, transactions, imports, the
+ * VFS, or an accelerated native deployment: all of them funnel through the
+ * same generation-store commit points this feed is emitted from).
+ *
+ * Design properties:
+ * - **Post-commit only.** Events are enqueued after the commit succeeds, so an
+ *   aborted commit (a losing `ifRev` CAS, a failed transaction) never emits.
+ * - **Commit-ordered.** Events are enqueued in commit order and dispatched
+ *   FIFO, so a subscriber observes mutations in the order they became durable.
+ * - **Never blocks the write path.** Dispatch happens in a microtask after the
+ *   committing call returns; a slow or throwing listener cannot delay or fail
+ *   a write. Listener errors are isolated per listener and per event.
+ * - **Zero cost when unused.** Callers consult {@link ChangeFeed.hasListeners}
+ *   before constructing event payloads; with no subscribers the write path
+ *   does no event work at all.
+ * - **Fire-and-forget.** No acknowledgement, backpressure, or replay. Events
+ *   carry the committed `generation`, so a consumer that needs catch-up
+ *   semantics can pair the live feed with `transactionLog()` / `asOf()`.
+ */
+
+/** The post-commit view of an entity carried by entity change events. */
+export interface ChangeEventEntity {
+  /** The entity's canonical UUID. */
+  id: string
+  /** The entity's NounType string (e.g. `'person'`, `'document'`). */
+  type: string
+  /** The entity's subtype, when set. */
+  subtype?: string
+  /** The entity's custom (indexed) metadata fields. */
+  metadata: Record<string, unknown>
+  /** The writing service, when set. */
+  service?: string
+}
+
+/** The post-commit view of a relationship carried by relation change events. */
+export interface ChangeEventRelation {
+  /** The relationship's canonical UUID. */
+  id: string
+  /** Source entity UUID. */
+  from: string
+  /** Target entity UUID. */
+  to: string
+  /** The relationship's VerbType string (e.g. `'contains'`). */
+  type: string
+  /** The relationship's custom metadata fields, when present. */
+  metadata?: Record<string, unknown>
+}
+
+/**
+ * One committed mutation, as delivered to {@link Brainy.onChange} listeners.
+ *
+ * Exactly one of `entity` / `relation` is populated for `kind: 'entity'` /
+ * `kind: 'relation'` events. `kind: 'store'` events (`clear` / `restore`)
+ * carry neither — they mean "the whole store changed; refetch what you care
+ * about".
+ *
+ * For `op: 'remove'` / `op: 'unrelate'` the payload is the record's LAST
+ * committed state (sourced from the commit's own before-image), so deletes are
+ * fully described rather than id-only.
+ */
+export interface BrainyChangeEvent {
+  /** What changed: one record, one relationship, or the whole store. */
+  kind: 'entity' | 'relation' | 'store'
+  /** The mutation, in Brainy's own API vocabulary. */
+  op:
+    | 'add'
+    | 'update'
+    | 'remove'
+    | 'relate'
+    | 'unrelate'
+    | 'updateRelation'
+    | 'clear'
+    | 'restore'
+  /** The mutated record's id (absent for store-level events). */
+  id?: string
+  /** Post-commit entity view (entity events only). */
+  entity?: ChangeEventEntity
+  /** Post-commit relation view (relation events only). */
+  relation?: ChangeEventRelation
+  /**
+   * The committed generation this mutation belongs to (Model B: every write
+   * is a generation; a transaction's items share one). Absent for store-level
+   * events and init-time bootstrap writes, which are not generation-stamped.
+   */
+  generation?: number
+  /** Commit timestamp (ms since epoch). */
+  timestamp: number
+}
+
+/** Listener signature for {@link Brainy.onChange}. */
+export type ChangeListener = (event: BrainyChangeEvent) => void
+
+/**
+ * A change event as constructed by a mutation method BEFORE its commit: the
+ * commit seam stamps `generation`/`timestamp` after the write becomes
+ * durable. An entity `remove` descriptor may omit `entity` — the seam fills
+ * it from the commit's own before-image (the record's last committed state).
+ */
+export type PendingChangeEvent = Omit<BrainyChangeEvent, 'generation' | 'timestamp'>
+
+/**
+ * @description Listener registry + commit-ordered async dispatcher for
+ * {@link BrainyChangeEvent}s. One instance per {@link Brainy}.
+ */
+export class ChangeFeed {
+  private listeners = new Set<ChangeListener>()
+  private queue: BrainyChangeEvent[] = []
+  private draining = false
+
+  /** Whether any listener is subscribed — the write path's zero-cost gate. */
+  get hasListeners(): boolean {
+    return this.listeners.size > 0
+  }
+
+  /**
+   * @description Subscribe to committed mutations.
+   * @param listener - Called once per committed mutation, in commit order.
+   * @returns An unsubscribe function.
+   */
+  subscribe(listener: ChangeListener): () => void {
+    this.listeners.add(listener)
+    return () => {
+      this.listeners.delete(listener)
+    }
+  }
+
+  /**
+   * @description Enqueue committed events and schedule dispatch. Call ONLY
+   * after the commit has succeeded — this feed must never announce a write
+   * that did not become durable. Safe to call with an empty array.
+   * @param events - The committed mutations, in commit order.
+   */
+  emit(events: BrainyChangeEvent[]): void {
+    if (events.length === 0 || this.listeners.size === 0) return
+    this.queue.push(...events)
+    if (!this.draining) {
+      this.draining = true
+      queueMicrotask(() => this.drain())
+    }
+  }
+
+  /** Deliver everything queued, FIFO, isolating listener errors. */
+  private drain(): void {
+    try {
+      while (this.queue.length > 0) {
+        const event = this.queue.shift()!
+        for (const listener of this.listeners) {
+          try {
+            listener(event)
+          } catch (err) {
+            // A subscriber's bug must never affect the write path or its
+            // sibling subscribers.
+            console.error('[Brainy] onChange listener threw:', err)
+          }
+        }
+      }
+    } finally {
+      this.draining = false
+    }
+  }
+
+  /** Drop all listeners (brain close/teardown). Queued events are discarded. */
+  close(): void {
+    this.listeners.clear()
+    this.queue.length = 0
+  }
+}

src/graph/graphAdjacencyIndex.ts

@@ -195,6 +195,25 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
     return next
   }
 
+  /**
+   * Eager cold-load of the persisted adjacency (readiness contract).
+   *
+   * Loads the LSM manifests + SSTables so `size()` reports the durable edge
+   * count at the rebuild gate — a warm reopen must load the persisted index,
+   * never re-derive it from a full canonical verb scan (the every-boot O(E)
+   * cost this method exists to eliminate). Idempotent; the lazy read paths
+   * call the same `ensureInitialized()` on demand.
+   *
+   * NOTE: the JS index deliberately does NOT expose `isReady()`. That signal
+   * (see `GraphIndexProvider.isReady`) asserts "traversals are trustworthy",
+   * which this side cannot honestly promise without comparing against the
+   * canonical store — the query-time known-edge probe
+   * (`verifyGraphAdjacencyLive` Strategy 2) remains the JS trust check.
+   */
+  async init(): Promise<void> {
+    await this.ensureInitialized()
+  }
+
   /**
    * Initialize the graph index (lazy initialization)
    * Added defensive auto-rebuild check for verbIdSet consistency
@@ -263,6 +282,16 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
       }
 
       hasMore = result.hasMore
+      if (hasMore && (!result.nextCursor || result.nextCursor === cursor)) {
+        // A stalled cursor with hasMore=true would re-read the same page
+        // forever — a silent full-CPU loop at cold open. Abort loudly; a
+        // graph read failing beats a process that spins without a log line.
+        throw new Error(
+          `GraphAdjacencyIndex: verb walk stalled after ${count} verbs — storage returned ` +
+            `hasMore=true with ${result.nextCursor ? 'a non-advancing' : 'no'} cursor. ` +
+            `Aborting the cold-load; run brain.repairIndex() if this persists.`
+        )
+      }
       cursor = result.nextCursor
     }
 
@@ -878,6 +907,11 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
     this.flushTimer = setInterval(async () => {
       await this.flush()
     }, this.config.flushInterval)
+    // Background maintenance must never keep the host process alive —
+    // close()/flush() handle durability; the interval is best-effort.
+    if (typeof this.flushTimer.unref === 'function') {
+      this.flushTimer.unref()
+    }
   }
 
   /**

src/graph/graphAudit.ts

@@ -0,0 +1,214 @@
+/**
+ * @module graph/graphAudit
+ * @description Read-only graph-truth audit — the graph sibling of `repairIndex()`'s
+ * diagnosis half. Verifies three layers against each other without mutating anything:
+ *
+ *   1. CANONICAL verb records (the storage walk — the source of truth)
+ *   2. the RELATIONSHIP READ PATH (`related()` with all visibility tiers — exactly
+ *      what application reads like a VFS `readdir` consult)
+ *   3. ENTITY ENDPOINTS (does each verb's source/target still exist?)
+ *
+ * and classifies every discrepancy into the three failure families production
+ * incidents have shown:
+ *
+ *   - `missingFromReads` — a canonical verb record the read path does NOT return
+ *     for its source: PRESENT BUT INVISIBLE (adjacency/membership staleness).
+ *   - `danglingEndpoints` — a canonical verb whose endpoint entity is gone:
+ *     the SCAR class (write-path loss / partial delete).
+ *   - `readOnlyVerbIds` — the read path returns an edge with NO canonical
+ *     record: GHOST edges (stale index entries).
+ *
+ * `visibilityHiddenCount` is reported separately: an internal/system edge that is
+ * indexed and present but hidden from DEFAULT reads is working as designed — the
+ * audit reads with all tiers included so design-hiding is never misclassified as
+ * index loss.
+ *
+ * Full counts are always exact; only the example LISTS are capped (`maxExamples`)
+ * — a capped report says so via `truncatedExamples`, never silently.
+ */
+
+import { prodLog } from '../utils/logger.js'
+
+/** One discrepant relationship, identified fully enough to inspect by hand. */
+export interface GraphAuditDiscrepancy {
+  verbId: string
+  from: string
+  to: string
+  type: string
+}
+
+export interface GraphAuditReport {
+  /** True iff every discrepancy count is zero — `related()` returns canonical truth. */
+  coherent: boolean
+  verbsInCanonical: number
+  entitiesInCanonical: number
+  /** Distinct source entities whose read path was actually consulted (coverage honesty). */
+  sourcesChecked: number
+
+  /** PRESENT BUT INVISIBLE: canonical records the read path omits. */
+  missingFromReadsCount: number
+  missingFromReads: GraphAuditDiscrepancy[]
+
+  /** SCAR CLASS: canonical verbs with a missing endpoint entity. */
+  danglingEndpointsCount: number
+  danglingEndpoints: Array<GraphAuditDiscrepancy & { missingEnd: 'from' | 'to' | 'both' }>
+
+  /** GHOST EDGES: read-path verb ids with no canonical record. */
+  readOnlyCount: number
+  readOnlyVerbIds: string[]
+
+  /** Canonical verbs hidden from DEFAULT reads by design (internal/system visibility). */
+  visibilityHiddenCount: number
+
+  /** Example lists above were capped at maxExamples; counts remain exact. */
+  truncatedExamples: boolean
+  durationMs: number
+}
+
+/** A canonical verb record, as the audit needs it. */
+export interface AuditVerbRecord {
+  id: string
+  type: string
+  sourceId: string
+  targetId: string
+  visibility?: string
+}
+
+/** The seams the audit runs over — injected so the walk is testable in isolation. */
+export interface GraphAuditDeps {
+  /** Stream every canonical entity id (id-only; no per-entity reads needed). */
+  eachNounId(consume: (id: string) => void): Promise<void>
+  /** Stream every canonical verb record. */
+  eachVerb(consume: (verb: AuditVerbRecord) => void): Promise<void>
+  /**
+   * The END-TO-END relationship read for one source, ALL visibility tiers
+   * included — must be the same path application reads consult.
+   */
+  readRelationsFrom(sourceId: string): Promise<Array<{ id: string }>>
+}
+
+export interface GraphAuditOptions {
+  /** Cap on entries per example list (counts stay exact). Default 100. */
+  maxExamples?: number
+}
+
+export async function runGraphAudit(
+  deps: GraphAuditDeps,
+  options: GraphAuditOptions = {}
+): Promise<GraphAuditReport> {
+  const maxExamples = options.maxExamples ?? 100
+  const started = Date.now()
+
+  // 1. Canonical entity ids — endpoint existence oracle.
+  const entityIds = new Set<string>()
+  await deps.eachNounId((id) => entityIds.add(id))
+
+  // 2. Canonical verb walk: group by source, check endpoints, note visibility.
+  const canonicalVerbIds = new Set<string>()
+  const bySource = new Map<string, AuditVerbRecord[]>()
+  let verbsInCanonical = 0
+  let visibilityHiddenCount = 0
+  let danglingEndpointsCount = 0
+  const danglingEndpoints: GraphAuditReport['danglingEndpoints'] = []
+
+  await deps.eachVerb((verb) => {
+    verbsInCanonical++
+    canonicalVerbIds.add(verb.id)
+    const list = bySource.get(verb.sourceId)
+    if (list) list.push(verb)
+    else bySource.set(verb.sourceId, [verb])
+
+    if (verb.visibility === 'internal' || verb.visibility === 'system') {
+      visibilityHiddenCount++
+    }
+
+    const fromMissing = !entityIds.has(verb.sourceId)
+    const toMissing = !entityIds.has(verb.targetId)
+    if (fromMissing || toMissing) {
+      danglingEndpointsCount++
+      if (danglingEndpoints.length < maxExamples) {
+        danglingEndpoints.push({
+          verbId: verb.id,
+          from: verb.sourceId,
+          to: verb.targetId,
+          type: verb.type,
+          missingEnd: fromMissing && toMissing ? 'both' : fromMissing ? 'from' : 'to'
+        })
+      }
+    }
+  })
+
+  // 3. Per-source read-path comparison. A verb must be returned by the read
+  //    path of ITS OWN source — the exact consult a readdir/traversal makes.
+  let missingFromReadsCount = 0
+  const missingFromReads: GraphAuditDiscrepancy[] = []
+  let readOnlyCount = 0
+  const readOnlyVerbIds: string[] = []
+  const readOnlySeen = new Set<string>()
+
+  for (const [sourceId, verbs] of bySource) {
+    const readIds = new Set((await deps.readRelationsFrom(sourceId)).map((r) => r.id))
+
+    for (const verb of verbs) {
+      if (!readIds.has(verb.id)) {
+        missingFromReadsCount++
+        if (missingFromReads.length < maxExamples) {
+          missingFromReads.push({
+            verbId: verb.id,
+            from: verb.sourceId,
+            to: verb.targetId,
+            type: verb.type
+          })
+        }
+      }
+    }
+
+    for (const readId of readIds) {
+      if (!canonicalVerbIds.has(readId) && !readOnlySeen.has(readId)) {
+        readOnlySeen.add(readId)
+        readOnlyCount++
+        if (readOnlyVerbIds.length < maxExamples) {
+          readOnlyVerbIds.push(readId)
+        }
+      }
+    }
+  }
+
+  const coherent =
+    missingFromReadsCount === 0 && danglingEndpointsCount === 0 && readOnlyCount === 0
+
+  const report: GraphAuditReport = {
+    coherent,
+    verbsInCanonical,
+    entitiesInCanonical: entityIds.size,
+    sourcesChecked: bySource.size,
+    missingFromReadsCount,
+    missingFromReads,
+    danglingEndpointsCount,
+    danglingEndpoints,
+    readOnlyCount,
+    readOnlyVerbIds,
+    visibilityHiddenCount,
+    truncatedExamples:
+      missingFromReadsCount > missingFromReads.length ||
+      danglingEndpointsCount > danglingEndpoints.length ||
+      readOnlyCount > readOnlyVerbIds.length,
+    durationMs: Date.now() - started
+  }
+
+  if (coherent) {
+    prodLog.info(
+      `[GraphAudit] coherent: ${verbsInCanonical} verbs across ${bySource.size} sources — ` +
+        `the read path returns canonical truth (${report.durationMs}ms)`
+    )
+  } else {
+    prodLog.warn(
+      `[GraphAudit] INCOHERENT: ${missingFromReadsCount} present-but-invisible, ` +
+        `${danglingEndpointsCount} dangling-endpoint, ${readOnlyCount} ghost ` +
+        `(of ${verbsInCanonical} canonical verbs, ${bySource.size} sources, ` +
+        `${visibilityHiddenCount} visibility-hidden by design) — ${report.durationMs}ms`
+    )
+  }
+
+  return report
+}

src/graph/lsm/LSMTree.ts

@@ -517,6 +517,11 @@ export class LSMTree {
         }
       }
     }, this.config.compactionInterval)
+    // Background compaction must never keep the host process alive —
+    // close() compacts/flushes deterministically; this interval is best-effort.
+    if (this.compactionTimer && typeof this.compactionTimer.unref === 'function') {
+      this.compactionTimer.unref()
+    }
   }
 
   /**
@@ -542,13 +547,26 @@ export class LSMTree {
         const data = metadata.data as PersistedManifestData
         this.manifest.sstables = new Map(Object.entries(data.sstables || {}))
         this.manifest.lastCompaction = data.lastCompaction || Date.now()
-        this.manifest.totalRelationships = data.totalRelationships || 0
 
-        // Load SSTables from storage
+        // Load SSTables from storage BEFORE publishing the persisted count.
+        // If the SSTable load throws, `size()` must keep reporting 0 — a tree
+        // that claims its persisted relationships while holding none serves
+        // silent-empty traversals as truth (the cold-load swallow class), and
+        // downstream self-heal keys off the honest 0.
         await this.loadSSTables()
+        this.manifest.totalRelationships = data.totalRelationships || 0
       }
     } catch (error) {
-      prodLog.debug('LSMTree: No existing manifest found, starting fresh')
+      // Reset anything partially loaded — an honest empty tree triggers the
+      // rebuild/self-heal paths; a half-loaded one masks them. (An absent
+      // manifest on a fresh store also lands here: empty is correct.)
+      this.manifest.sstables = new Map()
+      this.manifest.totalRelationships = 0
+      this.sstablesByLevel.clear()
+      prodLog.debug(
+        `LSMTree(${this.config.storagePrefix}): no loadable manifest/SSTables — starting empty ` +
+          `(${error instanceof Error ? error.message : String(error)})`
+      )
     }
   }
 
@@ -556,6 +574,7 @@ export class LSMTree {
    * Load SSTables from storage based on manifest
    */
   private async loadSSTables(): Promise<void> {
+    const failures: string[] = []
     const loadPromises: Promise<void>[] = []
 
     this.manifest.sstables.forEach((level, sstableId) => {
@@ -580,7 +599,12 @@ export class LSMTree {
             }
           }
         } catch (error) {
+          // A per-SSTable load failure means the persisted adjacency is INCOMPLETE.
+          // Record it and fail the whole load closed (below): a partially-loaded
+          // tree that still publishes its full manifest count via size() would
+          // serve silent-empty traversals as truth (the cold-load swallow class).
           prodLog.warn(`LSMTree: Failed to load SSTable ${sstableId}`, error)
+          failures.push(sstableId)
         }
       })()
 
@@ -588,6 +612,20 @@ export class LSMTree {
     })
 
     await Promise.all(loadPromises)
+
+    if (failures.length > 0) {
+      // Fail closed. loadManifest()'s catch resets sstables/totalRelationships/
+      // sstablesByLevel to honest-empty, so size() reports 0 and the graph
+      // self-heal (_initializeGraphIndex size()===0 → rebuild) restores the index
+      // from the canonical records. Honest-partial is never published.
+      throw new Error(
+        `LSMTree(${this.config.storagePrefix}): ${failures.length} of ` +
+          `${this.manifest.sstables.size} SSTable(s) failed to load ` +
+          `(${failures.join(', ')}) — failing the load closed so size() reports 0 ` +
+          `and the graph self-heal rebuilds from canonical.`
+      )
+    }
+
     prodLog.info(`LSMTree: Loaded ${this.manifest.sstables.size} SSTables`)
   }
 

src/graph/lsm/SSTable.ts

@@ -20,13 +20,13 @@ import { BloomFilter, SerializedBloomFilter } from './BloomFilter.js'
 import { compareCodePoints } from '../../utils/collation.js'
 
 // Swappable msgpack implementation — defaults to @msgpack/msgpack JS,
-// can be replaced with native msgpack (e.g., cortex's Rust-backed encoder)
+// can be replaced with native msgpack (e.g., cor's Rust-backed encoder)
 let _encode: (data: unknown) => Uint8Array = defaultEncode as (data: unknown) => Uint8Array
 let _decode: (data: Uint8Array) => unknown = defaultDecode as (data: Uint8Array) => unknown
 
 /**
  * Replace the msgpack encode/decode implementation at runtime.
- * Called by brainy.ts when a cortex 'msgpack' provider is registered.
+ * Called by brainy.ts when a cor 'msgpack' provider is registered.
  */
 export function setMsgpackImplementation(impl: { encode: (data: unknown) => Uint8Array, decode: (data: Uint8Array) => unknown }) {
   _encode = impl.encode

src/hnsw/connectionsCodec.ts

@@ -1,7 +1,7 @@
 /**
  * @module hnsw/connectionsCodec
  * @description Bridge between brainy's UUID-keyed HNSW connection sets and
- * cortex's int-keyed delta-varint encode/decode (the `graph:compression`
+ * cor's int-keyed delta-varint encode/decode (the `graph:compression`
  * provider). Translates UUIDs to stable int slots via the post-2.4.0 #1
  * `EntityIdMapper`, batches all of a node's per-level connection lists into
  * a single compact buffer, and reverses the path on load.
@@ -130,7 +130,7 @@ export class ConnectionsCodec {
  * @description Build the binary-blob storage key for a node's compressed
  * connections. Suffix-free; the adapter appends its own. Keys live under
  * `_hnsw_conn/` so they don't collide with the existing `_column_index/`
- * blobs and so a cortex-side reader knows exactly where to look.
+ * blobs and so a cor-side reader knows exactly where to look.
  */
 export function compressedConnectionsKey(nodeId: string): string {
   return `_hnsw_conn/${nodeId}`

src/hnsw/hnswIndex.ts

@@ -25,6 +25,29 @@ const DEFAULT_CONFIG: HNSWConfig = {
   ml: 16 // Max level
 }
 
+/**
+ * @description Thrown by {@link JsHnswVectorIndex.flush} when one or more dirty
+ * nodes (or the system record) could not be persisted. The failed nodes remain
+ * in the dirty set for the next flush; this error tells the caller the flush did
+ * NOT achieve durability instead of a node-count that lies. Mandate: loud
+ * errors, never quiet losses.
+ */
+export class HnswFlushError extends Error {
+  constructor(
+    public readonly failedNodeCount: number,
+    public readonly systemFailed: boolean,
+    public override readonly cause?: Error
+  ) {
+    super(
+      `HNSW flush did not achieve durability: ${failedNodeCount} node(s) failed to ` +
+        `persist${systemFailed ? ' and the system record (entryPoint/maxLevel) failed' : ''}. ` +
+        `Failed nodes remain dirty for retry.` +
+        (cause ? ` First error: ${cause.message}` : '')
+    )
+    this.name = 'HnswFlushError'
+  }
+}
+
 /**
  * Implements {@link VectorIndexProvider}: the vector-index surface Brainy calls
  * on whatever the `'vector'` factory returns (its own `JsHnswVectorIndex`, or a native
@@ -150,41 +173,69 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
     const startTime = Date.now()
     const nodeCount = this.dirtyNodes.size
 
-    // Batch persist all dirty nodes concurrently
+    // Batch persist all dirty nodes concurrently. A node whose connections FAIL
+    // to persist must stay dirty (retried on the next flush) — clearing it would
+    // silently drop the write forever. Track failures; only successfully-
+    // persisted (or deleted) nodes leave the dirty set, so nodes added to it
+    // during this flush are preserved.
+    const failedNodes = new Set<string>()
+    let firstError: Error | null = null
+
     if (this.dirtyNodes.size > 0) {
       const batchSize = 50 // Reasonable batch size for cloud storage
       const nodeIds = Array.from(this.dirtyNodes)
 
       for (let i = 0; i < nodeIds.length; i += batchSize) {
         const batch = nodeIds.slice(i, i + batchSize)
-        const promises = batch.map(nodeId => {
+        const promises = batch.map(async nodeId => {
           const noun = this.nouns.get(nodeId)
-          if (!noun) return Promise.resolve() // Node was deleted
-
-          return this.persistNodeConnections(nodeId, noun).catch(error => {
-            console.error(`[HNSW flush] Failed to persist node ${nodeId}:`, error)
-          })
+          if (!noun) return // Node was deleted — drop it from the dirty set.
+          try {
+            await this.persistNodeConnections(nodeId, noun)
+          } catch (error) {
+            failedNodes.add(nodeId)
+            if (firstError === null) firstError = error as Error
+            prodLog.error(`[HNSW flush] Failed to persist node ${nodeId}: ${(error as Error).message}`)
+          }
         })
 
-        await Promise.allSettled(promises)
+        await Promise.all(promises)
       }
 
-      this.dirtyNodes.clear()
+      // Remove only nodes that were persisted (or deleted mid-flush); keep the
+      // failed ones dirty for the next attempt.
+      for (const nodeId of nodeIds) {
+        if (!failedNodes.has(nodeId)) this.dirtyNodes.delete(nodeId)
+      }
     }
 
-    // Persist system data if dirty
+    // Persist system data if dirty — keep it dirty on failure so the next flush
+    // retries rather than losing the entry-point/maxLevel update.
+    let systemFailed = false
     if (this.dirtySystem) {
+      try {
         await this.storage.saveHNSWSystem({
           entryPointId: this.entryPointId,
           maxLevel: this.maxLevel
-      }).catch(error => {
-        console.error('[HNSW flush] Failed to persist system data:', error)
         })
-
         this.dirtySystem = false
+      } catch (error) {
+        systemFailed = true
+        if (firstError === null) firstError = error as Error
+        prodLog.error(`[HNSW flush] Failed to persist system data: ${(error as Error).message}`)
+      }
     }
 
     const duration = Date.now() - startTime
+
+    // Loud failure: if ANY node or the system record could not be persisted the
+    // flush did not achieve durability. Throw so callers (close(), explicit
+    // flush(), the flush-request watcher) see the failure instead of a success
+    // count that lies. The failed nodes/system stay dirty for retry.
+    if (failedNodes.size > 0 || systemFailed) {
+      throw new HnswFlushError(failedNodes.size, systemFailed, firstError ?? undefined)
+    }
+
     if (nodeCount > 0) {
       prodLog.info(`[HNSW] Flushed ${nodeCount} dirty nodes in ${duration}ms`)
     }
@@ -396,13 +447,14 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
       this.maxLevel = nounLevel
       this.nouns.set(id, noun)
 
-      // Persist system data for first noun (previously skipped)
+      // Persist system data for first noun (previously skipped). Surface a
+      // persist failure loudly — the entry point is the root of the whole index;
+      // silently dropping it while addItem() returns the id would strand every
+      // future search on a rootless index. Mandate: never a quiet loss.
       if (this.storage && this.persistMode === 'immediate') {
         await this.storage.saveHNSWSystem({
           entryPointId: this.entryPointId,
           maxLevel: this.maxLevel
-        }).catch(error => {
-          console.error('Failed to persist initial HNSW system data:', error)
         })
       } else if (this.persistMode === 'deferred') {
         this.dirtySystem = true

src/import/BackgroundDeduplicator.ts

@@ -41,6 +41,14 @@ export interface DeduplicationStats {
  * - Import-scoped deduplication (no cross-contamination)
  * - 3-tier strategy (ID → Name → Similarity)
  * - Uses existing indexes (EntityIdMapper, MetadataIndexManager, TypeAware HNSW)
+ *
+ * Lifecycle: ONE instance per brain, owned by Brainy (getBackgroundDeduplicator)
+ * so the debounce genuinely spans imports and brain.close() cancels pending
+ * work via cancelPending() — this pass merge-DELETES duplicate entities, so it
+ * must never fire against a closed brain. The enableDeduplication gate lives
+ * at the scheduling call site (ImportCoordinator); scheduleDedup itself is
+ * unconditional. The timer is unref'd — a pending pass never holds the
+ * process open.
  */
 export class BackgroundDeduplicator {
   private brain: Brainy
@@ -67,12 +75,15 @@ export class BackgroundDeduplicator {
       clearTimeout(this.debounceTimer)
     }
 
-    // Schedule for 5 minutes from now
+    // Schedule for 5 minutes from now. unref'd: a pending dedup pass must
+    // never hold the process open (exit-hang class) — if the process exits
+    // first, the pass simply never runs; imports are already durable.
     this.debounceTimer = setTimeout(() => {
       this.runBatchDedup().catch(error => {
         prodLog.error('[BackgroundDedup] Batch dedup failed:', error)
       })
     }, 5 * 60 * 1000)
+    this.debounceTimer.unref?.()
   }
 
   /**

src/import/ImportCoordinator.ts

@@ -13,7 +13,6 @@
 import { Brainy } from '../brainy.js'
 import { FormatDetector, SupportedFormat } from './FormatDetector.js'
 import { ImportHistory, type ImportHistoryEntry } from './ImportHistory.js'
-import { BackgroundDeduplicator } from './BackgroundDeduplicator.js'
 import { SmartExcelImporter } from '../importers/SmartExcelImporter.js'
 import { SmartPDFImporter } from '../importers/SmartPDFImporter.js'
 import { SmartCSVImporter } from '../importers/SmartCSVImporter.js'
@@ -112,7 +111,12 @@ export interface ValidImportOptions {
   /** Confidence threshold for entities */
   confidenceThreshold?: number
 
-  /** Enable entity deduplication across imports */
+  /**
+   * Enable entity deduplication (default: true). Gates BOTH passes: the
+   * inline merge during import AND the debounced background pass that runs
+   * ~5 minutes after the last import (which merge-DELETES duplicate entities).
+   * Set false for deployments that must never auto-remove records.
+   */
   enableDeduplication?: boolean
 
   /** Similarity threshold for deduplication (0-1) */
@@ -286,7 +290,6 @@ export class ImportCoordinator {
   private brain: Brainy
   private detector: FormatDetector
   private history: ImportHistory
-  private backgroundDedup: BackgroundDeduplicator
   private excelImporter: SmartExcelImporter
   private pdfImporter: SmartPDFImporter
   private csvImporter: SmartCSVImporter
@@ -300,7 +303,6 @@ export class ImportCoordinator {
     this.brain = brain
     this.detector = new FormatDetector()
     this.history = new ImportHistory(brain)
-    this.backgroundDedup = new BackgroundDeduplicator(brain)
     this.excelImporter = new SmartExcelImporter(brain)
     this.pdfImporter = new SmartPDFImporter(brain)
     this.csvImporter = new SmartCSVImporter(brain)
@@ -1459,9 +1461,16 @@ export class ImportCoordinator {
       }
     }
 
-    // Schedule background deduplication (debounced 5 minutes)
-    if (trackingContext && trackingContext.importId) {
-      this.backgroundDedup.scheduleDedup(trackingContext.importId)
+    // Schedule background deduplication (debounced 5 minutes, brain-owned so
+    // close() can cancel it). Honors the same enableDeduplication gate as the
+    // inline pass — false means NO dedup, inline or background.
+    if (
+      trackingContext &&
+      trackingContext.importId &&
+      options.enableDeduplication !== false
+    ) {
+      const backgroundDedup = await this.brain.getBackgroundDeduplicator()
+      backgroundDedup.scheduleDedup(trackingContext.importId)
     }
 
     return {

src/index.ts

@@ -6,7 +6,7 @@
  * - Brainy: The unified database with Triple Intelligence
  * - Triple Intelligence: Seamless fusion of vector + graph + field search
  * - Db: Immutable, generation-pinned database values (now/transact/asOf/with)
- * - Plugins: Extensible plugin system (cortex, storage adapters)
+ * - Plugins: Extensible plugin system (cor, storage adapters)
  * - Neural Import: AI-powered entity extraction & smart data import
  */
 
@@ -15,8 +15,29 @@ import { Brainy } from './brainy.js'
 
 export { Brainy }
 
+// The in-process change feed (brain.onChange) — event + listener types.
+export type {
+  BrainyChangeEvent,
+  ChangeEventEntity,
+  ChangeEventRelation,
+  ChangeListener
+} from './events/changeFeed.js'
+
+// Temporal VFS — a file version entry (vfs.history / readFile({ asOf })).
+export type { FileVersion } from './vfs/types.js'
+
 // Export diagnostics result type
 export type { DiagnosticsResult } from './brainy.js'
+export type {
+  GraphAuditReport,
+  GraphAuditDiscrepancy
+} from './graph/graphAudit.js'
+export {
+  checkOsLimits,
+  NOFILE_POOL_FLOOR,
+  MAX_MAP_COUNT_POOL_FLOOR
+} from './utils/osLimits.js'
+export type { OsLimitsReport } from './utils/osLimits.js'
 
 // Export Brainy configuration and types
 export type {
@@ -139,7 +160,7 @@ export { EntityNotFoundError, RelationNotFoundError } from './errors/notFound.js
 
 // Base error + typed migration-lock error — thrown by any data-plane call while a
 // brain runs its one-time 7.x→8.0 upgrade; catch to answer HTTP 503 + Retry-After.
-export { BrainyError, MigrationInProgressError, GraphIndexNotReadyError } from './errors/brainyError.js'
+export { BrainyError, MigrationInProgressError, GraphIndexNotReadyError, MetadataIndexNotReadyError, VectorIndexNotReadyError, ProtectedArtifactError, DerivedArtifactMissingError } from './errors/brainyError.js'
 export type { BrainyErrorType } from './errors/brainyError.js'
 
 // ============= 8.0 Db API — generational MVCC =============
@@ -163,8 +184,11 @@ export type {
 export {
   GenerationConflictError,
   SpeculativeOverlayError,
-  GenerationCompactedError
+  GenerationCompactedError,
+  StoreInconsistentError,
+  PendingFlushDurabilityError
 } from './db/errors.js'
+export type { UnreconciledRecord } from './db/errors.js'
 export type {
   TxOperation,
   TxAddOperation,
@@ -177,14 +201,30 @@ export type {
   TxLogEntry,
   CompactHistoryOptions,
   CompactHistoryResult,
+  HistoryStats,
   ChangedIds,
   DiffResult,
   HistoryVersion,
   EntityHistory
 } from './db/types.js'
+// The generation fact log — sequential after-image scan surface
+// (brain.scanFacts / brain.factSegmentPaths) for index heals and replays.
+export type {
+  CommitFact,
+  FactOp,
+  FactScanBatch,
+  SCANFACTS_FIRST_BATCH_MS,
+  FactScanHandle
+} from './db/factLog.js'
+// The generalized family stamp — which source generation a projection
+// reflects + the surface that verifies it whole; one verifier, both member
+// modes (enumerated byte-exact / rollup invariants).
+export { readFamilyStamp, verifyFamilyStamp, ENTITY_TREE_STAMP_PATH } from './db/familyStamp.js'
+export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.js'
 // Optional provider capability for generation-aware native indexes
 export { isVersionedIndexProvider } from './plugin.js'
 export type { VersionedIndexProvider } from './plugin.js'
+export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
 // Optional native graph-acceleration engine (cor 3.0) — the published provider
 // contract + its columnar wire types. Brainy feature-detects an implementation
 // and falls back to its pure-TS adjacency when absent.
@@ -286,7 +326,8 @@ import type {
   HNSWNoun,
   HNSWVerb,
   HNSWConfig,
-  StorageAdapter
+  StorageAdapter,
+  DerivedFamilyDeclaration
 } from './coreTypes.js'
 
 // Export vector index implementation (the JS HNSW path)
@@ -304,7 +345,8 @@ export type {
   HNSWNoun,
   HNSWVerb,
   HNSWConfig,
-  StorageAdapter
+  StorageAdapter,
+  DerivedFamilyDeclaration
 }
 
 // Export graph types

src/indexes/columnStore/ColumnSegmentCursor.ts

@@ -7,7 +7,7 @@
  * Multiple cursors can read the same segment concurrently (e.g. during k-way merge).
  *
  * For the TS baseline, segments are loaded fully into memory and cached via
- * UnifiedCache. The Cortex Rust implementation mmaps the segment file and
+ * UnifiedCache. The Cor Rust implementation mmaps the segment file and
  * indexes into it directly — same read interface, zero-copy access.
  */
 

src/indexes/columnStore/ColumnStore.ts

@@ -69,6 +69,34 @@ interface HeapEntry {
  * await store.flush()
  * const newest = await store.sortTopK('createdAt', 'desc', 10) // bigint[]
  */
+/**
+ * @description Thrown when a segment the manifest lists cannot be loaded —
+ * either it yields no bytes (missing/empty on disk) or its bytes are
+ * undecodable (corruption). Previously such a segment was silently skipped,
+ * dropping ALL of its entities from every `filter`/`rangeQuery`/`sortTopK` with
+ * no error — a manifest↔segments divergence that looked like a short result.
+ * Surfacing it loudly makes the divergence visible and repairable. (A genuine
+ * storage IO fault is a different class — it propagates as the underlying error,
+ * not wrapped in this.)
+ */
+export class ColumnSegmentLoadError extends Error {
+  /** The indexed field whose segment failed to load. */
+  public readonly field: string
+  /** The manifest-listed segment id that could not be loaded. */
+  public readonly segmentId: number | string
+  constructor(field: string, segmentId: number | string, reason: string) {
+    super(
+      `ColumnStore segment ${field}:${segmentId} is listed in the manifest but ` +
+        `could not be loaded (${reason}). The index is inconsistent with its ` +
+        `manifest — rebuild/repair the metadata index rather than trusting a ` +
+        `short query result.`
+    )
+    this.name = 'ColumnSegmentLoadError'
+    this.field = field
+    this.segmentId = segmentId
+  }
+}
+
 export class ColumnStore implements ColumnStoreProvider {
   private storage!: StorageAdapter
   private idMapper!: EntityIdMapper
@@ -476,9 +504,9 @@ export class ColumnStore implements ColumnStoreProvider {
    * Storage path (2.4.0 #4 / cortex-interchange contract):
    *   When the storage adapter exposes the binary-blob primitive
    *   (`saveBinaryBlob` — every brainy adapter ≥7.25.0 does), the segment
-   *   bytes are written as a raw blob at the shared cortex key
+   *   bytes are written as a raw blob at the shared cor key
    *   `_column_index/<field>/L<level>-NNNNNN` (suffix-free; the adapter
-   *   appends its own). This matches byte-for-byte what cortex's
+   *   appends its own). This matches byte-for-byte what cor's
    *   `NativeColumnStore` writes, so JS- and native-written indexes
    *   interchange without re-encoding.
    *
@@ -522,7 +550,7 @@ export class ColumnStore implements ColumnStoreProvider {
       )
 
       if (canUseBlob) {
-        // Raw blob, cortex-shared key convention.
+        // Raw blob, cor-shared key convention.
         const key = `${this.basePath}/${field}/L0-${String(segId).padStart(6, '0')}`
         await storage.saveBinaryBlob!(key, segBuffer)
       } else {
@@ -561,7 +589,7 @@ export class ColumnStore implements ColumnStoreProvider {
     }
 
     // Persist the global deleted bitmap alongside the manifest. Same
-    // raw-blob preference as segments — shared cortex key `<base>/<field>/DELETED`.
+    // raw-blob preference as segments — shared cor key `<base>/<field>/DELETED`.
     const deleted = this.deletedEntities.get(field)
     if (deleted && deleted.size > 0) {
       const serialized = deleted.serialize(true)
@@ -594,14 +622,15 @@ export class ColumnStore implements ColumnStoreProvider {
       let cursor = this.segmentCache.get(cacheKey)
 
       if (!cursor) {
-        const loaded = await this.loadSegmentCursor(field, seg)
-        if (loaded) {
-          cursor = loaded
+        // loadSegmentCursor either returns a cursor or THROWS — a corrupt /
+        // missing manifest-listed segment raises ColumnSegmentLoadError and a
+        // real storage fault propagates, so a listed segment is never silently
+        // dropped from the result set.
+        cursor = await this.loadSegmentCursor(field, seg)
         this.segmentCache.set(cacheKey, cursor)
       }
-      }
 
-      if (cursor) cursors.push(cursor)
+      cursors.push(cursor)
     }
 
     return cursors
@@ -615,11 +644,13 @@ export class ColumnStore implements ColumnStoreProvider {
    * `.cidx` object-path so indexes written before the format unification keep
    * loading correctly. Mirror of cortex's 2.3.1 read-side fallback.
    */
-  private async loadSegmentCursor(field: string, seg: SegmentMeta): Promise<ColumnSegmentCursor | null> {
+  private async loadSegmentCursor(field: string, seg: SegmentMeta): Promise<ColumnSegmentCursor> {
     const manifest = this.manifests.get(field)
-    if (!manifest) return null
+    if (!manifest) {
+      // Defensive: getSegmentCursors guards on the manifest before calling.
+      throw new ColumnSegmentLoadError(field, seg.id, 'no manifest for field')
+    }
 
-    try {
     let buf: Buffer | null = null
 
     const storage = this.storage as unknown as {
@@ -627,7 +658,9 @@ export class ColumnStore implements ColumnStoreProvider {
       readObjectFromPath: (path: string) => Promise<any>
     }
 
-      // Preferred: raw blob at the cortex-shared key.
+    // Preferred: raw blob at the cor-shared key. A real IO fault PROPAGATES —
+    // loadBinaryBlob throws on a fault and returns null only for genuine absence
+    // (a present-but-unreadable segment must not read as "missing").
     if (typeof storage.loadBinaryBlob === 'function') {
       const key = `${this.basePath}/${field}/L${seg.level}-${String(seg.id).padStart(6, '0')}`
       const blob = await storage.loadBinaryBlob(key)
@@ -638,16 +671,24 @@ export class ColumnStore implements ColumnStoreProvider {
     if (!buf) {
       const segPath = manifest.segmentPath(seg.level, seg.id)
       const stored = await storage.readObjectFromPath(segPath)
-        if (!stored) return null
+      if (stored) {
         if (stored._binary && stored.data) {
           buf = Buffer.from(stored.data, 'base64')
         } else if (Buffer.isBuffer(stored)) {
           buf = stored
-        } else {
-          return null
+        }
       }
     }
 
+    // A manifest-listed segment that yields NO loadable bytes is corruption, not
+    // benign absence: swallowing it silently dropped all of the segment's
+    // entities from every filter/rangeQuery/sortTopK with no error. Surface it
+    // loudly so the manifest↔segments divergence is visible (and repairable).
+    if (!buf) {
+      throw new ColumnSegmentLoadError(field, seg.id, 'manifest-listed segment has no loadable bytes')
+    }
+
+    try {
       const parsed = readSegmentFromBuffer(buf)
       return new ColumnSegmentCursor(
         parsed.header,
@@ -655,8 +696,13 @@ export class ColumnStore implements ColumnStoreProvider {
         parsed.entityIds,
         parsed.tombstones
       )
-    } catch {
-      return null
+    } catch (err) {
+      // Undecodable bytes for a listed segment — corruption, not a short result.
+      throw new ColumnSegmentLoadError(
+        field,
+        seg.id,
+        `segment decode failed: ${(err as Error).message}`
+      )
     }
   }
 

src/indexes/columnStore/types.ts

@@ -168,7 +168,7 @@ export const FLAG_MULTI_VALUE = 0x01
  * The plugin-provider contract for the column store.
  *
  * Brainy ships a TypeScript baseline that implements this interface.
- * Cortex registers a Rust-accelerated implementation at higher priority.
+ * Cor registers a Rust-accelerated implementation at higher priority.
  * The MetadataIndex coordinator calls whichever is registered.
  *
  * **8.0 u64 contract — BigInt entity ints at the boundary.** Entity ints flow

src/internals.ts

@@ -16,6 +16,32 @@ export type { EntityIdMapperOptions, EntityIdMapperData } from './utils/entityId
 export { getRecommendedCacheConfig, formatBytes, checkMemoryPressure } from './utils/memoryDetection.js'
 export type { MemoryInfo, CacheAllocationStrategy } from './utils/memoryDetection.js'
 // Entity field resolution — single source of truth for reading fields off
-// HNSWNounWithMetadata. First-party plugins (Cortex) use this to stay in
+// HNSWNounWithMetadata. First-party plugins (Cor) use this to stay in
 // lockstep with the entity shape contract.
 export { resolveEntityField, STANDARD_ENTITY_FIELDS } from './coreTypes.js'
+
+// The generalized family stamp — ONE verifier, both member modes, shared
+// verbatim with native providers so stamp verification is literally one
+// function, never two synchronized copies. Providers write the same shape
+// (their set-swap rewrites stamps, so adoption is migration-free).
+export {
+  readFamilyStamp,
+  writeFamilyStamp,
+  verifyFamilyStamp,
+  FAMILY_STAMPS_PREFIX,
+  ENTITY_TREE_STAMP_PATH
+} from './db/familyStamp.js'
+export type {
+  FamilyStamp,
+  StampMembers,
+  StampVerdict,
+  EnumeratedMember,
+  StampStorage
+} from './db/familyStamp.js'
+
+// Generation fact-log types — the scan surface a provider reaches through the
+// storage capability (`storage.scanFacts` / `storage.factLogHeadGeneration` /
+// `storage.factSegmentPaths`, wired by the host brain at init). Providers
+// never construct a FactLog themselves: open() is writer-side (it reconciles
+// by truncating/rewriting) and there is exactly one writer.
+export type { CommitFact, FactOp, FactScanBatch, FactScanHandle } from './db/factLog.js'

src/neural/embeddedPatterns.ts

@@ -2,7 +2,7 @@
  * 🧠 BRAINY EMBEDDED PATTERNS
  * 
  * AUTO-GENERATED - DO NOT EDIT
- * Generated: 2026-06-11T21:32:56.112Z
+ * Generated: 2026-07-02T21:43:26.976Z
  * Patterns: 220
  * Coverage: 94-98% of all queries
  * 

src/plugin.ts

@@ -2,7 +2,7 @@
  * Brainy Plugin System
  *
  * Simple plugin architecture for two use cases:
- * 1. Native acceleration (@soulcraft/cortex)
+ * 1. Native acceleration (@soulcraft/cor)
  * 2. Custom storage adapters (e.g., Redis, DynamoDB, custom backends)
  *
  * Plugins are loaded from an explicit `plugins: [...]` config list or
@@ -20,7 +20,7 @@ import type { MetadataIndexStats } from './utils/metadataIndex.js'
 import type { GraphIndexStats } from './graph/graphAdjacencyIndex.js'
 
 // Re-export the provider contracts that already live closer to their
-// implementations so a plugin author (Cortex) can import the *entire*
+// implementations so a plugin author (Cor) can import the *entire*
 // provider surface from one stable entrypoint: `@soulcraft/brainy/plugin`.
 export type { ColumnStoreProvider } from './indexes/columnStore/types.js'
 export type {
@@ -75,7 +75,7 @@ export interface BrainyPluginContext {
   /**
    * Register a provider for a named subsystem.
    *
-   * Well-known provider keys (used by cortex):
+   * Well-known provider keys (used by cor):
    * - 'metadataIndex'      — MetadataIndexManager replacement
    * - 'graphIndex'         — GraphAdjacencyIndex replacement
    * - 'entityIdMapper'     — EntityIdMapper replacement
@@ -103,7 +103,7 @@ export interface BrainyPluginContext {
 //
 // These interfaces are the type-level half of the provider-parity guarantee.
 // They capture only what Brainy actually invokes, so a native accelerator
-// (Cortex) that declares `implements MetadataIndexProvider` gets a compile
+// (Cor) that declares `implements MetadataIndexProvider` gets a compile
 // error the moment a method Brainy depends on is dropped or its signature
 // drifts. Brainy's own baseline classes (`MetadataIndexManager`,
 // `GraphAdjacencyIndex`, `JsHnswVectorIndex`, `EntityIdMapper`, `UnifiedCache`)
@@ -115,6 +115,62 @@ export interface BrainyPluginContext {
 // implementation then fails to compile until it provides the member.
 // ===========================================================================
 
+/**
+ * @description How a failed provider invariant should be remediated:
+ * - `'none'`    — informational; the invariant held or nothing to do.
+ * - `'repair'`  — a targeted, cheap fix exists (e.g. re-derive a count/manifest field).
+ * - `'rebuild'` — the derived state must be rebuilt from canonical (`provider.rebuild()`).
+ */
+export type InvariantHeal = 'none' | 'repair' | 'rebuild'
+
+/**
+ * @description The result of ONE provider invariant check. A failure
+ * (`holds === false`) NAMES what diverged, with numbers, so it is diagnosable
+ * from the report alone — never a bare boolean. `name` is a stable kebab-case id
+ * for telemetry / remediation routing.
+ */
+export interface InvariantResult {
+  /** Stable kebab-case id, e.g. `'manifest-residency'` / `'posted-count-floor'`. */
+  name: string
+  /** `true` when the invariant holds. */
+  holds: boolean
+  /** Human-readable detail; on failure, names the divergence WITH numbers. */
+  detail: string
+  /** The value the invariant expected (optional, for diagnosis). */
+  expected?: unknown
+  /** The value actually observed (optional, for diagnosis). */
+  actual?: unknown
+  /** How a failure should be remediated. Ignored when `holds === true`. */
+  heal: InvariantHeal
+}
+
+/**
+ * @description A provider's self-report of its own cross-layer invariants
+ * (the `validateInvariants()` hook). Contract:
+ * - It NEVER throws — a failure is DATA (`healthy: false` + a failing invariant),
+ *   not an exception.
+ * - It is BOUNDED (<50ms): residency checks + O(1) counts only, NO canonical
+ *   walks — safe to call on a live brain, repeatedly.
+ * - `serving` = can the provider answer queries right now (the `isReady()` truth);
+ *   `healthy` = do ALL invariants hold. A provider can be `serving` while an
+ *   invariant flags a latent divergence, or `healthy` but not-yet-`serving` on a
+ *   cold open.
+ */
+export interface ProviderInvariantReport {
+  /** Which provider produced this report, e.g. `'vector'` / `'graph'` / `'metadata'` / `'column'`. */
+  provider: string
+  /** `true` iff every invariant in {@link invariants} holds. */
+  healthy: boolean
+  /** `true` iff the provider can serve queries now (the `isReady()` truth). */
+  serving: boolean
+  /** Each checked invariant and its verdict. */
+  invariants: InvariantResult[]
+  /** Epoch millis when the check ran. */
+  checkedAt: number
+  /** How long the check took (must stay well under 50ms). */
+  durationMs: number
+}
+
 /**
  * The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`.
  * Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
@@ -125,6 +181,32 @@ export interface MetadataIndexProvider {
   flush(): Promise<void>
   rebuild(): Promise<void>
 
+  /**
+   * @description OPTIONAL honest durability signal (readiness contract,
+   * mirrors `isReady?()` on the graph and vector providers). `true` ⇔ the
+   * persisted field postings are loaded (or cheaply demand-loadable) and
+   * consistent with what the provider last persisted — a rebuild from the
+   * canonical records would be redundant. When exposed, the rebuild gate
+   * defers to this signal INSTEAD of the `getStats().totalEntries === 0`
+   * heuristic (a durable provider may legitimately report 0 resident entries
+   * on a cold open while its postings sit loadable on disk). Absent → the
+   * gate keeps the count heuristic. Never return `true` when the durable
+   * state failed to load.
+   */
+  isReady?(): boolean
+
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL. A native provider returns true from the moment its
    * `init()` detects a large epoch-drift until its background
@@ -275,6 +357,18 @@ export interface GraphIndexProvider {
    */
   isReady?(): boolean
 
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL eager cold-load. Called once during brain init — AFTER
    * the metadata provider's `init()` (so the id-mapper is hydrated; a native int
@@ -840,7 +934,7 @@ export interface AtGenerationVectors {
 /**
  * The object returned by the `'vector'` provider factory — Brainy's vector
  * index contract. Implementations include Brainy's own JS HNSW index and any
- * native acceleration provider (e.g. cortex's Adaptive DiskANN).
+ * native acceleration provider (e.g. cor's Adaptive DiskANN).
  *
  * Brainy calls this surface via `this.index.*` plus the transactional add/remove
  * operations. `enableCOW`, `getItem`, and `setPersistMode` are intentionally
@@ -902,6 +996,45 @@ export interface VectorIndexProvider {
   flush(): Promise<number>
   getPersistMode(): 'immediate' | 'deferred'
 
+  /**
+   * @description OPTIONAL eager cold-load (readiness contract, mirrors
+   * {@link GraphIndexProvider.init}). Called once during brain init — AFTER the
+   * metadata provider's `init()` (the id-mapper is hydrated first, so a
+   * provider whose vector slots resolve through interned ints reads a complete
+   * mapping) and BEFORE the rebuild gate — so a durable provider loads (or
+   * verifies it can demand-load) its persisted index and reports
+   * `isReady() === true` at the gate instead of eating a spurious
+   * rebuild-from-canonical on every open. The built-in JS index omits it:
+   * `rebuild()` IS its load path.
+   */
+  init?(): Promise<void>
+
+  /**
+   * @description OPTIONAL honest durability signal (readiness contract,
+   * mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted
+   * derived index is loaded (or cheaply demand-loadable) and consistent with
+   * what the provider last persisted — a rebuild from the canonical records
+   * would be redundant work. When exposed, the rebuild gate defers to this
+   * signal INSTEAD of the `size() === 0` heuristic (an mmap/disk-native index
+   * may legitimately report 0 resident entries while fully durable). Absent →
+   * the gate keeps the size heuristic. Never return `true` when the durable
+   * state failed to load — that converts a recoverable rebuild into silent
+   * empty results.
+   */
+  isReady?(): boolean
+
+  /**
+   * @description OPTIONAL. The provider's self-report of its own
+   * cross-layer invariants (manifest ↔ segments ↔ counts residency/coherence).
+   * MUST NOT throw — a failure is DATA (`healthy: false` + a failing invariant).
+   * MUST be bounded (<50ms): residency + O(1) counts only, NO canonical walks, so
+   * brainy's {@link } `validateIndexConsistency()` can call it on a live brain.
+   * Absent → brainy skips this provider in the cross-layer check (feature-detected).
+   * `repairIndex()` maps any failing invariant with `heal: 'rebuild'` to this
+   * provider's `rebuild()`.
+   */
+  validateInvariants?(): Promise<ProviderInvariantReport>
+
   /**
    * @description OPTIONAL. A native provider returns true from the moment its
    * `init()` detects a large epoch-drift until its background
@@ -980,7 +1113,7 @@ export interface CacheProvider {
 
 /**
  * The `'graph:compression'` provider — pure-function encode/decode for HNSW
- * connection lists as compact delta-varint byte sequences (cortex's
+ * connection lists as compact delta-varint byte sequences (cor's
  * `encodeConnections` / `decodeConnections`).
  *
  * Brainy's `JsHnswVectorIndex` consumes this via a `ConnectionsCodec` that translates

src/storage/adapters/baseStorageAdapter.ts

@@ -61,7 +61,7 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
 
   // Vector Index Persistence (Brainy 8.0 — algorithm-neutral surface).
   // Concrete adapters persist the per-entity HNSW graph node (level + connections)
-  // under these methods. Cortex's DiskANN-style native vector index doesn't use
+  // under these methods. Cor's DiskANN-style native vector index doesn't use
   // them (it persists its own single mmap'd `.dkann` file under
   // `_system/vector-index/<shard>.dkann` per BRAINY-8.0-RENAME-COORDINATION § B.2).
 
@@ -431,6 +431,12 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
     this.statisticsBatchUpdateTimerId = setTimeout(() => {
       this.flushStatistics()
     }, delayMs)
+    // Best-effort statistics flush — must not keep the process alive
+    // (close() flushes counts deterministically).
+    const statsTimer = this.statisticsBatchUpdateTimerId as unknown as { unref?: () => void }
+    if (statsTimer && typeof statsTimer.unref === 'function') {
+      statsTimer.unref()
+    }
   }
 
   /**

src/storage/adapters/memoryStorage.ts

@@ -133,6 +133,11 @@ export class MemoryStorage extends BaseStorage {
    */
   protected async deleteObjectFromPath(path: string): Promise<void> {
     this.objectStore.delete(path)
+    // Filesystem parity: on disk, objects and raw BYTE files are both just
+    // files — unlink removes whichever exists. Without this, deleteRawObject
+    // on a raw-bytes path (fact-log/generation segments) silently no-ops on
+    // memory storage: the delete "succeeds" and the bytes remain.
+    this.rawBytesStore.delete(path)
   }
 
   /**
@@ -183,6 +188,9 @@ export class MemoryStorage extends BaseStorage {
    * @param key - The blob key.
    */
   public async deleteBinaryBlob(key: string): Promise<void> {
+    // Registered-blob contract — parity with the filesystem adapter:
+    // a declared family member is undeletable (throws ProtectedArtifactError).
+    await this.assertBlobKeyDeletable(key)
     this.blobStore.delete(key)
   }
 
@@ -219,6 +227,45 @@ export class MemoryStorage extends BaseStorage {
     return [...this.txLogLines]
   }
 
+  // ===========================================================================
+  // Binary raw-byte primitives — in-memory mirror of the filesystem adapter's
+  // append-only substrate (the generation fact log's segments), so memory
+  // brains dual-write facts too and the compat suite runs on both adapters.
+  // ===========================================================================
+
+  /** Raw binary files, keyed by verbatim path. */
+  private rawBytesStore: Map<string, Uint8Array> = new Map()
+
+  /** Append bytes to a raw binary file, creating it when absent. */
+  public async appendRawBytes(rawPath: string, bytes: Uint8Array): Promise<void> {
+    const existing = this.rawBytesStore.get(rawPath)
+    if (!existing) {
+      this.rawBytesStore.set(rawPath, bytes.slice())
+      return
+    }
+    const merged = new Uint8Array(existing.length + bytes.length)
+    merged.set(existing, 0)
+    merged.set(bytes, existing.length)
+    this.rawBytesStore.set(rawPath, merged)
+  }
+
+  /** Read a raw binary file whole (a copy); absent → null. */
+  public async readRawBytes(rawPath: string): Promise<Uint8Array | null> {
+    const bytes = this.rawBytesStore.get(rawPath)
+    return bytes ? bytes.slice() : null
+  }
+
+  /** Replace a raw binary file (atomic by construction in memory). */
+  public async writeRawBytes(rawPath: string, bytes: Uint8Array): Promise<void> {
+    this.rawBytesStore.set(rawPath, bytes.slice())
+  }
+
+  /** Byte size of a raw binary file, or null when absent. */
+  public async rawByteSize(rawPath: string): Promise<number | null> {
+    const bytes = this.rawBytesStore.get(rawPath)
+    return bytes ? bytes.length : null
+  }
+
   /**
    * Serialize the entire in-memory store to a directory in the exact layout
    * the filesystem adapter uses (uncompressed JSON objects, `_blobs/*.bin`
@@ -351,6 +398,7 @@ export class MemoryStorage extends BaseStorage {
   public async clear(): Promise<void> {
     this.objectStore.clear()
     this.blobStore.clear()
+    this.rawBytesStore.clear()
     this.txLogLines = []
     this.statistics = null
 

src/storage/blobStorage.ts

@@ -15,6 +15,7 @@
 
 import { createHash } from 'crypto'
 import { unwrapBinaryData } from './binaryDataCodec.js'
+import { InMemoryMutex } from '../utils/mutex.js'
 
 /**
  * @description Key-value bridge the blob store persists through. Implemented
@@ -47,8 +48,19 @@ export interface BlobMetadata {
   compression: 'none' | 'zstd'
   /** Creation timestamp (epoch ms). */
   createdAt: number
-  /** Number of logical references to this blob (deduplicated writes). */
+  /** Number of LIVE logical references to this blob (deduplicated writes). */
   refCount: number
+  /**
+   * Number of persisted generation record-sets (Model-B before-images) that
+   * reference this hash — the blob's membership in the temporal history.
+   * Bytes are physically reclaimed only when BOTH counts are zero, and only
+   * by history compaction: live references protect the present, history
+   * references protect every `asOf` read inside the retention window (pins
+   * ride generation pinning, which compaction already respects). Absent on
+   * metas written before the temporal contract existed (treated as 0; the
+   * one-time open-time backfill makes legacy stores exact).
+   */
+  historyRefCount?: number
 }
 
 /**
@@ -147,7 +159,9 @@ interface CacheEntry {
  * @example
  * const hash = await blobStorage.write(buffer, { mimeType: 'image/png' })
  * const bytes = await blobStorage.read(hash) // verified against the hash
- * await blobStorage.delete(hash)             // decrements refCount first
+ * await blobStorage.release(hash)            // drop one LIVE reference
+ * // bytes are physically reclaimed only by history compaction, once no
+ * // live reference AND no in-window generation references the hash
  */
 export class BlobStorage {
   private adapter: BlobStoreAdapter
@@ -164,6 +178,19 @@ export class BlobStorage {
   private readonly CACHE_MAX_SIZE = 100 * 1024 * 1024 // 100MB default
   private readonly COMPRESSION_THRESHOLD = 1024 // 1KB - don't compress smaller
 
+  /**
+   * Per-hash write serialization. Every reference-count-bearing mutation
+   * (`write`'s dedup check-then-act, `delete`'s decrement-then-maybe-remove)
+   * is a read-modify-write over `blob-meta:<hash>` — unserialized, two
+   * concurrent writes of identical content both saw "absent" and both wrote
+   * `refCount: 1` (one reference lost → a later delete removed bytes another
+   * file still referenced), and concurrent increments/decrements could drop
+   * counts. Keyed by hash, so distinct content never contends; the process
+   * is the whole concurrency domain (storage enforces single-writer per
+   * directory).
+   */
+  private readonly hashLocks = new InMemoryMutex()
+
   /**
    * @param adapter - Key-value bridge to persist through.
    * @param options - `cacheMaxSize` bounds the LRU read cache (bytes,
@@ -222,6 +249,13 @@ export class BlobStorage {
   async write(data: Buffer, options: BlobWriteOptions = {}): Promise<string> {
     const hash = BlobStorage.hash(data)
 
+    // The dedup decision (exists → add a reference; absent → create with
+    // refCount 1) is check-then-act over the same metadata a concurrent
+    // same-content write mutates — serialized per hash so N concurrent
+    // writes of identical content yield exactly N references, never a lost
+    // count (a lost reference turns a later delete into premature removal
+    // of bytes another file still needs).
+    return this.hashLocks.runExclusive(hash, async () => {
       // Deduplication: identical content already stored — just add a reference.
       if (await this.has(hash)) {
         await this.incrementRefCount(hash)
@@ -261,6 +295,7 @@ export class BlobStorage {
       this.addToCache(hash, data, metadata)
 
       return hash
+    })
   }
 
   /**
@@ -334,23 +369,105 @@ export class BlobStorage {
   }
 
   /**
-   * @description Drop one reference to the blob. The stored bytes and
-   * metadata are physically deleted only when the reference count reaches
-   * zero — deduplicated content shared by other writers survives.
+   * @description Drop one LIVE reference to the blob. Never deletes bytes —
+   * blob content is immutable under the temporal model, exactly like every
+   * other record: a past generation's `asOf` read may still need these bytes
+   * even when no live file references them. Physical reclamation happens in
+   * ONE place only — history compaction via {@link reclaimIfUnreferenced},
+   * once no live reference AND no retained generation references the hash.
    *
    * @param hash - The blob's SHA-256 hash.
    */
-  async delete(hash: string): Promise<void> {
-    const refCount = await this.decrementRefCount(hash)
-
-    // Only delete if no references remain
-    if (refCount > 0) {
-      return
+  async release(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      await this.decrementRefCount(hash)
+    })
   }
 
+  /**
+   * @description Record that one persisted generation record-set references
+   * this hash (called by the commit path BEFORE the record-set is written —
+   * a crash between the two can only over-count, which leaks until the scrub
+   * recounts; it can never under-count, which would risk premature deletion).
+   * A missing meta (bytes never stored or already gone) is skipped with a
+   * warning — counting it could not make its bytes readable.
+   * @param hash - The blob's SHA-256 hash.
+   */
+  async recordHistoryReference(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) {
+        console.warn(
+          `[BlobStorage] history reference recorded for absent blob ${hash} — skipped`
+        )
+        return
+      }
+      metadata.historyRefCount = (metadata.historyRefCount ?? 0) + 1
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
+
+  /**
+   * @description Drop one history reference (called by compaction AFTER the
+   * referencing generation record-set is deleted — the safe ordering: a crash
+   * between the two over-counts, never under-counts). Floored at zero.
+   * @param hash - The blob's SHA-256 hash.
+   */
+  async releaseHistoryReference(hash: string): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return
+      metadata.historyRefCount = Math.max(0, (metadata.historyRefCount ?? 0) - 1)
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
+
+  /**
+   * @description Physically delete the blob's bytes + metadata IFF nothing
+   * references it: zero live references AND zero history references. The one
+   * reclamation point in the system, invoked by history compaction after it
+   * releases the reclaimed generations' references. Atomic per hash.
+   * @param hash - The blob's SHA-256 hash.
+   * @returns `true` when the bytes were reclaimed.
+   */
+  async reclaimIfUnreferenced(hash: string): Promise<boolean> {
+    return this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return false
+      if ((metadata.refCount ?? 0) > 0 || (metadata.historyRefCount ?? 0) > 0) {
+        return false
+      }
       await this.adapter.delete(`blob:${hash}`)
       await this.adapter.delete(`blob-meta:${hash}`)
       this.removeFromCache(hash)
+      return true
+    })
+  }
+
+  /**
+   * @description Set the history reference count to an absolute value — the
+   * backfill/scrub primitive (recounts derived from the actual generation
+   * records replace whatever the incremental counters hold). Idempotent.
+   * @param hash - The blob's SHA-256 hash.
+   * @param count - The exact history reference count.
+   */
+  async setHistoryRefCount(hash: string, count: number): Promise<void> {
+    await this.hashLocks.runExclusive(hash, async () => {
+      const metadata = await this.getMetadata(hash)
+      if (!metadata) return
+      metadata.historyRefCount = Math.max(0, count)
+      await this.adapter.put(`blob-meta:${hash}`, Buffer.from(JSON.stringify(metadata)))
+    })
+  }
+
+  /**
+   * @description Enumerate every stored blob hash (from the metadata keys) —
+   * the backfill/scrub walk. O(stored blobs).
+   * @returns All hashes with a stored metadata record.
+   */
+  async listHashes(): Promise<string[]> {
+    const keys = await this.adapter.list('blob-meta:')
+    return keys.map((k) => k.slice('blob-meta:'.length))
   }
 
   /**
@@ -403,6 +520,8 @@ export class BlobStorage {
 
   /**
    * Increment the reference count for an existing blob.
+   * Caller MUST hold the per-hash lock ({@link hashLocks}) — this is a raw
+   * read-modify-write with no serialization of its own.
    */
   private async incrementRefCount(hash: string): Promise<number> {
     const metadata = await this.getMetadata(hash)
@@ -417,6 +536,8 @@ export class BlobStorage {
 
   /**
    * Decrement the reference count for a blob (floored at zero).
+   * Caller MUST hold the per-hash lock ({@link hashLocks}) — this is a raw
+   * read-modify-write with no serialization of its own.
    */
   private async decrementRefCount(hash: string): Promise<number> {
     const metadata = await this.getMetadata(hash)

src/transaction/Transaction.ts

@@ -37,6 +37,24 @@ const DEFAULT_OPTIONS: Required<TransactionOptions> = {
   maxRollbackRetries: 3
 }
 
+/**
+ * The apply-phase budget for a batch of `opCount` operations.
+ *
+ * An explicit override wins untouched. Otherwise the budget SCALES with the
+ * batch: `max(30 000 ms, opCount × 2 000 ms)`. The per-op term is calibrated
+ * from field data — bulk imports on network-attached disks measure ~2 s per
+ * operation (each op pays canonical writes + fsync + index maintenance) — so
+ * a flat 30 s budget silently capped honest work at ~15 operations while
+ * looking generous for small batches. Scaling keeps small transacts
+ * fast-failing and gives bulk ones a budget proportional to the work they
+ * actually asked for; a trip still rolls back atomically and throws a
+ * retryable, fully-labeled TransactionTimeoutError.
+ */
+export function transactTimeoutBudget(opCount: number, override?: number): number {
+  if (override !== undefined) return override
+  return Math.max(30_000, opCount * 2_000)
+}
+
 /**
  * Transaction class
  */
@@ -98,49 +116,64 @@ export class Transaction implements TransactionContext {
     }
 
     try {
-      // Execute each operation in order
+      // Execute each operation in order. This loop is the sole rollback-guarded
+      // region: ANY error that escapes it — an operation failure OR a
+      // mid-flight timeout — is caught below and rolls back every operation
+      // applied so far, in reverse order. That single guarantee is the
+      // transaction's atomicity contract, and the generation-store commit path
+      // depends on it: its abort cleanup assumes a throw from here already
+      // restored the applied operations byte-identically (it only discards the
+      // uncommitted staging directory). A rollback per exit path — the previous
+      // design — let the timeout throw slip past rollback and strand the
+      // already-applied writes as torn, generation-less state in canonical
+      // storage.
       for (let i = 0; i < this.operations.length; i++) {
-        // Check timeout
+        // Budget check BEFORE starting the next operation. A trip here throws
+        // into the catch below and rolls back like any other failure — it must
+        // never bypass rollback.
         if (Date.now() - this.startTime > this.options.timeout) {
-          throw new TransactionTimeoutError(this.options.timeout, i)
+          throw new TransactionTimeoutError(this.options.timeout, i, {
+            elapsedMs: Date.now() - this.startTime,
+            totalOperations: this.operations.length,
+            operationName: this.operations[i]?.name
+          })
         }
 
         const operation = this.operations[i]
 
-        try {
         if (this.options.logging) {
           prodLog.info(`[Transaction] Executing operation ${i}: ${operation.name || 'unnamed'}`)
         }
 
-          // Execute operation
-          const rollbackAction = await operation.execute()
-
-          // Record rollback action (if provided)
-          if (rollbackAction) {
-            this.rollbackActions.push(rollbackAction)
-          }
-
+        let rollbackAction: RollbackAction | undefined
+        try {
+          rollbackAction = await operation.execute()
         } catch (error) {
-          // Operation failed - rollback and re-throw
-          const executionError = new TransactionExecutionError(
+          // Normalize an operation failure to a TransactionExecutionError; the
+          // outer catch performs the (single) rollback and surfaces it.
+          throw new TransactionExecutionError(
             `Operation ${i} failed: ${(error as Error).message}`,
             i,
             operation.name,
             error as Error
           )
-
-          await this.rollback(executionError)
-          throw executionError
-        }
         }
 
-      // All operations succeeded - commit
-      this.commit()
-
+        // Record rollback action (if the operation provided one)
+        if (rollbackAction) {
+          this.rollbackActions.push(rollbackAction)
+        }
+      }
     } catch (error) {
-      // Error already handled in rollback
+      // Single rollback point: undo everything applied so far, in reverse
+      // order, then surface the original error. A rollback failure supersedes
+      // it (rollback() throws TransactionRollbackError wrapping this error).
+      await this.rollback(error as Error)
       throw error
     }
+
+    // All operations succeeded — commit.
+    this.commit()
   }
 
   /**
@@ -208,15 +241,22 @@ export class Transaction implements TransactionContext {
       }
     }
 
-    this.state = 'rolled_back'
+    // Honest terminal state: only claim 'rolled_back' when EVERY undo applied.
+    // If any undo failed, the store is not cleanly rolled back — mark
+    // 'inconsistent' so the commit orchestration reconciles rather than trusts
+    // a false 'rolled_back'. (Fixes the state half of the post-commit response
+    // lie: the record whose undo failed is still durable.)
+    this.state = rollbackErrors.length > 0 ? 'inconsistent' : 'rolled_back'
     this.endTime = Date.now()
 
     if (this.options.logging) {
       const duration = this.endTime - (this.startTime || this.endTime)
-      prodLog.info(`[Transaction] Rolled back in ${duration}ms`)
+      prodLog.info(`[Transaction] ${this.state === 'inconsistent' ? 'Rollback INCOMPLETE' : 'Rolled back'} in ${duration}ms`)
     }
 
-    // If rollback encountered errors, wrap them with original error
+    // If rollback encountered errors, wrap them with original error. The
+    // orchestration keys on `instanceof TransactionRollbackError` to know the
+    // undo did not fully apply and reconciliation is required.
     if (rollbackErrors.length > 0) {
       throw new TransactionRollbackError(
         `Transaction rollback encountered ${rollbackErrors.length} errors during cleanup`,

src/transaction/errors.ts

@@ -77,11 +77,24 @@ export class InvalidTransactionStateError extends TransactionError {
 export class TransactionTimeoutError extends TransactionError {
   constructor(
     timeoutMs: number,
-    operationIndex: number
+    operationIndex: number,
+    telemetry?: {
+      elapsedMs?: number
+      totalOperations?: number
+      operationName?: string
+    }
   ) {
+    const progress =
+      telemetry?.totalOperations !== undefined
+        ? `${operationIndex}/${telemetry.totalOperations}`
+        : String(operationIndex)
+    const name = telemetry?.operationName ? ` ('${telemetry.operationName}')` : ''
+    const elapsed =
+      telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
     super(
-      `Transaction timed out after ${timeoutMs}ms at operation ${operationIndex}`,
-      { timeoutMs, operationIndex }
+      `Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
+        `The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`,
+      { timeoutMs, operationIndex, ...telemetry }
     )
     this.name = 'TransactionTimeoutError'
   }

src/transaction/operations/IndexOperations.ts

@@ -176,14 +176,36 @@ export class RemoveFromMetadataIndexOperation implements Operation {
  * generation is reused for the rollback removal, so an add and its undo
  * reference one watermark in a provider's per-generation edge chain.
  */
+/**
+ * A verb's interned endpoint ints — eager (already resolved), or a thunk
+ * evaluated when the operation EXECUTES. The lazy form exists for
+ * `transact()` forward references: a relate whose endpoint is added in the
+ * SAME batch cannot resolve ints at plan time (the entity does not exist yet
+ * — a strict native id mapper rightly refuses to assign, and even a permissive
+ * one would leak the assignment if the batch is rejected at precommit).
+ * Deferring to execute time resolves after the batch's add operations have
+ * applied, mirroring how `generationFn` is already evaluated lazily.
+ */
+export type VerbEndpointInts =
+  | { sourceInt: bigint; targetInt: bigint }
+  | (() => { sourceInt: bigint; targetInt: bigint })
+
+/** Resolve a {@link VerbEndpointInts} at execution time. */
+function resolveEndpointInts(
+  endpoints: VerbEndpointInts
+): { sourceInt: bigint; targetInt: bigint } {
+  return typeof endpoints === 'function' ? endpoints() : endpoints
+}
+
 export class AddToGraphIndexOperation implements Operation {
   readonly name = 'AddToGraphIndex'
 
   /**
    * @param index - The graph-index provider (JS baseline or native).
    * @param verb - The verb to index (`sourceInt`/`targetInt` mirrored on it).
-   * @param sourceInt - The source entity's interned int.
-   * @param targetInt - The target entity's interned int.
+   * @param endpointInts - The endpoints' interned ints — eager, or a thunk
+   *   evaluated at execute time (REQUIRED for transact forward references;
+   *   see {@link VerbEndpointInts}).
    * @param generationFn - Resolves the commit generation to stamp this edge at,
    *   evaluated when the operation executes (see class note).
    * @param onVerbInt - Optional hook invoked with the interned verb int
@@ -192,17 +214,18 @@ export class AddToGraphIndexOperation implements Operation {
   constructor(
     private readonly index: GraphIndexProvider,
     private readonly verb: GraphVerb,
-    private readonly sourceInt: bigint,
-    private readonly targetInt: bigint,
+    private readonly endpointInts: VerbEndpointInts,
     private readonly generationFn: () => bigint,
     private readonly onVerbInt?: (verbInt: bigint) => void
   ) {}
 
   async execute(): Promise<RollbackAction> {
     // Stamp this edge at the in-flight commit generation; reuse it for the
-    // rollback so add + undo reference the same watermark.
+    // rollback so add + undo reference the same watermark. Endpoint ints
+    // resolve HERE — after any same-batch adds have applied.
     const generation = this.generationFn()
-    const verbInt = await this.index.addVerb(this.verb, this.sourceInt, this.targetInt, generation)
+    const { sourceInt, targetInt } = resolveEndpointInts(this.endpointInts)
+    const verbInt = await this.index.addVerb(this.verb, sourceInt, targetInt, generation)
     this.onVerbInt?.(verbInt)
 
     // Return rollback action
@@ -231,28 +254,32 @@ export class RemoveFromGraphIndexOperation implements Operation {
   /**
    * @param index - The graph-index provider (JS baseline or native).
    * @param verb - The verb being removed (required for rollback re-add).
-   * @param sourceInt - The source entity's interned int (rollback re-add).
-   * @param targetInt - The target entity's interned int (rollback re-add).
+   * @param endpointInts - The endpoints' interned ints for the rollback
+   *   re-add — eager, or a thunk evaluated at execute time (required when the
+   *   verb or its endpoints were created in the SAME transact batch; see
+   *   {@link VerbEndpointInts}).
    * @param generationFn - Resolves the commit generation for this removal,
    *   evaluated when the operation executes.
    */
   constructor(
     private readonly index: GraphIndexProvider,
     private readonly verb: GraphVerb, // Required for rollback
-    private readonly sourceInt: bigint,
-    private readonly targetInt: bigint,
+    private readonly endpointInts: VerbEndpointInts,
     private readonly generationFn: () => bigint
   ) {}
 
   async execute(): Promise<RollbackAction> {
     // Resolve the removal generation once; reuse it for the rollback re-add.
+    // Endpoint ints resolve HERE (after any same-batch adds applied) and are
+    // captured for the rollback, whose re-add must use the same mappings.
     const generation = this.generationFn()
+    const { sourceInt, targetInt } = resolveEndpointInts(this.endpointInts)
     await this.index.removeVerb(this.verb.id, generation)
 
     // Return rollback action
     return async () => {
       // Re-add verb with original data
-      await this.index.addVerb(this.verb, this.sourceInt, this.targetInt, generation)
+      await this.index.addVerb(this.verb, sourceInt, targetInt, generation)
     }
   }
 }

src/transaction/operations/StorageOperations.ts

@@ -104,37 +104,74 @@ export class SaveNounOperation implements Operation {
 }
 
 /**
- * Delete noun metadata with rollback support
+ * Delete a noun — FULL canonical removal, with rollback support.
+ *
+ * Despite the historical name, this removes the WHOLE entity: both canonical
+ * legs (metadata + vector) AND the entity's container. Previously it deleted
+ * only the metadata leg via `deleteNounMetadata`, leaving the canonical
+ * `vectors.json` leg and the `<id>/` directory orphaned on disk — a "ghost"
+ * that reads as absent (getNoun needs both legs) yet inflates the enumerated
+ * count and can never be told apart from a damage scar. Routing through
+ * `storage.deleteNoun` removes both legs + the container in one place.
+ *
+ * Immutability is preserved: this cleans only the live-HEAD projection; the
+ * generation store retains the before-image so `asOf()` still reconstructs the
+ * deleted entity until retention expires.
  *
  * Rollback strategy:
- * - Restore deleted metadata
+ * - Restore BOTH legs from the before-image (vector leg raw, metadata leg via
+ *   the count-aware save so deleteNoun()'s decrement is reversed).
  */
 export class DeleteNounMetadataOperation implements Operation {
-  readonly name = 'DeleteNounMetadata'
+  readonly name = 'DeleteNoun'
 
   constructor(
     private readonly storage: StorageAdapter,
-    private readonly id: string
+    private readonly id: string,
+    /**
+     * OPTIONAL already-known metadata of the entity being removed (the caller's
+     * pre-delete read). Removal must never REQUIRE re-reading the thing being
+     * removed: if the reads here return null (replace race, or a ghost left by
+     * an earlier version), the count decrement downstream falls back to this
+     * record instead of being silently skipped — the skip minted permanent
+     * counter inflation (adds counted, paired removals not decremented).
+     */
+    private readonly priorMetadata?: NounMetadata | null
   ) {}
 
   async execute(): Promise<RollbackAction> {
-    // Get metadata before deletion (for rollback)
-    const previousMetadata = await this.storage.getNounMetadata(this.id)
+    // Capture the FULL before-image (both legs) so the undo restores the whole
+    // entity — a metadata-only rollback would leave the vector leg unrestored.
+    // A null metadata read falls back to the caller's pre-delete read.
+    const previousNoun = await this.storage.getNoun(this.id)
+    const previousMetadata = (await this.storage.getNounMetadata(this.id)) ?? this.priorMetadata ?? null
 
-    if (!previousMetadata) {
+    if (!previousNoun && !previousMetadata) {
       // Nothing to delete - no rollback needed
       return async () => {}
     }
 
-    // Delete metadata
-    await this.storage.deleteNounMetadata(this.id)
+    // Full removal: both canonical legs + the entity container + count decrement
+    // (the prior record keeps the decrement honest on a null canonical read).
+    await this.storage.deleteNoun(this.id, previousMetadata)
 
     // Return rollback action
     return async () => {
-      // Restore deleted metadata
+      // Restore the vector leg, then the metadata leg through the count-aware
+      // save so deleteNoun()'s decrement is reversed.
+      if (previousNoun) {
+        await this.storage.saveNoun({
+          id: previousNoun.id,
+          vector: previousNoun.vector,
+          connections: previousNoun.connections || new Map(),
+          level: previousNoun.level || 0
+        })
+      }
+      if (previousMetadata) {
         await this.storage.saveNounMetadata(this.id, previousMetadata)
       }
     }
+  }
 }
 
 /**
@@ -242,9 +279,9 @@ export class DeleteVerbMetadataOperation implements Operation {
       return async () => {}
     }
 
-    // Delete verb (metadata + vector)
-    // Note: StorageAdapter has deleteVerb but not deleteVerbMetadata
-    await this.storage.deleteVerb(this.id)
+    // Delete verb (metadata + vector). The pre-read rides along so the count
+    // decrement never depends on re-reading the record being removed.
+    await this.storage.deleteVerb(this.id, previousMetadata)
 
     // Return rollback action
     return async () => {

src/transaction/types.ts

@@ -14,6 +14,10 @@ export type TransactionState =
   | 'committed'    // Successfully committed
   | 'rolling_back' // Rolling back due to failure
   | 'rolled_back'  // Successfully rolled back
+  | 'inconsistent' // Rollback ran but ≥1 undo could not be applied — the store
+                   // is NOT cleanly rolled back; the commit orchestration must
+                   // reconcile (adopt-forward or fail-loud). Never claim
+                   // 'rolled_back' when an undo failed.
 
 /**
  * Rollback action - undoes an operation

src/types/brainy.types.ts

@@ -1384,7 +1384,7 @@ export interface MetricState {
   /**
    * Value multiset (String(value) → occurrence count) for exact percentile + distinctCount.
    * Maintained only for `percentile`/`distinctCount` metrics. JSON-serializable; mirrors
-   * Cortex's `value_counts` so JS and native agree bit-for-bit.
+   * Cor's `value_counts` so JS and native agree bit-for-bit.
    */
   valueCounts?: Record<string, number>
 }
@@ -1443,7 +1443,7 @@ export interface AggregateResult {
 }
 
 /**
- * Provider interface for Cortex-accelerated aggregation.
+ * Provider interface for Cor-accelerated aggregation.
  * When registered as 'aggregation' provider, Brainy delegates to this.
  */
 export interface AggregationProvider {
@@ -1673,6 +1673,21 @@ export interface BrainyConfig {
    */
   migrationWaitTimeoutMs?: number
 
+  /**
+   * Take an automatic **pre-upgrade backup** before a one-time 7.x → 8.0
+   * migration rebuilds the derived indexes, and remove it once the upgrade
+   * verifies (retain it on failure, for rollback). On the filesystem adapter
+   * this is a **hard-link snapshot** of the brain directory — near-zero cost and
+   * space (shared inodes; the store is immutable-by-rename), even at scale.
+   * The migration is already structurally safe (it only reads canonical records,
+   * derived indexes are fully reconstructable, and a failed upgrade self-heals on
+   * re-open), so this is catastrophe-insurance against a migration *bug*, not a
+   * data-loss guard — and NOT a substitute for an off-device backup. No-op for
+   * non-filesystem storage and for a brain with no persisted data.
+   * Default: `true`. Set `false` to opt out (e.g. you run your own backup).
+   */
+  migrationBackup?: boolean
+
   /**
    * Vector index configuration (Brainy 8.0).
    *
@@ -1684,7 +1699,7 @@ export interface BrainyConfig {
    * byte-for-byte so a `'balanced'`-default upgrade is a no-op.
    *
    * **Closed-form contract** locked in handoff thread
-   * BRAINY-8.0-RENAME-COORDINATION § A.2 + § G.2 (cortex-confirmed 2026-06-09).
+   * BRAINY-8.0-RENAME-COORDINATION § A.2 + § G.2 (cor-confirmed 2026-06-09).
    */
   vector?: {
     /**
@@ -1722,7 +1737,10 @@ export interface BrainyConfig {
    * Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
    * `remove`/`relate`) produces an immutable generation record-set serving
    * historical reads (`asOf()`, pinned `Db` values). Without compaction those
-   * accumulate, so Brainy **auto-compacts on every `flush()` and `close()`**.
+   * accumulate, so Brainy **auto-compacts at `close()`** (time-bounded per
+   * pass; 8.9.0 removed compaction from `flush()` — flush is durability work
+   * and never pays maintenance costs). A long-lived writer that never closes
+   * accumulates history until its next explicit `compactHistory()` call.
    * Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
    *
    * Modes:
@@ -1739,7 +1757,7 @@ export interface BrainyConfig {
    *   the oldest unpinned generations while ANY supplied cap is exceeded
    *   (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
    *
-   * `autoCompact: false` disables the automatic flush/close compaction (manage
+   * `autoCompact: false` disables the automatic close() compaction (manage
    * manually via `brain.compactHistory()`). `budgetBytes` is the settable
    * adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
    * Long-term archives belong in `db.persist(path)` snapshots, which compaction
@@ -1757,7 +1775,7 @@ export interface BrainyConfig {
         maxBytes?: number
         /** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
         budgetBytes?: number
-        /** Run compaction automatically on flush()/close() (default: true). */
+        /** Run compaction automatically at close() (default: true; 8.9.0 — flush() never compacts). */
         autoCompact?: boolean
       }
 
@@ -1789,11 +1807,15 @@ export interface BrainyConfig {
   eagerEmbeddings?: boolean
 
   // Plugin configuration
-  // Controls which plugins are loaded during init()
-  // - undefined (default): Auto-detect installed plugins (@soulcraft/cor, etc.)
-  // - false: No plugins — skip auto-detection entirely
-  // - []: No plugins — skip auto-detection entirely
-  // - ['@soulcraft/cor']: Load only specified plugins, no auto-detection
+  // Controls which plugins are loaded during init().
+  // - undefined (default): guarded auto-detection of the first-party
+  //   accelerator (@soulcraft/cor) — installing the package IS the opt-in.
+  //   Not installed → plain brainy, silently. Installed → it loads and
+  //   announces itself. Installed but broken → init() THROWS (an installed
+  //   accelerator never silently vanishes behind the JS engines).
+  // - false / []: no plugins, no detection (explicit opt-out)
+  // - ['@soulcraft/cor']: load exactly these packages; a listed plugin that
+  //   fails to load or is invalid THROWS (loud, never a silent JS fallback)
   plugins?: string[] | false
 
   // Logging configuration

src/utils/collation.ts

@@ -7,12 +7,12 @@
  * and is byte-for-byte identical to Rust's `str::cmp` (`as_bytes().cmp()`). This is:
  * - **deterministic** across environments (unlike `localeCompare`, whose default-locale
  *   ordering varies by OS / Node / ICU build — unsafe for a persisted sorted index), and
- * - **cross-language consistent** with `@soulcraft/cortex`'s native column store and
+ * - **cross-language consistent** with `@soulcraft/cor`'s native column store and
  *   aggregation sort, so query results are identical with or without the native plugin.
  *
  * Use this instead of `String.localeCompare` and the `<`/`>` operators (which compare
  * UTF-16 code units and diverge from code-point order for supplementary-plane characters)
- * wherever ordering is persisted or must match the native engine. See cortex `docs/ADR-001`.
+ * wherever ordering is persisted or must match the native engine. See cor `docs/ADR-001`.
  */
 
 const utf8 = new TextEncoder()

src/utils/crc32c.ts

@@ -0,0 +1,43 @@
+/**
+ * @module utils/crc32c
+ * @description CRC-32C (Castagnoli, polynomial 0x1EDC6F41, reflected 0x82F63B78)
+ * — the storage-industry frame checksum (ext4, iSCSI, SCTP, LSM segment files).
+ * Used to frame generation-fact segments: every appended record carries the
+ * CRC-32C of its payload, so a torn tail (crash mid-append) or bit rot is
+ * DETECTED at scan time and never silently read as data.
+ *
+ * Table-driven, dependency-free reference implementation. Native providers may
+ * substitute a hardware-accelerated (SSE4.2 / ARMv8 CRC) implementation — the
+ * polynomial is the contract, byte-identical results required.
+ */
+
+/** The 256-entry lookup table for the reflected CRC-32C polynomial. */
+const TABLE: Uint32Array = (() => {
+  const table = new Uint32Array(256)
+  for (let n = 0; n < 256; n++) {
+    let c = n
+    for (let k = 0; k < 8; k++) {
+      c = c & 1 ? 0x82f63b78 ^ (c >>> 1) : c >>> 1
+    }
+    table[n] = c >>> 0
+  }
+  return table
+})()
+
+/**
+ * Compute the CRC-32C checksum of a byte buffer.
+ *
+ * Known-answer vectors (RFC 3720 appendix / the standard test suite):
+ * - ASCII "123456789" → 0xE3069283
+ * - 32 zero bytes → 0x8A9136AA
+ *
+ * @param bytes - The payload to checksum.
+ * @returns The CRC-32C as an unsigned 32-bit integer.
+ */
+export function crc32c(bytes: Uint8Array): number {
+  let crc = 0xffffffff
+  for (let i = 0; i < bytes.length; i++) {
+    crc = TABLE[(crc ^ bytes[i]) & 0xff] ^ (crc >>> 8)
+  }
+  return (crc ^ 0xffffffff) >>> 0
+}

src/utils/entityIdMapper.ts

@@ -40,7 +40,7 @@ import type { EntityIdMapperProvider } from '../plugin.js'
  * The largest entity int the JS fallback `EntityIdMapper` will allocate.
  * The metadata index's roaring bitmaps are 32-bit-keyed (`Roaring32`), so
  * allowing the JS counter past this point would silently corrupt every
- * downstream bitmap. The cortex 3.0 binary mapper supports a U64 IdSpace
+ * downstream bitmap. The cor 3.0 binary mapper supports a U64 IdSpace
  * for brains above this ceiling — see the
  * [`EntityIdSpaceExceeded`](#EntityIdSpaceExceeded) message for the
  * migration pointer.
@@ -50,8 +50,8 @@ export const U32_ENTITY_ID_MAX = 0xffff_ffff
 /**
  * Thrown by the JS fallback `EntityIdMapper` when `nextId` would exceed
  * [`U32_ENTITY_ID_MAX`](#U32_ENTITY_ID_MAX). The JS path is the
- * cortex-free fallback; once a brain has more than ~4.29 B entities,
- * callers MUST install the cortex 3.0 `NativeBinaryEntityIdMapper` with
+ * cor-free fallback; once a brain has more than ~4.29 B entities,
+ * callers MUST install the cor 3.0 `NativeBinaryEntityIdMapper` with
  * `idSpace: 'u64'` (mmap-backed extendible-hash KV; persists the full
  * u64 range losslessly).
  *
@@ -96,7 +96,7 @@ export interface EntityIdMapperData {
  * Maps entity UUIDs to integer IDs for use with Roaring Bitmaps.
  *
  * Implements {@link EntityIdMapperProvider}: the surface a registered
- * `'entityIdMapper'` provider (e.g. Cortex's native mapper) must also satisfy.
+ * `'entityIdMapper'` provider (e.g. Cor's native mapper) must also satisfy.
  */
 export class EntityIdMapper implements EntityIdMapperProvider {
   private storage: StorageAdapter
@@ -162,7 +162,7 @@ export class EntityIdMapper implements EntityIdMapperProvider {
    * The JS fallback mapper caps at [`U32_ENTITY_ID_MAX`](#U32_ENTITY_ID_MAX)
    * to match the metadata index's Roaring32 bitmap width — once `nextId`
    * would exceed that, throws {@link EntityIdSpaceExceeded} so the caller
-   * loudly migrates to cortex's binary mapper with `idSpace: 'u64'`
+   * loudly migrates to cor's binary mapper with `idSpace: 'u64'`
    * rather than silently truncating entity ids.
    */
   getOrAssign(uuid: string): number {

src/utils/errorClassification.ts

@@ -0,0 +1,37 @@
+/**
+ * @module utils/errorClassification
+ * @description Shared classification of caught errors into "genuine absence" vs
+ * "real fault" — the antidote to blind `catch { return null }` handlers that
+ * cannot tell ENOENT (the object is legitimately not on disk) from EIO / EACCES
+ * / EMFILE / … (a transient or permission fault on data that IS on disk).
+ * Masking a fault as absence yields wrong results (a present record read as
+ * "not found") or a needless rebuild. Mandate: loud errors, never quiet losses.
+ */
+
+/**
+ * The error `code`s that denote GENUINE absence of a file/object. Only `ENOENT`
+ * ("no such file or directory") qualifies — on every platform Node maps a
+ * missing file/directory to ENOENT, and no other errno means "simply not
+ * there". Every other errno (EIO, EACCES, EPERM, EMFILE, ENFILE, EBUSY,
+ * ENOTDIR, EISDIR, ELOOP) is a real fault and must propagate, as must any error
+ * without an errno `code` (parse/decompress failures, generic Errors).
+ * `ENOTDIR`/`EISDIR` are deliberately faults: a path component of the wrong
+ * type is corruption, not benign absence. Named constant so a future
+ * genuine-absence code can be added in one reviewed place.
+ */
+const ABSENCE_CODES: ReadonlySet<string> = new Set(['ENOENT'])
+
+/**
+ * @description True IFF `e` represents genuine absence (an ENOENT-class errno),
+ * for which returning `null`/`[]`/`undefined` is the correct answer. Returns
+ * `false` for every real fault, so the canonical call site is:
+ * `catch (e) { if (isAbsentError(e)) return null; throw e }`.
+ *
+ * @param e - The caught value (typed `unknown`; non-objects are never absence).
+ * @returns Whether the error means "the thing is simply not there".
+ */
+export function isAbsentError(e: unknown): boolean {
+  if (e === null || typeof e !== 'object') return false
+  const code = (e as { code?: unknown }).code
+  return typeof code === 'string' && ABSENCE_CODES.has(code)
+}

src/utils/indexReadiness.ts

@@ -0,0 +1,38 @@
+/**
+ * @module indexReadiness
+ * @description The single honest-readiness classifier shared by the vector,
+ * graph and metadata index sites. It exists to kill "Pattern A" — the dishonest
+ * readiness proxy where `size() > 0` / `isInitialized` is treated as "this index
+ * actually serves queries." A cold native index that loaded its COUNT but not its
+ * SERVING structure passes those proxies and silently returns `[]`.
+ *
+ * This classifier reads ONLY the provider's OPTIONAL, honest `isReady()` signal
+ * (see {@link import('../plugin.js').VectorIndexProvider.isReady},
+ * {@link import('../plugin.js').GraphIndexProvider.isReady},
+ * {@link import('../plugin.js').MetadataIndexProvider.isReady}). It NEVER inspects
+ * `size()` or `isInitialized`. When `isReady()` is absent, callers must fall back
+ * to a KNOWN-ITEM PROBE (a real search/lookup that must return a known-present
+ * datum) before trusting an empty result — never a `size()` proxy.
+ */
+
+/** A provider that MAY expose the honest cold-load readiness signal. */
+export interface MaybeReadyProvider {
+  isReady?: () => boolean
+}
+
+/** Three-valued honest-readiness verdict. */
+export type IndexReadiness = 'ready' | 'not-ready' | 'unknown'
+
+/**
+ * @description Classify an index provider's honest readiness.
+ * @param provider - Any index provider (vector / graph / metadata) or `null`.
+ * @returns
+ *  - `'ready'`     when `isReady() === true`  (serving structure loaded — trust it);
+ *  - `'not-ready'` when `isReady() === false` (count/manifest loaded, NOT serving — rebuild);
+ *  - `'unknown'`   when the provider exposes no `isReady()` (caller must probe / keep the JS heuristic).
+ */
+export function assessIndexReadiness(provider: unknown): IndexReadiness {
+  const p = provider as MaybeReadyProvider | null | undefined
+  if (p == null || typeof p.isReady !== 'function') return 'unknown'
+  return p.isReady() ? 'ready' : 'not-ready'
+}

src/utils/metadataFilter.ts

@@ -5,6 +5,7 @@
  */
 
 import { SearchResult, HNSWNoun, HNSWNounWithMetadata } from '../coreTypes.js'
+import { BrainyError } from '../errors/brainyError.js'
 
 /**
  * Brainy Field Operators (BFO) - Our own field query system
@@ -30,6 +31,7 @@ export interface BrainyFieldOperators {
   
   // Array/Set operators
   oneOf?: any[]
+  in?: any[] // documented alias for oneOf
   noneOf?: any[]
   contains?: any
   excludes?: any
@@ -70,6 +72,65 @@ export interface MetadataFilterOptions {
   }
 }
 
+/**
+ * Value-level operators (the keys inside `{ field: { <op>: operand } }`) — the
+ * complete documented set, kept in lockstep with the `matchesQuery` switch below
+ * and the metadata-index path. `in` is the documented alias for `oneOf`.
+ */
+const VALUE_OPERATORS = new Set<string>([
+  'equals', 'eq', 'notEquals', 'ne',
+  'greaterThan', 'gt', 'greaterThanOrEqual', 'gte',
+  'lessThan', 'lt', 'lessThanOrEqual', 'lte',
+  'between', 'oneOf', 'in', 'noneOf',
+  'contains', 'excludes', 'hasAll', 'length',
+  'exists', 'missing', 'matches', 'startsWith', 'endsWith'
+])
+
+/** Filter-level logical operators (siblings of field names). */
+const LOGICAL_OPERATORS = new Set<string>(['allOf', 'anyOf', 'not'])
+
+/**
+ * Validate a `where` filter's operators up front, throwing a typed
+ * `BrainyError('INVALID_QUERY')` on the first unrecognized operator — so a typo
+ * like `{ subtype: { notIn: [...] } }` fails LOUD instead of silently matching
+ * nothing (the invisible-degrade class). Called by `find()` before either the
+ * index path or the in-memory matcher runs, so the throw fires even when the
+ * result set is empty (the index path would otherwise return `[]` without ever
+ * invoking the matcher). Nested fields use dot notation (`{ 'a.b': v }`), so a
+ * field's object value carries operators, never sub-field names.
+ *
+ * @param filter - the user-supplied `where` clause (validated raw, before any
+ *   internal field injection like visibility or `type`→`noun`).
+ * @throws BrainyError('INVALID_QUERY') naming the bad operator + the valid set.
+ */
+export function validateWhereFilter(filter: unknown): void {
+  if (!filter || typeof filter !== 'object' || Array.isArray(filter)) return
+  for (const [key, value] of Object.entries(filter as Record<string, unknown>)) {
+    if (LOGICAL_OPERATORS.has(key)) {
+      if (key === 'not') {
+        validateWhereFilter(value)
+      } else if (Array.isArray(value)) {
+        for (const sub of value) validateWhereFilter(sub)
+      }
+      continue
+    }
+    // A field key. Its value is a scalar/array (equality) — nothing to validate —
+    // or an object of value-operators, every key of which must be recognized.
+    if (value && typeof value === 'object' && !Array.isArray(value)) {
+      for (const op of Object.keys(value as Record<string, unknown>)) {
+        if (!VALUE_OPERATORS.has(op)) {
+          throw new BrainyError(
+            `Unknown filter operator "${op}" on field "${key}". Valid operators: ` +
+              `${[...VALUE_OPERATORS].sort().join(', ')}. For nested fields use dot ` +
+              `notation, e.g. { '${key}.subfield': value }.`,
+            'INVALID_QUERY'
+          )
+        }
+      }
+    }
+  }
+}
+
 /**
  * Check if a value matches a query with operators
  */
@@ -103,6 +164,7 @@ function matchesQuery(value: any, query: any): boolean {
         if (typeof value !== 'number' || typeof operand !== 'number' || !(value > operand)) return false
         break
       case 'gte':
+      case 'greaterThanOrEqual':
         if (typeof value !== 'number' || typeof operand !== 'number' || !(value >= operand)) return false
         break
       case 'lessThan':
@@ -110,6 +172,7 @@ function matchesQuery(value: any, query: any): boolean {
         if (typeof value !== 'number' || typeof operand !== 'number' || !(value < operand)) return false
         break
       case 'lte':
+      case 'lessThanOrEqual':
         if (typeof value !== 'number' || typeof operand !== 'number' || !(value <= operand)) return false
         break
       case 'between':
@@ -119,6 +182,7 @@ function matchesQuery(value: any, query: any): boolean {
         
       // Array/Set operators
       case 'oneOf':
+      case 'in': // documented alias for oneOf
         if (!Array.isArray(operand) || !operand.includes(value)) return false
         break
       case 'noneOf':
@@ -161,22 +225,26 @@ function matchesQuery(value: any, query: any): boolean {
         break
         
       default:
-        // Unknown operator, treat as field name
-        if (!matchesFieldQuery(value, op, operand)) return false
+        // Unknown operator. The old behavior treated any unknown key as a
+        // nested-object field name (an UNDOCUMENTED fallback — dot notation
+        // `{ 'a.b': v }` is the supported nested form), which silently swallowed
+        // operator typos: `{ x: { notIn: [...] } }` matched nothing instead of
+        // erroring. Fail loud. find() validates the whole where clause up front
+        // (validateWhereFilter), so a typo throws even on an empty result set;
+        // this is the belt-and-suspenders for any matcher caller that bypasses
+        // that path.
+        throw new BrainyError(
+          `Unknown filter operator "${op}". Valid operators: ` +
+            `${[...VALUE_OPERATORS].sort().join(', ')}. For nested fields use dot ` +
+            `notation, e.g. { 'address.city': 'NYC' }.`,
+          'INVALID_QUERY'
+        )
     }
   }
 
   return true
 }
 
-/**
- * Check if a field matches a query
- */
-function matchesFieldQuery(obj: any, field: string, query: any): boolean {
-  const value = getNestedValue(obj, field)
-  return matchesQuery(value, query)
-}
-
 /**
  * Get nested value from object using dot notation
  */

src/utils/metadataIndex.ts

@@ -77,7 +77,7 @@ export interface MetadataIndexConfig {
 }
 
 export interface MetadataIndexOptions {
-  entityIdMapper?: EntityIdMapper // Optional pre-configured EntityIdMapper (e.g., native from cortex)
+  entityIdMapper?: EntityIdMapper // Optional pre-configured EntityIdMapper (e.g., native from cor)
 }
 
 /**
@@ -107,7 +107,7 @@ interface FieldStats {
 /**
  * Implements {@link MetadataIndexProvider}: the metadata-index surface Brainy
  * calls on whatever the `'metadataIndex'` provider resolves to (its own
- * manager, or Cortex's native Rust engine).
+ * manager, or Cor's native Rust engine).
  */
 export class MetadataIndexManager implements MetadataIndexProvider {
   private storage: StorageAdapter
@@ -221,7 +221,7 @@ export class MetadataIndexManager implements MetadataIndexProvider {
     // Get global unified cache for coordinated memory management
     this.unifiedCache = getGlobalCache()
 
-    // Use injected EntityIdMapper (e.g., native from cortex) or create JS fallback
+    // Use injected EntityIdMapper (e.g., native from cor) or create JS fallback
     this.idMapper = options.entityIdMapper ?? new EntityIdMapper({
       storage,
       storageKey: 'brainy:entityIdMapper'
@@ -1166,8 +1166,17 @@ export class MetadataIndexManager implements MetadataIndexProvider {
   private extractIndexableFields(data: any): Array<{ field: string, value: any }> {
     const fields: Array<{ field: string, value: any }> = []
 
-    // Fields that should NEVER be indexed (vectors, embeddings, large arrays, HNSW internals)
-    const NEVER_INDEX = new Set(['vector', 'embedding', 'embeddings', 'connections', 'level', 'id'])
+    // Fields that should NEVER be indexed: bulk structural payloads that would
+    // blow up the index (the 384-dim vector, embeddings, the adjacency list).
+    // These are also caught by the array-size guard below, but naming them is
+    // belt-and-suspenders. NOTE: `level` was previously here (an HNSW node's
+    // layer) but it never actually reaches this path — every caller passes a
+    // metadata bag or Entity record, neither of which carries the node's
+    // `level` — so its only effect was to silently drop a legitimate USER
+    // metadata field named `level` (log level, skill level, access level…),
+    // making `where: { level: … }` return nothing. Removed. (`id` stays: it is
+    // the reserved entity-identity field, resolved specially by find().)
+    const NEVER_INDEX = new Set(['vector', 'embedding', 'embeddings', 'connections', 'id'])
 
     const extract = (obj: any, prefix = ''): void => {
       for (const [key, value] of Object.entries(obj)) {
@@ -1278,7 +1287,10 @@ export class MetadataIndexManager implements MetadataIndexProvider {
       return data.map(d => this.extractTextContent(d)).filter(Boolean).join(' ')
     }
     if (typeof data === 'object') {
-      const skipKeys = new Set(['vector', 'embedding', 'embeddings', 'connections', 'level', 'id'])
+      // Mirror of NEVER_INDEX for the text-extraction path: bulk structural
+      // payloads only. `level` removed for the same reason (it silently dropped
+      // a real user field from hybrid text search too).
+      const skipKeys = new Set(['vector', 'embedding', 'embeddings', 'connections', 'id'])
       const texts: string[] = []
       for (const [key, value] of Object.entries(data)) {
         // Skip internal fields and numeric keys (array indices)
@@ -1855,9 +1867,26 @@ export class MetadataIndexManager implements MetadataIndexProvider {
     // not once per AND-clause inside it.
     const unindexedFields: string[] = []
 
-    for (const [field, condition] of Object.entries(filter)) {
+    for (const [rawField, condition] of Object.entries(filter)) {
       // Skip logical operators
-      if (field === 'allOf' || field === 'anyOf' || field === 'not') continue
+      if (rawField === 'allOf' || rawField === 'anyOf' || rawField === 'not') continue
+
+      // Metadata is FLATTENED at index time (metadata.entry.title indexes as
+      // entry.title), so a `metadata.`-prefixed where key is almost always
+      // the caller spelling the STORAGE shape rather than the index shape.
+      // Accept both spellings: when the key as spelled is unindexed but its
+      // stripped spelling is, query the stripped one. A literal nested
+      // custom key named `metadata` still wins when indexed as spelled
+      // (checked first), so that rare shape keeps working.
+      let field = rawField
+      if (
+        rawField.startsWith('metadata.') &&
+        this.columnStore &&
+        !this.columnStore.hasField(rawField) &&
+        this.columnStore.hasField(rawField.slice('metadata.'.length))
+      ) {
+        field = rawField.slice('metadata.'.length)
+      }
 
       let fieldResults: string[] = []
 
@@ -2196,7 +2225,7 @@ export class MetadataIndexManager implements MetadataIndexProvider {
       if (b.value == null) return order === 'asc' ? -1 : 1
       if (a.value === b.value) return 0
       // Numbers compare numerically; everything else by code-point (UTF-8 byte) order.
-      // This makes the JS fallback sort match cortex's native column store exactly
+      // This makes the JS fallback sort match cor's native column store exactly
       // (numeric i64/f64 vs code-point strings) and stay deterministic across
       // environments, unlike the `<` operator's UTF-16 ordering for strings.
       let comparison: number

src/utils/metadataWriteBuffer.ts

@@ -104,6 +104,11 @@ export class MetadataWriteBuffer {
         })
       }
     }, this.flushIntervalMs)
+    // Best-effort background flush — must not keep the process alive
+    // (close() drains the buffer for durability).
+    if (this.flushTimer && typeof this.flushTimer.unref === 'function') {
+      this.flushTimer.unref()
+    }
 
     // Prevent timer from keeping the process alive
     if (this.flushTimer && typeof this.flushTimer === 'object' && 'unref' in this.flushTimer) {

src/utils/osLimits.ts

@@ -0,0 +1,141 @@
+/**
+ * @module utils/osLimits
+ * @description Detect-and-warn for OS resource limits that bite at POOL scale.
+ *
+ * A single brain rarely notices them, but a pool of brains — especially with a
+ * native accelerator memory-mapping many index files per brain — consumes file
+ * descriptors and memory mappings multiplicatively. On stock Linux defaults
+ * (RLIMIT_NOFILE soft 1024, vm.max_map_count 65530) the failure arrives as
+ * EMFILE or a failed mmap deep inside an index open, long after the real cause
+ * (the limit) stopped being visible. This module reads the limits at open and
+ * WARNS ONCE per process with the exact raise commands, so the operator learns
+ * the fix before the incident instead of from it.
+ *
+ * Read-only and Linux-only by construction: both sources are `/proc` files.
+ * On platforms where they are absent the check reports nulls and stays silent —
+ * no limit read means no claim made, never a guessed warning.
+ */
+
+import * as fs from 'node:fs'
+import { prodLog } from './logger.js'
+
+/** Soft-NOFILE floor below which pool-scale use is at EMFILE risk. */
+export const NOFILE_POOL_FLOOR = 65536
+
+/** vm.max_map_count floor below which mmap-heavy native indexes are at risk. */
+export const MAX_MAP_COUNT_POOL_FLOOR = 262144
+
+export interface OsLimitsReport {
+  /** RLIMIT_NOFILE soft limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileSoft: number | null
+  /** RLIMIT_NOFILE hard limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileHard: number | null
+  /** vm.max_map_count (null when unreadable). */
+  maxMapCount: number | null
+  /** Human-actionable warnings for limits below the pool floors. Empty = fine. */
+  warnings: string[]
+}
+
+/**
+ * Parse the `Max open files` row of a `/proc/<pid>/limits` document into
+ * soft/hard values. Returns nulls when the row is absent or malformed.
+ */
+export function parseProcLimits(content: string): { soft: number | null; hard: number | null } {
+  const line = content.split('\n').find((l) => l.startsWith('Max open files'))
+  if (!line) return { soft: null, hard: null }
+  const m = line.match(/^Max open files\s+(\S+)\s+(\S+)/)
+  if (!m) return { soft: null, hard: null }
+  const parse = (v: string): number | null => {
+    if (v === 'unlimited') return Infinity
+    const n = Number.parseInt(v, 10)
+    return Number.isNaN(n) ? null : n
+  }
+  return { soft: parse(m[1]), hard: parse(m[2]) }
+}
+
+/**
+ * Assess readable limits against the pool floors. Pure — feed it any values.
+ * A null (unreadable) limit produces NO warning: no measurement, no claim.
+ */
+export function assessOsLimits(limits: {
+  nofileSoft: number | null
+  nofileHard: number | null
+  maxMapCount: number | null
+}): string[] {
+  const warnings: string[] = []
+
+  if (limits.nofileSoft !== null && limits.nofileSoft < NOFILE_POOL_FLOOR) {
+    const hardNote =
+      limits.nofileHard !== null && limits.nofileHard >= NOFILE_POOL_FLOOR
+        ? ` (the hard limit ${limits.nofileHard === Infinity ? 'unlimited' : limits.nofileHard} already allows it — raise the soft limit only)`
+        : ''
+    warnings.push(
+      `RLIMIT_NOFILE soft limit is ${limits.nofileSoft} — below the ${NOFILE_POOL_FLOOR} recommended ` +
+        `for pool-scale use (a pool of brains with a native accelerator opens many index files per brain; ` +
+        `the failure mode is EMFILE deep inside an index open). Raise with \`ulimit -n ${NOFILE_POOL_FLOOR}\` ` +
+        `or LimitNOFILE=${NOFILE_POOL_FLOOR} in the service unit${hardNote}.`
+    )
+  }
+
+  if (limits.maxMapCount !== null && limits.maxMapCount < MAX_MAP_COUNT_POOL_FLOOR) {
+    warnings.push(
+      `vm.max_map_count is ${limits.maxMapCount} — below the ${MAX_MAP_COUNT_POOL_FLOOR} recommended ` +
+        `for mmap-heavy native indexes at pool scale (each mapped index segment consumes map entries; ` +
+        `the failure mode is a failed mmap mid-heal). Raise with ` +
+        `\`sysctl -w vm.max_map_count=${MAX_MAP_COUNT_POOL_FLOOR}\` (persist in /etc/sysctl.d/).`
+    )
+  }
+
+  return warnings
+}
+
+/**
+ * Read the limits from /proc and assess them. `readFile` is injectable for
+ * tests; absent/unreadable sources yield nulls (and therefore no warnings).
+ */
+export async function checkOsLimits(
+  readFile: (path: string) => Promise<string> = async (p) => fs.promises.readFile(p, 'utf-8')
+): Promise<OsLimitsReport> {
+  let nofileSoft: number | null = null
+  let nofileHard: number | null = null
+  let maxMapCount: number | null = null
+
+  try {
+    const parsed = parseProcLimits(await readFile('/proc/self/limits'))
+    nofileSoft = parsed.soft
+    nofileHard = parsed.hard
+  } catch {
+    // Not Linux (or /proc unavailable) — no measurement, no claim.
+  }
+
+  try {
+    const raw = (await readFile('/proc/sys/vm/max_map_count')).trim()
+    const n = Number.parseInt(raw, 10)
+    maxMapCount = Number.isNaN(n) ? null : n
+  } catch {
+    // Not Linux — same rule.
+  }
+
+  const warnings = assessOsLimits({ nofileSoft, nofileHard, maxMapCount })
+  return { nofileSoft, nofileHard, maxMapCount, warnings }
+}
+
+/** Once-per-process latch so a brain pool warns once, not once per brain. */
+let osLimitsWarned = false
+
+/**
+ * Run the check and warn (once per process) about limits below the pool
+ * floors. Called from brain open; safe everywhere (silent off-Linux).
+ */
+export async function warnOnLowOsLimits(): Promise<void> {
+  if (osLimitsWarned) return
+  osLimitsWarned = true
+  try {
+    const report = await checkOsLimits()
+    for (const warning of report.warnings) {
+      prodLog.warn(`[Brainy] OS limit check: ${warning}`)
+    }
+  } catch {
+    // The check must never affect open — measurement-only.
+  }
+}

src/utils/paramValidation.ts

@@ -209,8 +209,10 @@ export interface ValidationConfigOptions {
 }
 
 /**
- * Auto-configured limits based on system resources
- * These adapt to available memory and observed performance
+ * Auto-configured limits based on system resources.
+ * Derived from memory (explicit overrides > reserved memory > container limit >
+ * free memory). Query timing is recorded for diagnostics only — it never
+ * changes the cap (see `recordQuery`).
  */
 export class ValidationConfig {
   private static instance: ValidationConfig | null = null
@@ -323,24 +325,23 @@ export class ValidationConfig {
   }
 
   /**
-   * Learn from actual usage to adjust limits
+   * Record query timing for diagnostics. Telemetry ONLY — never mutates the cap.
+   *
+   * `maxLimit` is a MEMORY-protection bound; query duration says nothing about
+   * memory-per-result, so duration must never drive it. An earlier version
+   * "learned" here: while the lifetime-average query time exceeded 1s it shrank
+   * `maxLimit` by 20% per recorded query down to a floor of 1000 — below the
+   * documented `MIN_AUTO_QUERY_LIMIT` (10 000) and with no recovery once slow
+   * samples poisoned the cumulative average. On a production host a burst of
+   * slow aggregate queries silently strangled every consumer's `find()` to
+   * 1000 while the error message blamed "available free memory" — exactly the
+   * silent throttling this module's own contract forbids. The cap now comes
+   * from its construction-time basis (or explicit overrides) alone.
    */
   recordQuery(duration: number, resultCount: number) {
+    void resultCount
     this.queryCount++
     this.avgQueryTime = (this.avgQueryTime * (this.queryCount - 1) + duration) / this.queryCount
-
-    // Only auto-adjust if not using explicit overrides
-    if (this.limitBasis !== 'override') {
-      // If queries are consistently fast with large results, increase limits
-      if (this.avgQueryTime < 100 && resultCount > this.maxLimit * 0.8) {
-        this.maxLimit = Math.min(this.maxLimit * 1.5, 100000)
-      }
-
-      // If queries are slow, reduce limits
-      if (this.avgQueryTime > 1000) {
-        this.maxLimit = Math.max(this.maxLimit * 0.8, 1000)
-      }
-    }
   }
 }
 

src/utils/recallPreset.ts

@@ -7,7 +7,7 @@
  * provider (DiskANN-style) has taken over the `'vector'` provider key.
  *
  * **Public contract** — locked in handoff thread BRAINY-8.0-RENAME-COORDINATION
- * § A.2 (cortex-confirmed 2026-06-09):
+ * § A.2 (cor-confirmed 2026-06-09):
  * - `'fast'` — minimum-latency search, accepts lower recall
  * - `'balanced'` — Brainy 7.x defaults, the right pick for almost everyone
  * - `'accurate'` — maximum recall, accepts higher latency
@@ -16,7 +16,7 @@
  *
  * **Knob mapping** — the JS HNSW preset values below match what Brainy 7.x
  * shipped as defaults for `'balanced'`. Native acceleration providers (e.g.
- * cortex's DiskANN wrapper) read the same `recall` value off the index
+ * cor's DiskANN wrapper) read the same `recall` value off the index
  * config and translate it to their own internal knobs.
  */
 

src/utils/resultRanking.ts

@@ -8,7 +8,7 @@
  * This module exposes a swappable `sort:topK` seam:
  * - {@link rankIndicesByScore} returns the indices of `scores` ordered exactly as a
  *   descending stable sort would order them, then truncated to `k`.
- * - {@link setSortTopKImplementation} lets a native plugin (e.g. cortex's Rust
+ * - {@link setSortTopKImplementation} lets a native plugin (e.g. cor's Rust
  *   partial-sort / heap-select) replace the JS implementation. The native provider
  *   only has to return the same *index ordering* the JS path produces.
  *
@@ -147,7 +147,7 @@ export function reorderByIndices<T>(items: T[], order: number[]): T[] {
 }
 
 /**
- * Replace the `sort:topK` implementation at runtime (e.g. cortex's native partial sort).
+ * Replace the `sort:topK` implementation at runtime (e.g. cor's native partial sort).
  * Called by `brainy.ts` when a `sort:topK` provider is registered. Pass
  * {@link sortTopKIndicesJs} to restore the JS default.
  *

src/utils/roaring/index.ts

@@ -31,7 +31,7 @@ let _impl: typeof WasmRoaringBitmap32 = WasmRoaringBitmap32
 
 /**
  * Replace the RoaringBitmap32 implementation at runtime.
- * Called by brainy.ts when a cortex 'roaring' provider is registered.
+ * Called by brainy.ts when a cor 'roaring' provider is registered.
  */
 export function setRoaringImplementation(impl: typeof WasmRoaringBitmap32) {
   _impl = impl

src/utils/typeValidation.ts

@@ -1,3 +1,11 @@
+/**
+ * @module utils/typeValidation
+ * @description O(1) runtime validation for the {@link NounType} / {@link VerbType}
+ * enums. Precomputes a `Set` of each enum's values once at module load, then
+ * exposes type-guard predicates (`isValidNounType` / `isValidVerbType`) and their
+ * throwing counterparts used by the write-path parameter validators. Pure and
+ * side-effect-free apart from the two module-level lookup sets.
+ */
 import { NounType, VerbType } from '../types/graphTypes.js'
 
 // Type sets for O(1) validation

src/utils/unifiedCache.ts

@@ -63,7 +63,7 @@ export interface UnifiedCacheConfig {
  *
  * Implements {@link CacheProvider}: the surface Brainy calls on whatever cache
  * is installed as the global cache (its own instance, or a registered
- * `'cache'` provider such as Cortex's native eviction engine).
+ * `'cache'` provider such as Cor's native eviction engine).
  */
 export class UnifiedCache implements CacheProvider {
   private cache = new Map<string, CacheItem>()
@@ -79,6 +79,7 @@ export class UnifiedCache implements CacheProvider {
   private readonly memoryInfo: MemoryInfo
   private readonly allocationStrategy: CacheAllocationStrategy
   private memoryPressureCheckTimer: NodeJS.Timeout | null = null
+  private fairnessMonitorTimer: NodeJS.Timeout | null = null
   private lastMemoryWarning = 0
 
   constructor(config: UnifiedCacheConfig = {}) {
@@ -313,12 +314,19 @@ export class UnifiedCache implements CacheProvider {
   }
 
   /**
-   * Fairness monitoring - prevent one type from hogging cache
+   * Fairness monitoring - prevent one type from hogging cache.
+   *
+   * The interval is unref'd: a background cache monitor must never keep the
+   * host process alive (this exact timer made bare scripts hang after
+   * `brain.close()` — the loop could not drain while it stayed ref'd).
    */
   private startFairnessMonitor(): void {
-    setInterval(() => {
+    this.fairnessMonitorTimer = setInterval(() => {
       this.checkFairness()
     }, this.config.fairnessCheckInterval!)
+    if (this.fairnessMonitorTimer.unref) {
+      this.fairnessMonitorTimer.unref()
+    }
   }
 
   private checkFairness(): void {
@@ -444,7 +452,7 @@ export class UnifiedCache implements CacheProvider {
   /**
    * Dynamically resize the cache maximum. If the new size is smaller than
    * the current usage, evicts lowest-value items until the cache fits within
-   * the new budget. Used by Cortex's ResourceManager to rebalance cache vs
+   * the new budget. Used by Cor's ResourceManager to rebalance cache vs
    * instance memory as brainy instances are created and evicted.
    *
    * @param newMaxSize - New maximum cache size in bytes

src/vfs/PathResolver.ts

@@ -525,6 +525,10 @@ export class PathResolver {
         console.log(`[PathResolver] Cache stats: ${Math.round(hitRate * 100)}% hit rate, ${this.pathCache.size} entries, ${this.hotPaths.size} hot paths`)
       }
     }, 60000)  // Every minute
+    // Cache maintenance must never keep the host process alive.
+    if (this.maintenanceTimer && typeof this.maintenanceTimer.unref === 'function') {
+      this.maintenanceTimer.unref()
+    }
   }
 
   /**

src/vfs/VirtualFileSystem.ts

@@ -36,6 +36,7 @@ import {
   VFSErrorCode,
   WriteOptions,
   ReadOptions,
+  FileVersion,
   MkdirOptions,
   ReaddirOptions,
   CopyOptions,
@@ -352,6 +353,52 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     }
   }
 
+  /**
+   * @description Recover VFS content blobs orphaned by a 7→8 upgrade. 7.x stored
+   * VFS blobs under the copy-on-write area (`_cow/`) of the branch/versioning
+   * system that 8.0 removed; a brain upgraded before the migration adopted them
+   * reads a stranded blob and throws "Blob metadata not found" (every page
+   * backed by it 500s). This adopts every orphaned `_cow/` blob into the 8.0
+   * content-addressed store (`_cas/`) IN PLACE — no snapshot restore — then
+   * clears the VFS caches so the recovered content reads live immediately.
+   *
+   * Idempotent and non-destructive: blobs already adopted are skipped and the
+   * `_cow/` originals are never deleted (a re-run or rollback stays possible).
+   * Safe to run on a healthy or native-8.0 brain (returns zeros). Delegates to
+   * {@link BaseStorage.adoptLegacyCowBlobs}.
+   *
+   * @returns `{ cowBlobs, adopted, alreadyPresent, incomplete }` counts.
+   * @example
+   * const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/brain' } })
+   * await brain.init()
+   * const r = await brain.vfs.adoptOrphanedBlobs()
+   * console.log(`Adopted ${r.adopted} stranded VFS blobs; ${r.alreadyPresent} already present.`)
+   */
+  async adoptOrphanedBlobs(): Promise<{
+    cowBlobs: number
+    adopted: number
+    alreadyPresent: number
+    incomplete: number
+  }> {
+    // The recovery scan works on raw storage objects, so it only needs the
+    // brain's storage wired up — not the VFS's own init. brain.init() is
+    // idempotent (a no-op on an already-open brain) and, on a cold brain,
+    // sets up storage AND runs the on-open auto-adoption itself; the explicit
+    // scan below is then the idempotent "force re-scan" equivalent.
+    await this.brain.init()
+    const storage = this.brain['storage'] as unknown as
+      | { adoptLegacyCowBlobs?: () => Promise<{ cowBlobs: number; adopted: number; alreadyPresent: number; incomplete: number }> }
+      | undefined
+    if (!storage || typeof storage.adoptLegacyCowBlobs !== 'function') {
+      return { cowBlobs: 0, adopted: 0, alreadyPresent: 0, incomplete: 0 }
+    }
+    const result = await storage.adoptLegacyCowBlobs()
+    // Drop any cached content/stat so a re-read hits the freshly adopted blobs.
+    this.contentCache.clear()
+    this.statCache.clear()
+    return result
+  }
+
   // ============= File Operations =============
 
   /**
@@ -360,6 +407,14 @@ export class VirtualFileSystem implements IVirtualFileSystem {
   async readFile(path: string, options?: ReadOptions): Promise<Buffer> {
     await this.ensureInitialized()
 
+    // Temporal read: the file's exact bytes as of a past generation or
+    // instant. Materializes the entity from the generation history — content
+    // blobs referenced by any in-window generation are retention-protected,
+    // so the bytes are guaranteed present. Bypasses the content cache.
+    if (options?.asOf !== undefined) {
+      return this.readFileAt(path, options.asOf, options)
+    }
+
     // Check cache first
     if (options?.cache !== false && this.contentCache.has(path)) {
       const cached = this.contentCache.get(path)!
@@ -426,6 +481,122 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     }
   }
 
+  /**
+   * @description The temporal read behind `readFile(path, { asOf })`: resolve
+   * the path's CURRENT entity, materialize its state at the target
+   * generation/instant from the Model-B history, and read that version's
+   * content blob (retention-protected, so present for every in-window
+   * generation). The pinned view is always released so compaction is never
+   * blocked by a read.
+   * @param path - The file path (resolved against the live tree).
+   * @param asOf - A generation number or wall-clock `Date`.
+   * @param options - `encoding` applies; caching does not (never cached).
+   * @returns The exact bytes the file held at that generation.
+   * @throws VFSError ENOENT when the file did not exist at that generation;
+   *   the generation store's compacted-generation error when `asOf` is past
+   *   the retention window's horizon.
+   */
+  private async readFileAt(
+    path: string,
+    asOf: number | Date,
+    options?: ReadOptions
+  ): Promise<Buffer> {
+    const entityId = await this.pathResolver.resolve(path)
+    const db = await this.brain.asOf(asOf)
+    try {
+      const entity = await db.get(entityId)
+      if (!entity) {
+        throw new VFSError(
+          VFSErrorCode.ENOENT,
+          `File did not exist as of ${String(asOf)}: ${path}`,
+          path,
+          'readFile'
+        )
+      }
+      const storage = (entity.metadata as Record<string, any> | undefined)?.storage
+      if (storage?.type !== 'blob' || typeof storage.hash !== 'string') {
+        throw new VFSError(
+          VFSErrorCode.EIO,
+          `File had no blob storage as of ${String(asOf)}: ${path}`,
+          path,
+          'readFile'
+        )
+      }
+      const content = await this.blobStorage.read(storage.hash)
+      if (options?.encoding) {
+        return Buffer.from(content.toString(options.encoding))
+      }
+      return content
+    } finally {
+      await db.release()
+    }
+  }
+
+  /**
+   * @description A file's version history within the retention window: one
+   * entry per generation that wrote the file, ascending — the newest entry is
+   * the current state. Pair with `readFile(path, { asOf: entry.generation })`
+   * to fetch any version's exact bytes, and restore by writing those bytes
+   * back (a NEW write — history is never rewritten). Bounded by the retention
+   * window: versions whose generations were compacted are not listed.
+   * @param path - The file path (resolved against the live tree).
+   * @returns The versions, oldest first.
+   * @example
+   * const versions = await brain.vfs.history('/pages/home.json')
+   * const before = await brain.vfs.readFile('/pages/home.json', {
+   *   asOf: versions[versions.length - 2].generation
+   * })
+   */
+  async history(path: string): Promise<FileVersion[]> {
+    await this.ensureInitialized()
+    const entityId = await this.pathResolver.resolve(path)
+
+    // Boundary note: reaches Brainy's internal generation store (the same
+    // bracket-access precedent as the blobStorage getter) — the per-id
+    // generation chain and horizon are not on the public Brainy surface.
+    const generationStore = (this.brain as unknown as {
+      generationStore: {
+        horizon(): number
+        generation(): number
+        generationsTouching(
+          kind: 'noun' | 'verb',
+          id: string,
+          fromGen: number,
+          toGen: number
+        ): Promise<number[]>
+      }
+    }).generationStore
+
+    const gens = await generationStore.generationsTouching(
+      'noun',
+      entityId,
+      generationStore.horizon(),
+      generationStore.generation()
+    )
+
+    const versions: FileVersion[] = []
+    for (const gen of gens) {
+      const db = await this.brain.asOf(gen)
+      try {
+        const entity = await db.get(entityId)
+        const meta = entity?.metadata as Record<string, any> | undefined
+        const storage = meta?.storage
+        if (storage?.type === 'blob' && typeof storage.hash === 'string') {
+          versions.push({
+            generation: gen,
+            timestamp: db.timestamp,
+            hash: storage.hash,
+            size: typeof meta?.size === 'number' ? meta.size : (storage.size ?? 0),
+            ...(typeof meta?.mimeType === 'string' && { mimeType: meta.mimeType })
+          })
+        }
+      } finally {
+        await db.release()
+      }
+    }
+    return versions
+  }
+
   /**
    * Write a file
    */
@@ -447,14 +618,18 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     // Ensure parent directory exists
     const parentId = await this.ensureDirectory(parentPath)
 
-    // Check if file already exists
+    // Check if file already exists (and capture its current content hash so
+    // the overwrite can release the superseded live reference afterwards).
     let existingId: string | null = null
+    let previousHash: string | undefined
     try {
       existingId = await this.pathResolver.resolve(path, { cache: false })
       // Verify the entity still exists in the brain
       const existing = await this.brain.get(existingId)
       if (!existing) {
         existingId = null  // Entity was deleted but cache wasn't cleared
+      } else if (existing.metadata?.storage?.type === 'blob') {
+        previousHash = existing.metadata.storage.hash as string | undefined
       }
     } catch (err) {
       // File doesn't exist, which is fine
@@ -505,14 +680,32 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       Object.assign(metadata, await this.extractMetadata(buffer, mimeType))
     }
 
+    // For embedding: use text content, for storage: use raw data. Computed
+    // for BOTH branches — an overwrite must refresh the entity's `data` (and
+    // therefore its embedding), or semantic search and any `data` read keep
+    // serving the FIRST version's text forever.
+    const embeddingData = mimeDetector.isTextFile(mimeType)
+      ? buffer.toString('utf-8')
+      : `File: ${name} (${mimeType}, ${buffer.length} bytes)`
+
     if (existingId) {
-      // Update existing file
-      // No entity.data - content is in BlobStorage
+      // Update existing file — content bytes live in BlobStorage; `data`
+      // carries the embedding text and MUST track the new content.
       await this.brain.update({
         id: existingId,
+        data: embeddingData,
         metadata
       })
 
+      // The update committed: drop the superseded live reference. When the
+      // content is unchanged this cancels the dedup increment the write()
+      // above just made (net: one live reference per referencing file); when
+      // it changed, the old bytes stay retention-protected for asOf reads
+      // via the update's own generation record.
+      if (previousHash) {
+        await this.blobStorage.release(previousHash)
+      }
+
       // Ensure Contains relationship exists (fix for missing relationships)
       const existingRelations = await this.brain.related({
         from: parentId,
@@ -532,9 +725,6 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       }
     } else {
       // Create new file entity
-      // For embedding: use text content, for storage: use raw data
-      const embeddingData = mimeDetector.isTextFile(mimeType) ? buffer.toString('utf-8') : `File: ${name} (${mimeType}, ${buffer.length} bytes)`
-
       const entity = await this.brain.add({
         data: embeddingData,  // Always provide string for embeddings
         type: this.getFileNounType(mimeType),
@@ -603,14 +793,21 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       throw new VFSError(VFSErrorCode.EISDIR, `Is a directory: ${path}`, path, 'unlink')
     }
 
-    // Delete blob from BlobStorage (decrements ref count)
-    if (entity.metadata.storage?.type === 'blob') {
-      await this.blobStorage.delete(entity.metadata.storage.hash)
-    }
-
-    // Delete the entity
+    // Delete the entity FIRST, then drop its live blob reference — never
+    // release a reference while the referencing entity might survive (a
+    // failed remove must not leave a live file whose bytes compaction could
+    // later reclaim).
     await this.brain.remove(entityId)
 
+    // Drop the file's LIVE reference to its content blob. The bytes are NOT
+    // deleted — the remove's own generation record references this hash, so
+    // `readFile(path, { asOf })` keeps serving it inside the retention
+    // window; history compaction reclaims the bytes when the last referencing
+    // generation is reclaimed.
+    if (entity.metadata.storage?.type === 'blob') {
+      await this.blobStorage.release(entity.metadata.storage.hash)
+    }
+
     // Invalidate caches
     this.pathResolver.invalidatePath(path)
     this.invalidateCaches(path)
@@ -986,23 +1183,26 @@ export class VirtualFileSystem implements IVirtualFileSystem {
       // Phase 1: Gather all descendants in ONE batch fetch
       const descendants = await this.gatherDescendants(entityId, Infinity)
 
-      // Phase 2: Parallel blob cleanup (chunked to avoid overwhelming storage)
-      // Blob deletion is reference-counted, so safe to call for all files
+      // Phase 2: Batch delete all entities (including root directory) FIRST —
+      // never release a blob reference while its referencing entity might
+      // survive a failed delete (see unlink's ordering note).
+      const allIds = [...descendants.map(d => d.id), entityId]
+      await this.brain.removeMany({ ids: allIds, continueOnError: false })
+
+      // Phase 3: Drop the deleted files' LIVE blob references (chunked).
+      // Live-reference drops only — the bytes stay for in-window asOf reads
+      // and are reclaimed by history compaction (see unlink's note).
       const blobFiles = descendants.filter(d =>
         d.metadata.vfsType === 'file' && d.metadata.storage?.type === 'blob'
       )
 
-      const BLOB_CHUNK_SIZE = 20  // Parallel delete 20 blobs at a time
+      const BLOB_CHUNK_SIZE = 20  // Parallel release 20 blob references at a time
       for (let i = 0; i < blobFiles.length; i += BLOB_CHUNK_SIZE) {
         const chunk = blobFiles.slice(i, i + BLOB_CHUNK_SIZE)
         await Promise.all(chunk.map(f =>
-          this.blobStorage.delete(f.metadata.storage!.hash)
+          this.blobStorage.release(f.metadata.storage!.hash)
         ))
       }
-
-      // Phase 3: Batch delete all entities (including root directory)
-      const allIds = [...descendants.map(d => d.id), entityId]
-      await this.brain.removeMany({ ids: allIds, continueOnError: false })
     } else {
       // No children or not recursive - just delete the directory entity
       await this.brain.remove(entityId)
@@ -1546,6 +1746,10 @@ export class VirtualFileSystem implements IVirtualFileSystem {
         }
       }
     }, 60000)  // Every minute
+    // Cache maintenance must never keep the host process alive.
+    if (this.backgroundTimer && typeof this.backgroundTimer.unref === 'function') {
+      this.backgroundTimer.unref()
+    }
   }
 
   private getDefaultConfig(): Required<Omit<VFSConfig, 'rootEntityId'>> & { rootEntityId?: string } {
@@ -1750,15 +1954,27 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     const newParentPath = this.getParentPath(newPath)
 
     if (oldParentPath !== newParentPath) {
-      // Remove from old parent
+      // Remove the OLD parent's containment edge(s) — by edge id, resolved from
+      // the graph's own adjacency (the removal law: a removal never requires
+      // reading the thing being removed). This step used to be skipped as "not
+      // critical", which left the moved entity a child of BOTH directories:
+      // readdir(oldDir) kept listing it, re-creating the old path showed the
+      // name twice, and tree-walking consumers saw the file in two places.
       if (oldParentPath) {
         const oldParentId = await this.pathResolver.resolve(oldParentPath)
-        // unrelate takes the relation ID, not params - need to find and remove relation
-        // For now, skip unrelate as it's not critical for rename
+        const staleEdges = await this.brain.related({
+          from: oldParentId,
+          to: entityId,
+          type: VerbType.Contains
+        })
+        for (const edge of staleEdges) {
+          await this.brain.unrelate(edge.id)
+        }
       }
 
-      // Add to new parent
-      if (newParentPath && newParentPath !== '/') {
+      // Add to the new parent. The root ('/') is a REAL parent — skipping it
+      // orphaned a move-to-root out of readdir('/') entirely.
+      if (newParentPath) {
         const newParentId = await this.pathResolver.resolve(newParentPath)
         await this.brain.relate({
           from: newParentId,
@@ -1799,6 +2015,95 @@ export class VirtualFileSystem implements IVirtualFileSystem {
     this.triggerWatchers(newPath, 'rename')
   }
 
+  /**
+   * Reconcile every VFS entity's containment edges against its canonical
+   * `metadata.path` — the path is the truth (maintained by write/rename); the
+   * `Contains` edges are a projection of it. Heals the "cosmetic ghost" class
+   * left by pre-fix renames that added the new parent's edge without removing
+   * the old one (an entity listed in TWO directories; a re-created old path
+   * showing its name twice), plus duplicate edges from the same parent left by
+   * concurrent writers.
+   *
+   * CONSERVATIVE by design: only VFS containment edges (subtype
+   * `'vfs-contains'` or `metadata.isVFS`) are ever touched — a user's own
+   * knowledge-graph `Contains` edge between the same entities is never
+   * removed. An entity whose expected parent path has no entity is logged
+   * loudly and left alone (never orphaned further). Operator-invoked via
+   * `brain.repairIndex()`.
+   *
+   * The canonical pagination walk (not an index query) is deliberate: this is
+   * a repair op — the projections are the thing under suspicion, so the walk
+   * reads the source of truth.
+   *
+   * @returns Counts of stale edges removed and missing expected edges restored.
+   */
+  async repairContainment(): Promise<{ removed: number; restored: number }> {
+    await this.ensureInitialized()
+
+    // Pass 1: canonical walk → every VFS entity's id + path.
+    const idByPath = new Map<string, string>()
+    const vfsEntities: Array<{ id: string; path: string }> = []
+    let cursor: string | undefined
+    for (;;) {
+      const page = await (this.brain as any).storage.getNounsWithPagination({ limit: 500, cursor })
+      for (const noun of page.items) {
+        const meta = (noun as any).metadata ?? noun
+        const p = meta?.path
+        if (meta?.vfsType && typeof p === 'string') {
+          idByPath.set(p, noun.id)
+          vfsEntities.push({ id: noun.id, path: p })
+        }
+      }
+      if (!page.hasMore) break
+      cursor = page.nextCursor
+    }
+
+    let removed = 0
+    let restored = 0
+    for (const { id, path } of vfsEntities) {
+      if (path === '/') continue // the root has no parent
+      const expectedParentId = idByPath.get(this.getParentPath(path))
+      if (!expectedParentId) {
+        console.warn(
+          `[VFS] repairContainment: no entity found for parent of ${path} — leaving its edges untouched.`
+        )
+        continue
+      }
+
+      const incoming = await this.brain.related({ to: id, type: VerbType.Contains })
+      let expectedSeen = false
+      for (const edge of incoming) {
+        const isVfsEdge = edge.subtype === 'vfs-contains' || (edge.metadata as any)?.isVFS === true
+        if (!isVfsEdge) continue // never touch user knowledge edges
+        if (edge.from === expectedParentId && !expectedSeen) {
+          expectedSeen = true // keep exactly one correct edge
+          continue
+        }
+        // Stale parent (a pre-fix rename ghost) or a duplicate of the correct
+        // edge (concurrent-writer artifact) — remove it, loudly.
+        await this.brain.unrelate(edge.id)
+        removed++
+        console.warn(
+          `[VFS] repairContainment: removed ${edge.from === expectedParentId ? 'duplicate' : 'stale'} ` +
+            `containment edge ${edge.from} -> ${id} (${path})`
+        )
+      }
+      if (!expectedSeen) {
+        await this.brain.relate({
+          from: expectedParentId,
+          to: id,
+          type: VerbType.Contains,
+          subtype: 'vfs-contains',
+          metadata: { isVFS: true }
+        })
+        restored++
+        console.warn(`[VFS] repairContainment: restored missing containment edge for ${path}`)
+      }
+    }
+
+    return { removed, restored }
+  }
+
   /**
    * Copy a file or directory to a new path.
    *

src/vfs/types.ts

@@ -195,6 +195,36 @@ export interface ReadOptions {
   // VFS-specific options
   cache?: boolean                // Use cache if available (default: true)
   decompress?: boolean           // Auto-decompress (default: true)
+
+  /**
+   * Read the file's content as it stood at a past generation (a number) or
+   * wall-clock instant (a Date) — the temporal read. Serves the exact bytes
+   * the file held then: the historical entity is materialized from the
+   * generation history, and its content blob is retention-protected (bytes
+   * referenced by any in-window generation are never reclaimed). Bounded by
+   * the retention window: reading past the compaction horizon throws.
+   * Bypasses the content cache.
+   */
+  asOf?: number | Date
+}
+
+/**
+ * One entry of a file's version history (see `vfs.history(path)`): the state
+ * the file held immediately after `generation` committed.
+ * `readFile(path, { asOf: generation })` returns these exact bytes while the
+ * generation remains inside the retention window.
+ */
+export interface FileVersion {
+  /** The generation whose write produced this version. */
+  generation: number
+  /** Commit timestamp of that generation (ms since epoch). */
+  timestamp: number
+  /** Content hash of this version's bytes in the content-addressed store. */
+  hash: string
+  /** Content size in bytes. */
+  size: number
+  /** Detected MIME type at that version. */
+  mimeType?: string
 }
 
 export interface MkdirOptions {