docs(8.0): RELEASES.md 8.0.0 release-candidate entry — full breaking-change inventory + upgrade guide

RELEASES.md

@@ -10,6 +10,292 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so
 
 ---
 
+## v8.0.0 — release candidate (unreleased, branch `feat/8.0-u64-ids`)
+
+**Affected products:** every consumer — this is a major release with removed
+surfaces, hard renames (no aliases, no deprecation period), and one flipped
+default. This entry **is** the migration guide: the find/replace pairs, the
+sed snippet, and the upgrade checklist below are the complete story — there is
+no separate migration doc. Latest published version remains 7.31.x until this
+ships.
+
+### Headline: Database as a Value
+
+8.0 replaces Brainy's two overlapping version-control subsystems (copy-on-write
+branching and per-entity versioning, ~5,100 LOC combined) with **one
+mechanism**: generational MVCC over immutable, generation-stamped records,
+exposed through a Datomic-style immutable database value — the **`Db`**.
+
+```ts
+const db = brain.now()                  // pin the current state — O(1), no I/O
+
+await brain.transact([
+  { op: 'update', id: invoiceId, metadata: { status: 'paid' } }
+], { meta: { author: 'billing-service', reason: 'PO-7741' } })
+
+await db.get(invoiceId)                 // still 'pending' — pinned, forever
+await brain.get(invoiceId)              // 'paid' — live
+await db.release()                      // unpin when done
+```
+
+What you get:
+
+- **`brain.now()`** — pins the current generation as an immutable `Db` view.
+  True snapshot isolation: the view reads exactly its pinned state no matter
+  what commits afterwards, including deletes. Readers never block writers and
+  writers never block readers.
+- **`brain.transact(ops, { meta, ifAtGeneration })`** — atomic multi-write
+  batches (`add` / `update` / `remove` / `relate` / `unrelate`) committed as
+  exactly one generation. Either every operation applies or none do.
+  `ifAtGeneration` is whole-store compare-and-swap (`GenerationConflictError`
+  on conflict) — the big sibling of the per-entity `ifRev` CAS from 7.31.
+  `meta` is reified Datomic-style into an append-only transaction log,
+  readable via `brain.transactionLog()`.
+- **`brain.asOf(generation | Date | snapshotPath)`** — time travel with the
+  **full query surface**: `get()`, `find()` in every mode, semantic search,
+  graph traversal, cursors, aggregation, all at the pinned past state.
+- **`db.with(ops)`** — speculative writes applied in memory on top of a view.
+  Nothing touches disk, the generation counter, or the indexes. What-if
+  analysis, then `transact()` the same ops for real.
+- **`db.persist(path)`** — instant self-contained snapshots. On filesystem
+  storage they are built from hard links (no entity data is copied; later
+  writes to the source can never alter the snapshot because rewrites swap
+  inodes). `brain.restore(path, { confirm: true })` replaces the whole store
+  from one; `Brainy.load(path)` opens one read-only with the full query
+  surface.
+- **`db.since(olderDb)`** — exactly the entity and relationship ids that
+  committed transactions touched between two views.
+- **`brain.compactHistory({ retainGenerations, retainMs })`** — explicit
+  retention. Compaction never breaks a pinned read.
+
+The precise guarantees (and the honest limits — history granularity is
+`transact()` commits; single-operation writes advance the clock but produce no
+historical records) are documented in:
+
+- [docs/concepts/consistency-model.md](docs/concepts/consistency-model.md) —
+  the guarantees, each proven by a dedicated test in
+  `tests/integration/db-mvcc.test.ts`
+- [docs/guides/snapshots-and-time-travel.md](docs/guides/snapshots-and-time-travel.md)
+  — the recipes: backup, restore, time-travel debugging, what-if, audit trails
+- [docs/ADR-001-generational-mvcc.md](docs/ADR-001-generational-mvcc.md) — the
+  design record: persisted layout, commit protocol, crash recovery, proof table
+
+### Removed surfaces and their replacements
+
+| Removed in 8.0 | Replacement |
+|---|---|
+| `brain.fork(name)` | Speculation: `db.with(ops)` (in-memory). Long-lived writable copy: `brain.restore()` a persisted snapshot into a fresh data directory. |
+| `brain.checkout(branch)` | Open the snapshot you want — `Brainy.load(path)` read-only, or restore into its own directory. No in-place switching: every handle always sees one unambiguous store. |
+| `brain.listBranches()` / `brain.getCurrentBranch()` / `brain.deleteBranch()` | A "branch" is now a name → snapshot-path mapping owned by your application. |
+| `brain.commit({ message })` | `brain.transact(ops, { meta })` — every batch is an atomic, logged, time-travelable commit; audit fields live in the database, not in commit messages. |
+| `brain.getHistory()` / `brain.streamHistory()` | `brain.transactionLog({ limit })` + `db.since(olderDb)`. |
+| `brain.versions.*` (`save` / `list` / `compare` / `restore` / `prune`) | A pinned `Db` or persisted snapshot captures *every* entity at that moment; `asOf()` reads any entity's past state; `restore()` for whole-store rollback. |
+| `brain.data()` (DataAPI: `clear` / `import` / `export` / `getStats`) | `brain.clear()`, `brain.import()`, `db.persist(path)` (the full-fidelity export format), `brain.restore(path, { confirm: true })`, `brain.stats()`. |
+| Cloud + browser storage adapters (`s3` / `gcs` / `r2` / `opfs` storage types, Azure Blob, plus the `storage.branch` option) | `filesystem` and `memory` only. Cloud backup is now an operator concern: `db.persist()` produces a plain directory — sync it with `gsutil` / `aws s3 cp` / `rclone` / `azcopy`. Passing a removed storage type throws at construction with this exact guidance. |
+| Browser support | Node.js 22 LTS or Bun ≥ 1.0 (`engines` enforces `node: 22.x`); Deno works through its Node compatibility layer. |
+| `brain.migrateToDiskAnn()` / `brain.migrateToHnsw()` | Gone — there is one canonical `'vector'` provider slot. A registered native vector provider selects its own operating mode; there is nothing to call. |
+| CLI `fork` / `branch` / `checkout` / `migrate` | CLI `snapshot <path>` / `restore <path>` / `history` / `generation`. |
+
+If you used 7.x COW branches, **materialize every branch you care about while
+still on 7.x** (fork → export, or copy each branch's store) — 8.0 does not
+read COW branch state.
+
+### Renames — find/replace pairs
+
+Hard renames, no aliases. Every pair below is verified against the 8.0 source.
+
+| Brainy 7.x | Brainy 8.0 |
+|---|---|
+| `HnswProvider` (from `@soulcraft/brainy/plugin`) | `VectorIndexProvider` |
+| `HNSWIndex` (exported class) | `JsHnswVectorIndex` |
+| Provider keys `'hnsw'` and `'diskann'` | `'vector'` (the only key Brainy consults for the vector index) |
+| `config.hnsw = { quantization, vectorStorage }` | `config.vector = { recall?, quantization?, persistMode? }` (shape change — see below) |
+| `hnswPersistMode: 'immediate' \| 'deferred'` (top level) | `config.vector.persistMode` |
+| `config.storage.type: 's3' \| 'gcs' \| 'r2' \| 'opfs'` | `'filesystem'` (or `'memory'` / `'auto'`) |
+| `config.storage.branch` | Removed (no COW branches) |
+| `brain.stats().indexHealth.hnsw` | `brain.stats().indexHealth.vector` |
+| Storage adapter contract `saveHNSWData()` / `getHNSWData()` | `saveVectorIndexData()` / `getVectorIndexData()` |
+| Cache category `'hnsw'` (`CacheProvider` union) | `'vectors'` |
+
+Mechanical renames as one command (run from your repo root, review the diff):
+
+```bash
+find . -name '*.ts' -not -path '*/node_modules/*' -print0 | xargs -0 sed -i \
+  -e 's/\bHnswProvider\b/VectorIndexProvider/g' \
+  -e 's/\bHNSWIndex\b/JsHnswVectorIndex/g' \
+  -e 's/\bsaveHNSWData\b/saveVectorIndexData/g' \
+  -e 's/\bgetHNSWData\b/getVectorIndexData/g' \
+  -e 's/indexHealth\.hnsw\b/indexHealth.vector/g' \
+  -e "s/registerProvider('hnsw'/registerProvider('vector'/g" \
+  -e "s/registerProvider('diskann'/registerProvider('vector'/g" \
+  -e "s/getProvider('hnsw'/getProvider('vector'/g" \
+  -e "s/getProvider('diskann'/getProvider('vector'/g"
+```
+
+The config shape changes need a human (they are not 1:1 textual):
+
+```ts
+// 7.x
+new Brainy({
+  hnswPersistMode: 'deferred',
+  hnsw: { quantization: { enabled: true, bits: 8, rerankMultiplier: 3 } }
+})
+
+// 8.0
+new Brainy({
+  vector: {
+    recall: 'balanced',        // 'fast' | 'balanced' | 'accurate'
+    quantization: { enabled: true, bits: 8 },   // rerankMultiplier is fixed at 3
+    persistMode: 'deferred'
+  }
+})
+```
+
+`config.vector.recall` replaces the algorithm-internal HNSW knobs: `'balanced'`
+(the default) maps to exactly the 7.x default parameters, so an upgrade without
+explicit knobs changes nothing about search behaviour. `config.hnsw.vectorStorage`
+is removed. The on-disk vector index file names are **unchanged** (e.g.
+`hnsw-system.json`) — existing filesystem stores need no data migration for
+this rename; only the API surface moved.
+
+### Behavior changes
+
+- **`subtype` is required by default.** 7.30's opt-in strict mode is now the
+  default: every `add()` / `addMany()` / `update()` / `relate()` /
+  `relateMany()` / `updateRelation()` rejects writes whose type carries no
+  non-empty `subtype` (`src/brainy.ts:9759` — `requireSubtype ?? true`).
+  Escape hatches: `requireSubtype: false` (last-resort opt-out for legacy
+  data) or `requireSubtype: { except: [NounType.Thing, ...] }` (per-type
+  allowlist). Per-type rules registered via `brain.requireSubtype(type, opts)`
+  still compose. `brain.audit()` reports entries missing a subtype and
+  `brain.migrateField()` backfills them.
+- **Verb ids are Brainy-generated UUIDs — by contract.** `relate()` now throws
+  a teaching error if a caller passes an `id` field
+  (`src/utils/paramValidation.ts:526`). In 7.x a supplied id was silently
+  ignored (a generated UUID was used anyway); 8.0 says so out loud, because
+  graph-index providers key verb-int interning on the raw UUID bytes.
+- **`update({ vector })` is honored on its own.** 7.x only applied a supplied
+  pre-computed vector when `data` was also passed (it was silently dropped
+  otherwise). 8.0 applies an explicit `params.vector` with dimension
+  validation (`src/brainy.ts:1702-1713`; proven by
+  `tests/unit/brainy/update.test.ts:256`).
+- **`brain.clear()` re-resolves all indexes exactly as `init()` does** —
+  including plugin-provided vector/metadata/entityIdMapper factories and VFS
+  root re-creation. In 7.x, clearing a plugin-accelerated brain could leave
+  the metadata index rebuilt without its native id-mapper wiring.
+- **`find({ connected: { …, subtype } })` with `depth > 1` now works.** 7.30
+  threw `NOT_SUPPORTED` for multi-hop subtype-filtered traversal; 8.0
+  implements the BFS in the JS graph index and routes to a native provider
+  when one is registered.
+- **Every write advances the generation clock; only `transact()` writes
+  history.** Single-operation writes (`add` / `update` / `delete` / `relate`
+  outside `transact()`) bump `brain.generation()` so watermarks and CAS stay
+  sound, but they do not stage historical records — they remain visible
+  through earlier pins and are not reported by `db.since()`. Writes you want
+  to travel back through go through `transact()`. This is the documented
+  contract, stated rather than papered over.
+- **Unchanged from 7.31:** per-entity `_rev`, `update({ ifRev })`
+  (`RevisionConflictError`), and `add({ ifAbsent })` work exactly as before,
+  and `ifRev` is also accepted on `transact()` update operations (a conflict
+  rejects the whole batch).
+
+### For provider and plugin authors
+
+The provider contracts in `@soulcraft/brainy/plugin` (`src/plugin.ts`) changed
+shape:
+
+- **`GraphIndexProvider` speaks BigInt at the boundary.** Reads take entity
+  ints and return entity/verb ints as `bigint[]`; the coordinator owns all
+  UUID ↔ int conversion. `addVerb(verb, sourceInt, targetInt)` receives the
+  resolved endpoint ints (also mirrored on the new optional
+  `GraphVerb.sourceInt` / `GraphVerb.targetInt` fields) and returns the
+  interned verb int. The batch reverse resolver
+  `verbIntsToIds(verbInts: bigint[]): Promise<(string | null)[]>` is
+  **required** — the provider owns durable verb-int interning; Brainy keeps
+  only a bounded in-memory warm cache. Implementations may stay u32 internally
+  (`Number(bigint)` narrowing is lossless under the `EntityIdSpaceExceeded`
+  guard) but must speak the bigint contract.
+- **`VersionedIndexProvider`** is a new optional 4-method capability —
+  `generation()`, `isGenerationVisible(g)`, `pin(g)`, `release(g)`, BigInt
+  generations — feature-detected on every registered index provider. Providers
+  are *post-commit appliers*: the storage-record commit is the source of
+  truth; on open, a provider behind the committed watermark replays the gap or
+  requests a rebuild. Explicit pins override any time-based snapshot retention
+  the provider has. Speculative `db.with()` overlays never reach providers. A
+  provider implementing this serves historical reads with **no rebuild** —
+  the open-core materializer is the correctness baseline, the provider is the
+  accelerator.
+- **The vector index registers under `'vector'`** and implements
+  `VectorIndexProvider`. The `'hnsw'` and `'diskann'` keys are retired and
+  never looked up. `CacheProvider`'s category union uses `'vectors'` in place
+  of `'hnsw'`.
+
+### Scale and cost — how the mechanisms behave
+
+Mechanism descriptions, not benchmark numbers (none are published for 8.0 yet):
+
+- `brain.now()` pins in O(1) and adds **zero read overhead until history
+  actually moves** — while nothing has committed past the pin, every read
+  delegates to the live fast paths.
+- A `transact()` commit pays O(ids touched) extra writes (before-images +
+  delta + manifest) — never O(store size). Single-operation writes pay an
+  in-memory counter bump with coalesced persistence.
+- `db.persist()` on filesystem storage is a hard-link farm: snapshot creation
+  copies no entity data and the snapshot shares disk space with the source.
+  Cross-device targets fall back to per-file byte copies; persisting an
+  in-memory brain serializes a real, durable, loadable store.
+- Historical **index-accelerated** queries (semantic search, traversal,
+  cursors, aggregation) on the open-core path pay a one-time O(n at the
+  pinned generation) in-memory index materialization per `Db`, cached until
+  `release()`. Record-path reads (`get`, metadata `find`, filter `related`)
+  at any reachable generation are served directly from the immutable record
+  layer. A native `VersionedIndexProvider` serves the same reads rebuild-free.
+
+The full cost model is in
+[docs/concepts/consistency-model.md](docs/concepts/consistency-model.md).
+
+### Upgrade checklist (from 7.x)
+
+1. **While still on 7.x:** back up your data directory (a plain file copy is
+   fine). If you used COW branches, materialize each branch you need — 8.0
+   does not read branch state. If you ran a cloud storage adapter, export your
+   data with 7.x tooling (`(await brain.data()).export()`) — 8.0 opens
+   `filesystem` and `memory` stores only.
+2. **Runtime:** Node.js 22 LTS or Bun ≥ 1.0.
+3. **Config:** change removed storage types to `'filesystem'`; delete
+   `storage.branch`; move `config.hnsw` / `hnswPersistMode` to
+   `config.vector` per the shape example above.
+4. **Renames:** run the sed snippet, then fix the manual shape changes.
+5. **Removed APIs:** replace `fork` / `checkout` / branch methods / `commit` /
+   `getHistory` / `streamHistory` / `versions` / `data()` per the table above.
+6. **Subtypes:** if your data predates subtype discipline, start with
+   `requireSubtype: false`, run `brain.audit()`, backfill with
+   `brain.migrateField()`, then remove the opt-out so the 8.0 default
+   enforcement protects you going forward.
+7. **CLI scripts:** `fork` / `branch` / `checkout` / `migrate` →
+   `snapshot <path>` / `restore <path>` / `history` / `generation`.
+8. **Plugin authors:** apply the contract renames and the BigInt graph
+   contract; optionally implement `VersionedIndexProvider`.
+9. **Verify, then take your first snapshot:** run your test suite, then
+   `const db = brain.now(); await db.persist('/backups/post-8.0-upgrade');
+   await db.release()`.
+
+What did **not** change: the core query surface (`find` / `search` / `get` /
+`add` / `update` / `relate` and friends), the aggregation engine, the VFS,
+neural extraction, and the on-disk entity/relationship layout — 8.0 creates
+its MVCC bookkeeping (`_system/generation.json`, `_system/manifest.json`,
+`_system/tx-log.jsonl`, `_generations/`) alongside the existing files on
+first use.
+
+### Cortex compatibility
+
+Brainy 8.0 pairs with `@soulcraft/cortex` 3.0 (shipping in lockstep). Cortex
+2.x cannot accelerate Brainy 8.0: 8.0 never consults the `'hnsw'` /
+`'diskann'` provider keys, and the graph/column provider contracts are BigInt
+at the boundary. Upgrade both packages together.
+
+---
+
 ## v7.31.2 — 2026-06-09
 
 **Affected products:** none in behaviour; affects anyone reading Brainy's TypeScript