chore: the forge is the address — retire the archived mirror from every live surface

Ruled today: the project's one public home is source.soulcraft.com. The
old public repo is archived history and no longer part of any release.

- package.json repository/homepage/bugs now point at the forge (this is
  what the npm page links as Repository/Homepage/Issues)
- README CI badge reads the forge pipeline; CONTRIBUTING drops the
  mirror paragraph (forge account or email patch were already the ruled
  contribution paths)
- release.sh: mirror push + external release step removed; publishes go
  forge-first (box-held write token, temp userconfig so the token never
  hits argv; a forge-publish failure aborts before the storefront so the
  pair can never diverge), then npmjs with the scope-override pin (the
  fleet npmrc maps @soulcraft to the forge and scope mappings beat
  --registry); release page created via forge API when a token is
  present, loud skip otherwise; changelog compare links point home
- dead external CI workflow removed (.forgejo/workflows/ci.yml is the
  live pipeline)

Historical CHANGELOG links to the archive stay as written - history is
history and the archive serves them read-only.

CHANGELOG.md

@@ -2,6 +2,59 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [8.10.0](https://github.com/soulcraftlabs/brainy/compare/v8.9.0...v8.10.0) (2026-07-23)
+
+- docs: adoption storefront — contributing guide, security policy, README support + cor section (9a99a7b)
+- fix(release): push the public mirror explicitly and verify the tag lands at the right commit before publishing (6ba94c8)
+- docs: project guide version line points at npm instead of a hardcoded stale number (3a1efc9)
+- feat: vector provider identity is a required name field (hnsw-js), rendered [vector-index:<name>] (3be4ba9)
+- feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names (55b867c)
+
+
+### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19)
+
+- docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78)
+- fix: release drains in-flight writer-lock heartbeat — no phantom lock after unlink (70e4bc8)
+- feat: flush() never compacts — history maintenance moves to close() with bounded passes (300d9f2)
+
+
+### [8.8.2](https://github.com/soulcraftlabs/brainy/compare/v8.8.1...v8.8.2) (2026-07-19)
+
+- fix: one field-resolution law across aggregation hooks, source.where, removeMany, and find() spellings (945d92d)
+- chore: push public docs to the soulcraft.com ingest door on release (42037d0)
+
+
+### [8.8.1](https://github.com/soulcraftlabs/brainy/compare/v8.8.0...v8.8.1) (2026-07-18)
+
+- fix: O(1) adaptive retention accounting + historyStats fleet audit (6207e48)
+- fix: import dedup off-switch honesty + brain-owned lifecycle for the background pass (4fcef7b)
+
+
+### [8.8.0](https://github.com/soulcraftlabs/brainy/compare/v8.7.1...v8.8.0) (2026-07-17)
+
+- feat: OS-limit detection for pool-scale deployments (16a73b8)
+
+
+### [8.7.1](https://github.com/soulcraftlabs/brainy/compare/v8.7.0...v8.7.1) (2026-07-17)
+
+- fix: race-proof writer-lock acquisition + machine-readable conflict through init (01a3b46)
+
+
+### [8.7.0](https://github.com/soulcraftlabs/brainy/compare/v8.6.0...v8.7.0) (2026-07-17)
+
+- feat: scaled transact budgets + labeled timeout diagnostics + envelope docs (6ef9fcb)
+
+
+### [8.6.0](https://github.com/soulcraftlabs/brainy/compare/v8.5.2...v8.6.0) (2026-07-17)
+
+- feat: brain.auditGraph() — read-only graph-truth audit (2a03fae)
+
+
+### [8.5.2](https://github.com/soulcraftlabs/brainy/compare/v8.5.1...v8.5.2) (2026-07-17)
+
+- fix: exception-safe aggregation backfill + generation-verified adoption + loud open-path guards (a77b064)
+
+
 ### [8.5.1](https://github.com/soulcraftlabs/brainy/compare/v8.5.0...v8.5.1) (2026-07-17)
 
 - fix: aggregation state adoption on reopen + single-flight backfill + query-cap ratchet removal (da55be7)

CLAUDE.md

@@ -12,7 +12,7 @@ Handoff file: `/home/dpsifr/.strategy/PLATFORM-HANDOFF.md`
 
 **Brainy's current open actions:** None. MIT open-source — no platform-specific actions.
 
-**Current version:** `@soulcraft/brainy@7.31.5` (latest published; 8.0.0 release candidate on `feat/8.0-u64-ids`)
+**Current version:** run `npm view @soulcraft/brainy version` (never trust a hardcoded number here — this line went stale for months); consumer-facing changes tracked in `RELEASES.md`
 
 ---
 

CONTRIBUTING.md

@@ -1,298 +1,66 @@
 # Contributing to Brainy
 
-Thank you for your interest in contributing to Brainy! This document provides guidelines and instructions for contributing to the project.
+Brainy is MIT-licensed and genuinely open to outside contributions. This page
+is the honest, current path — please don't rely on older instructions you
+may find elsewhere in the repo's history.
 
-## Code of Conduct
+## Where the project lives
 
-By participating in this project, you agree to abide by our Code of Conduct:
+The source of truth is a self-hosted forge: **source.soulcraft.com/soulcraft/brainy**.
-- Be respectful and inclusive
+It's anonymously readable and cloneable — no account needed to browse, clone,
-- Welcome newcomers and help them get started
+or build.
-- Focus on constructive criticism
-- Respect differing viewpoints and experiences
 
-## How to Contribute
+## How to contribute
 
-### Reporting Issues
+**Found a bug, or have an idea?** Email **brainy@soulcraft.com**. No account,
+no ceremony — you'll get a receipt, and it goes to a human.
 
-Before creating an issue, please check existing issues to avoid duplicates.
+**Want to send a patch?** Two ways, both first-class:
 
-When creating an issue, include:
+- **Email a patch.** Run `git format-patch` against your change and email the
-- Clear, descriptive title
+  output to **brainy@soulcraft.com**. This is a genuinely supported path, not
-- Detailed description of the problem
+  a fallback — plenty of good contributions arrive this way.
-- Steps to reproduce
+- **Open a pull request on the forge.** Request an account at
-- Expected vs actual behavior
+  **source.soulcraft.com** (registration is request-with-approval, so allow
-- System information (OS, Node version, Brainy version)
+  a little lag), clone, push a branch, and open a PR there. Maintainers
-- Code examples if applicable
+  review and land it.
 
-### Suggesting Features
+Either way, for anything beyond a small fix, opening an issue first (email is
+fine) to talk through the approach saves everyone rework.
 
-Feature requests are welcome! Please provide:
+## Development setup
-- Clear use case
-- Proposed API/interface
-- Examples of how it would work
-- Any potential challenges or considerations
 
-### Pull Requests
-
-#### Before Starting
-
-1. Check existing issues and PRs
-2. Open an issue to discuss significant changes
-3. Fork the repository
-4. Create a feature branch from `main`
-
-#### Development Setup
-
-**Quick Setup (Recommended):**
 ```bash
-# Clone your fork
+git clone https://source.soulcraft.com/soulcraft/brainy.git
-git clone https://github.com/your-username/brainy.git
 cd brainy
-
-# Run setup script (installs all dependencies including Rust)
-./scripts/setup-dev.sh
-```
-
-**Manual Setup:**
-```bash
-# Clone your fork
-git clone https://github.com/your-username/brainy.git
-cd brainy
-
-# Install system dependencies (Ubuntu/Debian)
-sudo apt-get install -y build-essential pkg-config libssl-dev
-
-# Install Rust (for WASM embedding engine)
-curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
-source ~/.cargo/env
-rustup target add wasm32-unknown-unknown
-cargo install wasm-pack
-
-# Install Node.js dependencies
 npm install
-
-# Build Candle WASM embedding engine
-npm run build:candle
-
-# Build TypeScript
 npm run build
-
-# Run tests
 npm test
 ```
 
-#### Making Changes
+Tests run on [Vitest](https://vitest.dev/). `npm test` runs the unit suite;
+see `package.json` for `test:integration`, `test:coverage`, and friends.
 
-1. **Follow the code style**
+## Standards
-   - TypeScript for all source code
-   - Clear variable and function names
-   - Comments for complex logic
-   - JSDoc for public APIs
 
-2. **Write tests**
+- **Strict TypeScript.** No `any` escape hatches to dodge the type checker.
-   - Add tests for new features
+- **Tests exercise real behavior.** No mocking away the thing you're supposed
-   - Update tests for changes
+  to be testing.
-   - Ensure all tests pass
+- **No stubs, no TODO-code.** If something can't be finished, say so and
+  leave it out — don't merge a placeholder.
+- **JSDoc on every exported function, class, and type.**
+- **[Conventional Commits](https://www.conventionalcommits.org/).** `feat:`,
+  `fix:`, `docs:`, `perf:`, `refactor:`, `test:`, `chore:`. Never
+  `BREAKING CHANGE` in a commit message — major version bumps are a separate,
+  deliberate decision.
+- **Performance claims are measured or labeled projected.** If a PR or its
+  description states a number, cite the benchmark that produced it (see
+  [docs/performance-envelopes.md](docs/performance-envelopes.md) for the
+  pattern). Don't state an estimate as if it were measured.
 
-3. **Update documentation**
+## License
-   - Update README if needed
-   - Add/update API documentation
-   - Include examples
 
-#### Commit Guidelines
+Brainy is [MIT licensed](LICENSE). Contributions are accepted under the same
+license — there's no CLA to sign.
 
-Follow conventional commits format:
+Thank you for considering a contribution.
-
-```
-type(scope): description
-
-[optional body]
-
-[optional footer]
-```
-
-Types:
-- `feat`: New feature
-- `fix`: Bug fix
-- `docs`: Documentation changes
-- `style`: Code style changes
-- `refactor`: Code refactoring
-- `perf`: Performance improvements
-- `test`: Test changes
-- `chore`: Build/tooling changes
-
-Examples:
-```bash
-feat(triple): add graph traversal depth limit
-fix(storage): handle concurrent write conflicts
-docs(api): update search method documentation
-```
-
-#### Submitting PR
-
-1. Push to your fork
-2. Create PR against `main` branch
-3. Fill out PR template
-4. Ensure CI checks pass
-5. Wait for review
-
-### Testing
-
-#### Running Tests
-
-```bash
-# Run all tests
-npm test
-
-# Run specific test file
-npm test tests/core.test.ts
-
-# Run with coverage
-npm run test:coverage
-
-# Watch mode
-npm run test:watch
-```
-
-#### Writing Tests
-
-```typescript
-import { describe, it, expect } from 'vitest'
-import { Brainy } from '../src'
-
-describe('Feature Name', () => {
-  it('should do something specific', async () => {
-    const brain = new Brainy()
-    await brain.init()
-    
-    // Test implementation
-    const result = await brain.search("test")
-    
-    expect(result).toBeDefined()
-    expect(result.length).toBeGreaterThan(0)
-  })
-})
-```
-
-## Architecture Guidelines
-
-### Adding New Features
-
-1. **Check existing functionality**
-   - Review `ARCHITECTURE.md`
-   - Check if similar features exist
-   - Consider if it should be an augmentation
-
-2. **Design considerations**
-   - Maintain backward compatibility
-   - Consider performance impact
-   - Think about all storage adapters
-   - Plan for extensibility
-
-3. **Implementation checklist**
-   - [ ] Core functionality
-   - [ ] Tests (unit and integration)
-   - [ ] Documentation
-   - [ ] TypeScript types
-   - [ ] Examples
-   - [ ] Performance benchmarks (if applicable)
-
-### Creating Augmentations
-
-Augmentations extend Brainy's functionality:
-
-```typescript
-import { BrainyAugmentation } from '../types'
-
-export class MyAugmentation extends BrainyAugmentation {
-  name = 'MyAugmentation'
-  
-  async onInit(brain: Brainy): Promise<void> {
-    // Initialize augmentation
-  }
-  
-  async onAdd(item: any, brain: Brainy): Promise<any> {
-    // Process before adding
-    return item
-  }
-  
-  async onSearch(query: any, results: any[], brain: Brainy): Promise<any[]> {
-    // Process search results
-    return results
-  }
-}
-```
-
-### Performance Considerations
-
-- Use batch operations where possible
-- Implement caching strategically
-- Consider memory usage
-- Profile performance impacts
-- Add benchmarks for critical paths
-
-## Documentation
-
-### API Documentation
-
-Use JSDoc for all public APIs:
-
-```typescript
-/**
- * Searches for similar items using vector similarity
- * @param query - Search query (text or vector)
- * @param options - Search options
- * @returns Array of search results with scores
- * @example
- * ```typescript
- * const results = await brain.search("machine learning", { limit: 10 })
- * ```
- */
-async search(query: string | Vector, options?: SearchOptions): Promise<SearchResult[]> {
-  // Implementation
-}
-```
-
-### Examples
-
-Add examples for new features:
-
-```typescript
-// examples/feature-name.ts
-import { Brainy } from 'brainy'
-
-async function exampleUsage() {
-  const brain = new Brainy()
-  await brain.init()
-  
-  // Show feature usage
-  // Include comments explaining what's happening
-  // Handle errors appropriately
-}
-
-exampleUsage().catch(console.error)
-```
-
-## Release Process
-
-1. **Version bump**: Follow semantic versioning
-2. **Update CHANGELOG**: Document all changes
-3. **Run tests**: Ensure all tests pass
-4. **Build**: Generate distribution files
-5. **Tag**: Create git tag for version
-6. **Publish**: Release to npm
-
-## Getting Help
-
-- **Discord**: Join our community
-- **Issues**: Ask questions on GitHub
-- **Discussions**: Share ideas and get feedback
-
-## Recognition
-
-Contributors will be recognized in:
-- CHANGELOG.md for their contributions
-- README.md contributors section
-- GitHub contributors page
-
-Thank you for contributing to Brainy! 🧠

README.md

@@ -13,7 +13,7 @@
 <p align="center">
   <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
   <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
-  <a href="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml"><img src="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://source.soulcraft.com/soulcraft/brainy/actions"><img src="https://source.soulcraft.com/soulcraft/brainy/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
   <a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
   <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
   <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>
@@ -23,8 +23,9 @@
   <a href="#quick-start">Quick start</a> ·
   <a href="#one-query-three-engines">One query</a> ·
   <a href="#feature-tour">Features</a> ·
-  <a href="#from-laptop-to-hundreds-of-millions">Scale with Cor</a> ·
+  <a href="#when-you-outgrow-brainy">Scale with Cor</a> ·
-  <a href="#documentation">Docs</a>
+  <a href="#documentation">Docs</a> ·
+  <a href="#support--community">Support</a>
 </p>
 
 ---
@@ -172,9 +173,11 @@ await brain.vfs.search('React components with hooks')          // semantic file
 
 **[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)**
 
-## From laptop to hundreds of millions
+## When you outgrow Brainy
 
-Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**:
+Brainy's pure-TypeScript engines carry real workloads a long way on their own — see the measured, per-operation numbers (not marketing figures) in **[docs/performance-envelopes.md](docs/performance-envelopes.md)** for what to expect, unaccelerated, on plain filesystem storage.
+
+When a deployment needs native-scale vector/graph performance — memory-mapped indexes that don't need your dataset in RAM, billion-scale ambitions — add the native engine. **The API doesn't change:**
 
 ```bash
 npm install @soulcraft/cor
@@ -187,13 +190,14 @@ await brain.init()   // @soulcraft/cor detected — same code, native engines un
 
 Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate.
 
-Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both.
+Open core, commercial accelerator: Brainy is MIT and complete on its own — Cor is more headroom for when you need it, not capability held back to sell you later. Licensing and support: **cor@soulcraft.com**.
 
 ## Performance
 
+- Per-operation p50/p95 at 1k and 10k entities, pure-JS floor, measured and re-run every release that touches a measured path: **[docs/performance-envelopes.md](docs/performance-envelopes.md)**.
 - JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41).
 - Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan.
-- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
+- Capacity planning and architecture: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
 
 ## Use cases
 
@@ -212,6 +216,10 @@ Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is
 
 **Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use.
 
-## Contributing & license
+## Support & community
 
-Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors.
+- **Bugs and ideas** → **brainy@soulcraft.com** — no account needed, you'll get a receipt.
+- **Security reports** → **security@soulcraft.com** — see **[SECURITY.md](SECURITY.md)**.
+- **Contributing** → see **[CONTRIBUTING.md](CONTRIBUTING.md)**.
+
+MIT © Brainy Contributors.

RELEASES.md

@@ -8,8 +8,307 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so
 - Debugging data, query, or storage behaviour
 - A new Brainy feature is available that you want to adopt
 
+## Removed APIs — 7.x → 8.x (the complete ledger)
+
+Every public API removed at the 8.0 major, with its sanctioned replacement. If your code
+still calls a left-column name on 8.x it throws (or the config key is rejected) — the
+replacement is always a one-line change. (Standing contract from 8.9.0 forward: removals
+happen only at majors, after ≥1 minor of loud runtime deprecation naming the replacement.)
+
+| Removed (7.x) | Replacement (8.x) |
+|---|---|
+| `brain.search(query, k)` | `find({ query })` — semantic; `find({ query, searchMode })` for hybrid |
+| `brain.getRelations({...})` | `related(id, opts)` for adjacency; `find({ connected: {...} })` for scoped traversal |
+| `brain.neural()` clustering | `find({ vector })` + aggregation `GROUP BY` |
+| `Db.search()` | `db.find({ vector })` |
+| Pre-8.0 storage path aliases (`directory`, `basePath`, …) | one `storage.path` key (old aliases throw) |
+| Reserved keys inside `metadata` bags (silently remapped in 7.x) | top-level params (`subtype`, `visibility`, `confidence`, `weight`, …) — reserved-in-bag throws |
+| 7.x COW branches layout (`branches/main/`) | generational MVCC (`asOf()`, `now()`, `db.persist(path)`) — on-disk migration is automatic at first 8.x open |
+
+The fork/snapshot family (`brain.snapshot()`, `createSnapshot()`, `restoreSnapshot()`)
+is sometimes cited as a 7.x removal — those methods never existed on 7.x; the 8.0 Db API
+(`asOf`/`persist`/`restore({confirm})`) is their first real implementation.
+
 ---
 
+## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
+
+From a production deployment's cold-restart incident: the FIRST writes after every
+restart on a large brain measured 33–35s each (page cache cold) against the transact
+apply budget — Brainy's own op-count-scaled budget, `max(30s, opCount × 2s)`, e.g.
+32,000ms for a 16-op batch. There is no external deadline in this story, and no
+post-completion veto either: every write is itself a multi-operation transaction (a
+single `add()` applies several operations — canonical writes, the vector-index insert,
+the metadata-index update), so ONE cold operation that legitimately runs ~33s consumes
+the whole budget, the gate before the NEXT operation trips, and the write rolls back
+atomically (zero loss, by design) — refused, retried, refused again, until the page
+cache warms passively (~30 minutes). The cure is not weaker atomicity; it is the two
+new knobs below — a budget floor sized for cold stores, and a warm contract that pays
+demand-load cost OFF the transaction path.
+
+- **The budget's start-gating contract is now explicit, documented, and pinned by
+  regression tests.** The budget gates STARTING the next operation — completed work is
+  never rolled back for elapsed time — and a transaction's first operation now
+  unconditionally starts by code, not merely because elapsed time happens to be ~0 when
+  it is checked. This has been the shipped schedule since 8.7.0 (no behavior change for
+  existing integrations); it is now stated in `Transaction.execute()`'s contract JSDoc
+  and enforced by tests so it cannot silently regress. Mid-batch atomicity is unchanged:
+  a trip before operation `i+1` still rolls back `0..i` and throws a retryable
+  `TransactionTimeoutError`.
+- **The budget's 30s floor is now configurable**: `new Brainy({ transactionBudgetFloorMs })`
+  raises (or lowers) the floor of `max(transactionBudgetFloorMs, opCount × 2000)` for every
+  internal transact batch. Useful for a store whose cold writes legitimately run past 30s
+  per operation, so a bulk batch gets a proportionally larger runway instead of tripping
+  mid-batch on cold-cache latency.
+- **New: `brain.warm()`** — eagerly loads/faults-in the vector index, metadata index, and
+  graph adjacency so the first real operation after a cold restart runs at steady-state
+  cost instead of paying demand-load latency on the critical path. Returns a `WarmReport`
+  with one honest outcome per surface — never conflate the first two:
+  - `'warmed'` — the surface's own provider `warm()` hook ran, or a full-hydration seam
+    loaded every shard/field/segment from storage. Steady-state cost is paid.
+  - `'probed'` — no `warm()` hook was available, so a best-effort read (one `search()` call
+    for the vector index) faulted in *some* backing storage as a side effect — real work,
+    but never reported as `'warmed'`.
+  - `'unavailable'` — nothing ran (no hook, no hydration seam, or nothing to probe).
+- **New config: `warmOnOpen: true`** makes `init()` await `brain.warm()` before it resolves
+  — a deliberate blocking trade-off: startup takes longer, the first request doesn't.
+  Default `false` (unchanged lazy behavior).
+- **New optional provider hook: `warm?(): Promise<void>`** on the vector and graph
+  acceleration provider contracts (`src/plugin.ts`) — a native provider can implement it to
+  eagerly pretouch its own backing storage (e.g. mmap pretouch); absence means brainy falls
+  back to the probe/hydration behavior above.
+- **Transaction op-name strings changed in journals/timings**: the vector-index
+  transaction operations were renamed from `AddToHNSW`/`RemoveFromHNSW` to backend-neutral
+  `AddToVectorIndex(...)`/`RemoveFromVectorIndex(...)` — the old names hard-coded an
+  algorithm that may not be the one actually running (a non-HNSW native vector provider
+  emitting `"RemoveFromHNSW"` has sent an operator hunting an index that doesn't exist).
+  The parenthesized suffix names the ACTIVE backend: `hnsw-js` for the built-in engine, or
+  the native provider's own identity when it self-identifies. **If you parse these op-name
+  strings** (log processors, journal tooling), update your matcher from
+  `AddToHNSW`/`RemoveFromHNSW` to `AddToVectorIndex(`/`RemoveFromVectorIndex(`.
+- **Provider identity is now a REQUIRED `name` field** on the vector provider contract
+  (`VectorIndexProvider.name`, `src/plugin.ts`) — every implementation self-reports its own
+  identity truthfully (its algorithm/engine), never inheriting a default. It renders as the
+  op-name suffix above and, wherever the vector index identifies itself in prose log lines,
+  as the tag `[vector-index:<name>]`. **Native provider adoption is a one-line change**:
+  declare `readonly name = '<your-provider-name>'`. A provider instance that still lacks
+  `name` at runtime (an older native build compiled against the previous, optional field) is
+  never crashed on and never silently mislabeled: it stamps `unknown-provider` and emits one
+  loud warning naming the missing field, so the gap is discoverable instead of a permanent
+  fossil label in every journal line.
+
+## v8.9.0 — 2026-07-19 (flush is durability-only: history maintenance moves to close())
+
+The write path stops paying maintenance costs — the last structural piece of the
+flush-storm class (a production deployment measured single writes blocked 25–191s behind
+history reclaim running inline on flush under memory pressure):
+
+- **`flush()` never compacts history.** It persists the current window's deltas and
+  nothing else — its cost no longer depends on history backlog or retention mode, in any
+  configuration. **`close()` is the auto-compaction site** (time-bounded per pass, ~5s;
+  an early stop is a consistent prefix and the next pass resumes).
+- **`compactHistory()` gains `timeBudgetMs`** — bound your own maintenance windows; the
+  same resumable-prefix guarantee applies.
+- **The documented trade**: a long-lived writer that never closes accumulates history
+  until its next explicit `compactHistory()`. Predictable writes, explicit maintenance.
+  If you run bounded retention on an always-on service, schedule a periodic
+  `compactHistory({ ...caps, timeBudgetMs })` in your maintenance window.
+- **New public doc: `docs/performance-envelopes.md`** — measured per-op envelopes
+  (p50/p95 at stated scales, hardware, and backend, with the measuring script cited).
+  Refresh rule going forward: any release touching a measured path re-runs that op's
+  benchmark and updates the envelope in the same release.
+- **New in this file: the Removed APIs 7.x→8.x table** (top of this document) — every
+  removal with its sanctioned replacement, one place, per the engine-currency contract.
+  Standing from here: removals only at majors, after ≥1 minor of loud runtime deprecation.
+
+## v8.8.2 — 2026-07-19 (one field-resolution law: reserved-field aggregates stop drifting)
+
+Four fixes from a consumer conformance audit, all rooted in the same disease — two field-resolution
+regimes where there must be one:
+
+- **Aggregates grouped by a RESERVED field (`subtype`, `visibility`, …) now decrement on
+  delete.** The delete/update hooks fed the aggregation engine a partial entity view (type,
+  service, data, metadata only), so a reserved-field `groupBy` resolved to a nonexistent group
+  on the way DOWN — counts drifted upward forever after any delete, and updates that moved an
+  entity between reserved-field groups double-counted it. The hooks now pass the full-fidelity
+  entity view (every reserved field top-level, the same shape the add path uses). If your
+  deployment derives stats from reserved-field aggregates, re-define those aggregates once
+  after upgrading (a changed definition triggers one rescan) or run them fresh — the drifted
+  persisted counts do not self-heal retroactively.
+- **Aggregation `source.where` on reserved fields now filters** instead of silently matching
+  nothing: the matcher resolves fields through the same resolver `groupBy` uses (top-level
+  standard fields + custom metadata), so `where: { subtype: 'note' }` means what it says.
+- **`removeMany()` refuses empty/invalid selectors loudly.** A bare array passed positionally
+  (`removeMany([id])` instead of `removeMany({ ids: [id] })`), an empty params object, or
+  `ids: []` used to resolve successfully having deleted nothing. All three now throw.
+- **`find()` accepts both where-key spellings.** Metadata is flattened at index time
+  (`metadata.entry.title` indexes as `entry.title`); a `metadata.`-prefixed where key now
+  falls back to its flattened spelling when the prefixed one isn't indexed — the
+  "unindexed field(s), returning []" confusion for storage-shaped spellings is gone. (A
+  literal nested custom key named `metadata` still wins when indexed as spelled.)
+
+## v8.8.1 — 2026-07-18 (flush no longer walks the whole generation history + the import dedup off-switch is now honest)
+
+### The flush-storm fix (production incident, reported by a long-running deployment)
+
+Under the default adaptive retention, **every `flush()` re-walked the entire committed
+generation history** to compute total history bytes for the budget check — O(all
+generations) with disk re-reads past the 4,096-entry delta-cache bound. On a brain with
+70,000+ accumulated generations that turned every write into a full-tail scan (60-100s
+writes), even though the budget (free-RAM-based) never tripped and nothing was ever
+reclaimed. Fixed:
+
+- `historyBytes()` now maintains a **running total**: seeded by one walk on first use,
+  then updated incrementally at every commit and reclaim — the adaptive retention check
+  on every flush is O(1). Invariant regression-pinned (running total ≡ fresh walk through
+  both commit paths and compaction).
+- New **`brain.historyStats()`** (read-only, exported `HistoryStats`): generation count,
+  total on-disk bytes, generation/timestamp range, compaction horizon, retention mode,
+  and the effective adaptive budget — the one-call fleet-audit for sizing retention
+  exposure per brain.
+- Interim guidance for keep-everything deployments already affected: `retention: 'all'`
+  skips the adaptive accounting entirely (and is the correct policy if you never want
+  history reclaimed). The accumulated files are harmless at rest; this release removes
+  the per-write cost of their existence.
+
+### The import dedup off-switch (lifecycle honesty)
+
+The post-import background deduplication pass (a merge-DELETE writer that runs ~5 minutes
+after an import, merging entities judged duplicates by id / name / vector similarity) had
+three lifecycle defects, all fixed:
+
+- **`enableDeduplication: false` now actually disables it.** The background pass was
+  scheduled unconditionally — an import that explicitly opted out could still have
+  entities auto-removed 5 minutes later. The flag now gates BOTH the inline merge and
+  the background pass (regression-pinned).
+- **One deduplicator per brain, owned by the brain.** Each `import()` call constructed its
+  own coordinator + deduplicator, so the "debounced" timer never actually debounced across
+  imports (N imports = N delete timers). The brain now owns a single instance — the
+  debounce genuinely spans imports — and `close()` cancels pending work, so a delete pass
+  can never fire against a closed brain.
+- **The 5-minute timer is unref'd** — a pending pass no longer holds the process open
+  (the exit-hang class; this timer had escaped the earlier sweep).
+
+Retention note for keep-everything deployments: with `enableDeduplication: false` on
+import calls and `retention: 'all'` in config, no engine path removes records
+automatically.
+
+## v8.8.0 — 2026-07-17 (OS-limit detection for pool-scale deployments)
+
+Small minor: brains now detect the two OS limits that bite at pool scale and warn **before**
+the incident instead of during it.
+
+- At open (once per process, Linux-only, measurement-only), Brainy reads `RLIMIT_NOFILE`
+  (soft/hard, from `/proc/self/limits`) and `vm.max_map_count`, and warns loudly when either
+  sits below the pool-scale floors (soft NOFILE < 65 536; max_map_count < 262 144) — with the
+  exact raise commands (`ulimit -n` / `LimitNOFILE=` / `sysctl vm.max_map_count`). On stock
+  defaults the failure otherwise arrives as `EMFILE` or a failed mmap deep inside an index
+  open, long after the real cause stopped being visible. An unreadable limit produces **no**
+  warning — no measurement, no claim (non-Linux platforms stay silent).
+- Exported for ops doors: `checkOsLimits()` returns the full `OsLimitsReport`
+  (values + warnings) programmatically, with the floors exported as constants.
+
+## v8.7.1 — 2026-07-17 (writer-lock acquisition is race-proof + machine-readable through init)
+
+Two hardenings of the multi-process writer lock (the `locks/_writer.lock` lease that makes a
+second writer on the same brain directory fail loudly):
+
+- **Lock acquisition claims atomically.** The acquire path used read-then-write, leaving a
+  window where two processes racing an *absent* lock could both "succeed" — and the loser
+  kept running unlocked, silently. The claim is now an atomic create-exclusive write
+  (`O_EXCL`): exactly one racer wins; the loser re-evaluates and either fails loudly with
+  the winner's details or performs a verified stale-takeover. Bounded retries; contention
+  beyond them fails loudly rather than degrading into a lockless open.
+- **`BRAINY_WRITER_LOCKED` survives `init()`.** The conflict error documents a
+  machine-readable contract (`err.code`, `err.lockInfo` with the holder's pid/host/
+  heartbeat), but init's error wrapping silently stripped both, leaving consumers a message
+  to regex against. The error now passes through unwrapped.
+
+Measured while verifying (for operators sizing audits): `brain.auditGraph()` at a
+production-consumer scale of ~2,600 relationships / 800 entities costs ~0.1 s warm and
+~0.5 s cold, with exact scar counting across reopen.
+
+## v8.7.0 — 2026-07-17 (bulk-transact ergonomics: scaled budgets + timeout telemetry)
+
+The bulk-import ergonomics release, from a consumer's measured production incident (a serial
+import on network-attached storage at ~2 s/op met a flat 30 s transaction budget):
+
+- **The transact apply budget now scales with the batch** — `max(30 s, opCount × 2 s)` — or
+  is exactly what you pass as the new `TransactOptions.timeoutMs`. A flat 30 s cap silently
+  limited honest bulk work to ~15 operations on slow disks while looking generous for small
+  batches. Internal batch paths (e.g. `removeMany` chunks) get the same scaling.
+- **`TransactionTimeoutError` is a diagnosis, not just a failure**: it now reports the
+  operation it stopped at as `i/N` with the operation's name, elapsed vs budgeted time, and
+  states the batch rolled back atomically and is retryable. Its `context` carries the same
+  fields programmatically.
+- **The transact envelope is documented** — batch sizing, budget math, chunking with
+  `ifAbsent` idempotency, and the precompute pattern (`embedBatch` + per-op `vector`) that
+  keeps model inference out of the commit path. Guide: `docs/guides/optimistic-concurrency.md`.
+- Note: `brain.embed()` / `brain.embedBatch()` (the precompute APIs) already ship — public,
+  with native-provider passthrough, verified end-to-end (batch and single paths produce
+  bit-identical vectors; a vector-supplied `add` is fully searchable). Honest measurement:
+  on the default WASM engine, batch throughput ≈ sequential (~160 ms/text) — the precompute
+  win is keeping inference out of the budgeted commit path, not raw embedding speed.
+
+## v8.6.0 — 2026-07-17 (brain.auditGraph — the graph-truth verification instrument)
+
+A minor release adding one new public API, from the fleet's graph-trust program: a read-only
+audit that **proves whether relationship reads return stored truth** on a given brain.
+
+- **`brain.auditGraph(options?)`** walks every canonical relationship record, queries the same
+  read path applications use (`related()` / VFS `readdir`) with all visibility tiers included,
+  and classifies every discrepancy into its failure family: `missingFromReads` (records the
+  read path omits — a stale adjacency index), `danglingEndpoints` (relationships whose endpoint
+  entity no longer exists — the historical partial-delete scar class), and `readOnlyVerbIds`
+  (read-path edges with no stored record — ghosts). Design-hidden internal/system edges are
+  counted separately so intentional hiding is never misclassified as loss. Counts are exact;
+  example lists cap at `maxExamples` with an explicit `truncatedExamples` flag; the result is
+  narrated loudly on incoherence. Mutates nothing — safe on a live brain.
+- The operational pairing: audit → if incoherent, `repairIndex()` → audit again. A `coherent`
+  report after the repair is the verified all-clear. Run it after any engine upgrade, restore,
+  or migration. Guide: `docs/guides/inspection.md`.
+- Also: `getNounIds` pagination now refuses an undecodable resume cursor loudly (the same
+  contract `getNouns`/`getVerbs` gained in 8.5.2 — the third and final walk brought under it).
+- Types exported: `GraphAuditReport`, `GraphAuditDiscrepancy`.
+
+## v8.5.2 — 2026-07-17 (aggregation backfill: exception-safe, generation-verified, and loud)
+
+Hardening patch from a migration incident (a byte-copied store on new hardware; the service
+entered a silent full-CPU loop at boot). Four changes, all in the aggregation engine's
+backfill/adoption path:
+
+- **Backfill walks are exception-safe and non-destructive.** A rescan now builds into a
+  staging map and swaps in atomically on completion; a mid-walk failure drops the staging map,
+  keeps the previous live state serving, and surfaces the storage error to the failing query.
+  Previously the walk wiped live state *before* a scan that could throw, never cleared the
+  pending flag on failure, and re-ran a full walk on every subsequent query — a silent
+  wipe/walk/throw loop at the caller's retry rate.
+- **Failed walks are latched.** After a walk fails, retries within a 30-second cooldown rethrow
+  the recorded error instantly instead of re-walking — a tight caller-side retry loop now costs
+  one loud error per query, never a full store walk per query.
+- **Adoption is generation-verified.** Persisted aggregation state is stamped with the store's
+  committed generation at flush; reopen adoption requires the stamp to equal the current
+  watermark. Stale state (unclean shutdown) or over-counting state (a fact-log truncation on a
+  copied store pulled the watermark back) triggers exactly one loud rescan — never a silent
+  adopt. Pre-8.5.2 state on generation-aware stores rescans once after upgrade, then is stamped.
+- **The path narrates.** Adoption decisions, no-adoptable-state outcomes, walk start/finish
+  (entity count + duration), and walk failures all log by default; a non-advancing storage
+  pagination cursor aborts the walk loudly instead of looping forever.
+
+Plus three guards from a full audit of every loop in the open/init path:
+
+- **Invalid pagination cursors fail loudly.** A supplied-but-undecodable resume token to
+  `getNouns`/`getVerbs` used to silently restart the walk at offset 0 — to a `while(hasMore)`
+  caller that re-serves page 1 forever (an unbounded silent CPU loop). It now throws with a
+  clear message instead.
+- **The graph cold-load verb walk has a stall guard.** `hasMore=true` with a missing or
+  non-advancing cursor aborts loudly instead of re-reading the same page forever.
+- **A derived index AHEAD of the store is named at open.** Brainy already surfaced a provider
+  generation *behind* the committed watermark; the *ahead* direction (the signature of a
+  byte-copy of a live service, or a log truncation during crash recovery) now logs a loud
+  warning explaining what happened and that `brain.repairIndex()` forces a heal — instead of
+  passing unnamed into whatever the derived index does next.
+
 ## v8.5.1 — 2026-07-17 (aggregation state survives restarts + the query-cap ratchet removed)
 
 Patch release from a production incident (aggregate/count paths taking 40–90 s on an idle box

SECURITY.md

@@ -0,0 +1,36 @@
+# Security Policy
+
+## Reporting a vulnerability
+
+Email **security@soulcraft.com**. That's the one door for security reports
+across the company, and it works the same way for Brainy: every report is
+read by a human, you'll get a private receipt, and we'll work with you on
+coordinated disclosure — please don't open a public issue for anything
+that isn't already public.
+
+Include what you'd want if you were on the other end: affected version,
+how to reproduce, and what you think the impact is. If you have a patch or
+a suggested fix, send it along — it's welcome but not required.
+
+There is no bounty program today. We're saying that plainly so you know
+what to expect going in.
+
+## Response time
+
+We respond as fast as truth allows. That means: no fixed SLA, no promise of
+a reply within a specific number of hours — but a real report from a real
+person gets read promptly and taken seriously. If you haven't heard anything
+in a reasonable stretch, a follow-up email is completely fine.
+
+## Supported versions
+
+The latest `8.x` minor release line receives security fixes. If you're
+running an older major version, please upgrade before reporting — we can't
+commit to backporting fixes to unsupported lines.
+
+## Scope
+
+This policy covers the `@soulcraft/brainy` package itself — the code in
+this repository. If you're evaluating a deployment that also uses
+`@soulcraft/cor`, report issues in that package the same way, to the same
+address; we'll route internally.

docs/guides/import-anything.md

@@ -300,7 +300,10 @@ await brain.import(data, {
  // Deduplication
  enableDeduplication: true, // Check for duplicate entities (default: true)
  deduplicationThreshold: 0.85, // Similarity threshold for duplicates (0-1, default: 0.85)
- // Note: Auto-disabled for imports >100 entities
+ // Notes: false disables BOTH the inline merge and the background pass that
+ // runs ~5 min after the last import (merged duplicates are deleted).
+ // The inline pass auto-disables for imports >100 entities (O(n²) cost);
+ // the background pass still covers those unless the flag is false.
 
  // Performance
  chunkSize: 100, // Batch size for processing (default: varies by operation)

docs/guides/import-quick-reference.md

@@ -86,11 +86,22 @@ await brain.import(file, {
 
 ```typescript
 await brain.import(file, {
-  enableDeduplication: true,         // Check for duplicates (default: false)
+  enableDeduplication: true,         // Check for duplicates (default: true)
   deduplicationThreshold: 0.85       // Similarity threshold (default: 0.85)
 })
 ```
 
+Deduplication merges entities judged duplicates — the non-primary records are
+**deleted**. Set `enableDeduplication: false` to disable it entirely: the flag
+gates both the inline merge during import and the background pass that runs
+about 5 minutes after the last import.
+
+```typescript
+await brain.import(file, {
+  enableDeduplication: false         // No merging, inline or background
+})
+```
+
 ### Import Tracking
 
 Track and organize imports by project:

docs/guides/inspection.md

@@ -166,6 +166,34 @@ brainy inspect diff /data/brain-prod /data/brain-staging
 Sample-based — for a full diff, dump both with `inspect dump` and compare
 the JSONL.
 
+## Auditing graph-read truth
+
+`brain.auditGraph()` (8.6.0+) proves — or disproves — that relationship reads
+return canonical truth on a given brain, without mutating anything. It walks
+every stored relationship record, asks the same read path your application
+uses (`related()`, VFS `readdir`) with every visibility tier included, and
+classifies every discrepancy:
+
+```typescript
+const report = await brain.auditGraph()
+
+report.coherent              // true = related()/readdir can be trusted on this brain
+report.missingFromReadsCount // records the read path omits — stale index
+report.danglingEndpointsCount // relationships whose endpoint entity is gone
+report.readOnlyCount         // read-path edges with NO stored record — ghosts
+report.visibilityHiddenCount // internal/system edges hidden by design (not a fault)
+```
+
+Counts are always exact; the example lists (`missingFromReads`,
+`danglingEndpoints`, `readOnlyVerbIds`) are capped at `maxExamples`
+(default 100) and `truncatedExamples` says so when they are.
+
+Run it after any engine upgrade, restore, or migration. If it reports
+discrepancies, run `brain.repairIndex()` and audit again — a `coherent`
+report after the repair is the verified statement that the heal worked.
+Cost: one relationship-record walk plus one indexed read per distinct
+source entity — safe on a live brain.
+
 ## Repairing a corrupted store
 
 If invariants fail and you suspect index corruption, `inspect repair`

docs/guides/optimistic-concurrency.md

@@ -180,3 +180,35 @@ Brainy 8.0 has exactly two write-coordination counters, at two granularities:
 They compose: a `transact()` batch can carry per-entity `ifRev` checks *and* a whole-store `ifAtGeneration`; any failed check rejects the entire batch before anything is staged. Generations also power snapshots and time travel (`brain.now()`, `brain.asOf()`, `db.persist()`) — see the [consistency model](../concepts/consistency-model.md) and [Snapshots & Time Travel](./snapshots-and-time-travel.md).
 
 A snapshot or historical view captures each entity *including* its `_rev` at that moment, so reading the past and writing back with `ifRev` against the live state works exactly as you'd hope: the write fails if the entity moved since the state you copied from.
+
+## The transact envelope: batch size, budget, and bulk imports
+
+`transact()` applies its batch atomically under one commit — which means the whole batch
+shares one **apply budget**. Since 8.7.0 the budget scales with the batch:
+`max(30 s, opCount × 2 s)`, or exactly what you pass as `timeoutMs`. A tripped budget rolls
+the entire batch back (nothing partial survives) and throws a retryable
+`TransactionTimeoutError` that names the operation it stopped at, the batch size, and the
+elapsed vs budgeted time — a diagnosis, not just a failure:
+
+```
+Transaction timed out at operation 41/120 ('add') — 246012ms elapsed, budget 240000ms.
+The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.
+```
+
+Practical envelope guidance for bulk work:
+
+1. **Precompute embeddings outside the commit path.** Embedding inside `transact()` spends
+   the budget on model inference. Use `brain.embedBatch(texts)` and pass each vector via
+   the op's `vector` field — the commit then pays only storage costs, and a retried batch
+   never re-pays inference. (The win is *where* the inference happens, not raw embedding
+   throughput: on the default WASM engine, batch and sequential embedding measure
+   comparably, ~160 ms/text; native embedding providers may batch faster.)
+2. **Chunk very large imports** into batches of a few hundred ops with one `transact()`
+   each. You lose whole-import atomicity but keep per-chunk atomicity, bounded memory, and
+   resumability — pair with `ifAbsent` upserts so a retried chunk is idempotent.
+3. **Slow disks change the math, not the contract.** On network-attached storage a single
+   op can cost ~2 s (canonical write + fsync + index maintenance). The scaled default
+   absorbs that; pass an explicit `timeoutMs` only when you know better than the scale.
+4. **`addMany`/`relateMany` are the convenience tier** — they chunk and batch-embed for
+   you, with per-item error reporting instead of batch atomicity. Choose by what you need:
+   atomic-all-or-nothing → `transact()`; resilient bulk load → `addMany`.

docs/guides/snapshots-and-time-travel.md

@@ -344,8 +344,12 @@ For per-entity write coordination (rather than whole-store history), the
 ## Keeping history bounded
 
 Under Model-B every write is a generation, so history can grow quickly —
-Brainy auto-compacts on every `flush()`/`close()` under the **`retention`**
+Brainy auto-compacts at `close()` (time-bounded per pass) under the
-knob (configured on the constructor):
+**`retention`** knob (configured on the constructor). Since 8.9.0, `flush()`
+never compacts: flushing is durability work and costs only what the current
+window's writes cost, regardless of history backlog. A long-lived writer that
+never closes keeps its history until its next explicit `compactHistory()` —
+schedule one in your maintenance window if you run bounded retention:
 
 ```typescript
 // Zero-config: ADAPTIVE — keep as much history as free disk/RAM allows,
@@ -359,10 +363,13 @@ new Brainy({ retention: 'all' })
 new Brainy({ retention: { maxGenerations: 1000, maxAge: 7 * 86_400_000, maxBytes: 512 * 1024 ** 2 } })
 ```
 
-Reclaim manually at any time (the same caps):
+Reclaim manually at any time (the same caps, plus an optional per-pass time
+budget for maintenance windows — an early stop is a consistent prefix and the
+next pass resumes):
 
 ```typescript
 await brain.compactHistory({ maxGenerations: 100, maxAge: 7 * 24 * 60 * 60 * 1000 })
+await brain.compactHistory({ maxBytes: 512 * 1024 ** 2, timeBudgetMs: 10_000 })
 ```
 
 Compaction never breaks a pinned read — record-sets are reclaimed only when

docs/performance-envelopes.md

@@ -0,0 +1,83 @@
+---
+title: Performance Envelopes
+slug: guides/performance-envelopes
+public: true
+category: guides
+template: guide
+order: 40
+description: Measured per-operation latency envelopes at stated scales — what to expect, on what hardware, and exactly how each number was produced.
+next:
+  - guides/find-limits
+---
+
+# Performance Envelopes
+
+Every number on this page is **measured, never projected** — produced by the script
+cited at the bottom, against the built package (the artifact you install), on the stated
+hardware. Each entry says what was measured, at what scale, on which storage backend.
+When a release touches a measured path, that operation is re-measured and this page
+updates in the same release.
+
+Two scopes to keep straight:
+
+- **These envelopes are the pure-JS engine** (no native accelerator registered) on
+  filesystem storage. This is the floor every deployment gets from `npm install` alone.
+- **Accelerated deployments** (the optional native provider) publish their own numbers —
+  this page never claims them.
+
+## Read operations
+
+Reads are where the architecture pays off: after the write path has done its indexing
+work, queries answer from purpose-built indexes without scanning.
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `get(id)` (warm) | p50 < 0.1ms | p50 < 0.1ms | served from cache/metadata index |
+| `find` (metadata: indexed equality + range, limit 100) | p50 1.0ms · p95 1.8ms | p50 7.0ms · p95 8.9ms | column-store bitmap paths |
+| `related(id)` (per-node adjacency) | p50 < 0.1ms · p95 0.2ms | p50 < 0.1ms | LSM adjacency index — O(degree), scale-independent |
+| `find` (semantic: embed + HNSW, 1k docs) | p50 178ms · p95 393ms | — | dominated by WASM query embedding (measured on a machine under concurrent load — treat the p95 as an upper bound); the vector search itself is single-digit ms |
+
+## Write operations
+
+Under Model-B **every write is its own durable generation** — a single-op `add` pays
+serialization, before-image staging, and fsync before it acks. That durability is priced
+into the write path visibly, by design:
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `add` (single-op) | p50 167ms · p95 171ms | p50 165ms · p95 172ms | full durable generation per write — flat across scale |
+| `addMany` (bulk) | ~163ms/entity | ~187ms/entity | **currently per-item commits** — see the honest note below |
+| `relateMany` | ~0.8ms/edge | ~0.9ms/edge | edges batch efficiently today |
+| `flush` (steady-state, 1 pending write) | p50 8ms · p95 10ms | p50 45ms · p95 52ms | durability-only since 8.9.0 — cost no longer depends on history backlog or retention mode |
+
+**The honest note on bulk writes:** `addMany` today commits each item as its own
+generation (the same durability as single-op `add`, serialized by the single-writer
+lock), so bulk-load cost is N × single-op cost. Batched chunk commits (one generation
+and one fsync window per chunk, as `removeMany` already does) are designed into the
+unified-commit work on the current roadmap. Until that ships, size bulk imports
+accordingly — 10k entities is minutes, not seconds, on filesystem storage.
+
+## Open / close
+
+| Operation | 1,000 entities | 10,000 entities | Notes |
+|---|---|---|---|
+| `open` (empty store) | ~560ms | ~190ms | includes embedder initialization |
+| `open` (warm, populated, clean shutdown) | 763ms | 4.9s | pure-JS vector index load dominates and grows with entity count; the native accelerator exists precisely to remove this |
+| `close` | bounded | bounded | auto-compaction pass is time-bounded (~5s max) since 8.9.0 |
+
+A store that was NOT cleanly closed pays index rebuilds on top of the warm-open
+number (tens of seconds at 10k) — clean shutdown is worth engineering for.
+
+## How these were produced
+
+- **Hardware**: Intel Core i9-14900HX (32 threads), 62GB RAM, NVMe, Linux, Node v22.
+- **Backend**: `storage: { type: 'filesystem' }`, pure JS (no native providers).
+- **Embeddings**: deterministic stub for non-semantic ops (isolates engine cost);
+  the real WASM embedder for the semantic row (that's what you'll run).
+- **Method**: p50/p95 over 50–200 samples per op against the built `dist/`;
+  the measuring script ships in the repo history and re-runs per release.
+
+Numbers on different hardware will differ; the *shape* (sub-2ms indexed reads,
+~160ms embedding-bound semantic queries, durability-priced writes) is the envelope
+you should hold your deployment against. If your measurements diverge from these
+shapes by an order of magnitude, something is wrong — file it.

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.5.1",
+  "version": "8.10.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "@soulcraft/brainy",
-      "version": "8.5.1",
+      "version": "8.10.0",
       "license": "MIT",
       "dependencies": {
         "@msgpack/msgpack": "^3.1.2",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "8.5.1",
+  "version": "8.10.0",
   "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
   "main": "dist/index.js",
   "module": "dist/index.js",
@@ -128,13 +128,13 @@
   "publishConfig": {
     "access": "public"
   },
-  "homepage": "https://github.com/soulcraftlabs/brainy",
+  "homepage": "https://source.soulcraft.com/soulcraft/brainy",
   "bugs": {
-    "url": "https://github.com/soulcraftlabs/brainy/issues"
+    "url": "https://source.soulcraft.com/soulcraft/brainy/issues"
   },
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/soulcraftlabs/brainy.git"
+    "url": "git+https://source.soulcraft.com/soulcraft/brainy.git"
   },
   "files": [
     "dist/**/*.js",

scripts/push-docs.js

@@ -0,0 +1,116 @@
+#!/usr/bin/env node
+/**
+ * @module scripts/push-docs
+ * @description Push this repo's PUBLIC docs to the soulcraft.com docs ingest
+ * door after an npm publish (VENUE-DOCS-RELEASE-PUSH — retires the old
+ * build-time docs sync).
+ *
+ * Contract (mirrors the reference implementation on the serving side):
+ *   POST {base}/api/docs/ingest
+ *   headers: x-service-secret: $DOCS_INGEST_SECRET, Content-Type: application/json
+ *   body: { docs: [{ slug, title, markdown, nav: { order, section } }] }
+ *   batches of 10, idempotent per slug.
+ *
+ * A doc is public iff its frontmatter has `public: true` AND a `slug`. The
+ * frontmatter is stripped; `category` → nav.section, `order` → nav.order.
+ *
+ * Deliberately NOT pushed: the combined /docs landing index. It spans BOTH
+ * engine corpora (this repo's and the native accelerator's), so a per-repo
+ * push would clobber the union — the index is authored on the serving side.
+ *
+ * Env: DOCS_INGEST_SECRET (required), DOCS_INGEST_BASE (default
+ * https://soulcraft.com). Exits 0 with a LOUD warning when the secret is
+ * absent (the npm publish has already happened; the serving side runs its
+ * interim sync on request) and exits 1 when a push actually fails — the docs
+ * site would silently trail npm otherwise, and that must be visible.
+ */
+import * as fs from 'node:fs'
+import * as path from 'node:path'
+
+const BASE = (process.env.DOCS_INGEST_BASE || 'https://soulcraft.com').replace(/\/+$/, '')
+const SECRET = process.env.DOCS_INGEST_SECRET
+const DOCS_DIR = path.join(path.dirname(new URL(import.meta.url).pathname), '..', 'docs')
+const BATCH = 10
+
+if (!SECRET) {
+  console.warn(
+    '⚠️  DOCS PUSH SKIPPED: DOCS_INGEST_SECRET is not set.\n' +
+      '    soulcraft.com/docs now TRAILS this npm release until docs are pushed.\n' +
+      '    Either export DOCS_INGEST_SECRET and re-run `node scripts/push-docs.js`,\n' +
+      '    or ping venue on VENUE-DOCS-RELEASE-PUSH for the interim sync.'
+  )
+  process.exit(0)
+}
+
+/** Minimal frontmatter split — returns [meta, body] or [null, raw]. */
+function parseFrontmatter(raw) {
+  const m = raw.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/)
+  if (!m) return [null, raw]
+  const meta = {}
+  for (const line of m[1].split('\n')) {
+    const kv = line.match(/^(\w[\w-]*):\s*(.*)$/)
+    if (kv) meta[kv[1]] = kv[2].trim().replace(/^["']|["']$/g, '')
+  }
+  return [meta, m[2]]
+}
+
+const docs = []
+;(function walk(dir) {
+  for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+    const full = path.join(dir, entry.name)
+    if (entry.isDirectory()) walk(full)
+    else if (entry.name.endsWith('.md')) {
+      const [meta, body] = parseFrontmatter(fs.readFileSync(full, 'utf-8'))
+      if (!meta || meta.public !== 'true' || !meta.slug) continue
+      docs.push({
+        slug: meta.slug,
+        title: meta.title || meta.slug,
+        markdown: body.trim(),
+        nav: {
+          order: Number.parseInt(meta.order || '99', 10) || 99,
+          section: meta.category || 'guides'
+        }
+      })
+    }
+  }
+})(DOCS_DIR)
+
+if (docs.length === 0) {
+  console.error('❌ DOCS PUSH FAILED: zero public docs collected — refusing to push an empty corpus.')
+  process.exit(1)
+}
+docs.sort((a, b) => a.slug.localeCompare(b.slug))
+console.log(`Pushing ${docs.length} public docs to ${BASE}/api/docs/ingest …`)
+
+let failed = false
+for (let i = 0; i < docs.length; i += BATCH) {
+  const batch = docs.slice(i, i + BATCH)
+  try {
+    const res = await fetch(`${BASE}/api/docs/ingest`, {
+      method: 'POST',
+      headers: {
+        'x-service-secret': SECRET,
+        'Content-Type': 'application/json',
+        'User-Agent': 'brainy-docs-push/1.0'
+      },
+      body: JSON.stringify({ docs: batch }),
+      signal: AbortSignal.timeout(120_000)
+    })
+    if (!res.ok) {
+      throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`)
+    }
+    console.log(`  batch ${i / BATCH + 1}: ${batch.map((d) => d.slug).join(', ')} → ok`)
+  } catch (err) {
+    failed = true
+    console.error(`  batch ${i / BATCH + 1} FAILED: ${err instanceof Error ? err.message : err}`)
+  }
+}
+
+if (failed) {
+  console.error(
+    '❌ DOCS PUSH INCOMPLETE — soulcraft.com/docs may trail npm. ' +
+      'Re-run `node scripts/push-docs.js` or ping venue on VENUE-DOCS-RELEASE-PUSH.'
+  )
+  process.exit(1)
+}
+console.log('✅ Docs pushed.')

scripts/release.sh

@@ -142,7 +142,7 @@ else
 fi
 
 # Create new changelog entry
-CHANGELOG_ENTRY="### [${NEW_VERSION}](https://github.com/soulcraftlabs/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d))
+CHANGELOG_ENTRY="### [${NEW_VERSION}](https://source.soulcraft.com/soulcraft/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d))
 
 ${COMMITS}
 "
@@ -175,30 +175,75 @@ echo -e "${BLUE}7️⃣  Creating git tag v${NEW_VERSION}...${NC}"
 git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}"
 echo -e "${GREEN}✅ Tag created${NC}\n"
 
-# Step 9: Push to GitHub
+# Step 9: Push to origin — the forge is the one home (ruled 2026-07-23; the
-echo -e "${BLUE}8️⃣  Pushing to GitHub...${NC}"
+# old public GitHub repo is archived history, no longer part of any release).
+echo -e "${BLUE}8️⃣  Pushing to origin...${NC}"
 git push --follow-tags origin "$CURRENT_BRANCH"
-echo -e "${GREEN}✅ Pushed to GitHub${NC}\n"
+echo -e "${GREEN}✅ Pushed to origin${NC}\n"
 
-# Step 10: Publish to npm
+# Step 10: Publish — forge FIRST (home), npmjs second (the world's storefront).
-echo -e "${BLUE}9️⃣  Publishing to npm (dist-tag: ${NPM_TAG})...${NC}"
+# The fleet-wide ~/.npmrc maps the @soulcraft scope to the forge registry, and
-npm publish --tag "$NPM_TAG"
+# a scope mapping BEATS `--registry` on the command line — so each publish
-# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
+# names its registry via the scope override explicitly. Nothing implicit.
-npm access get status @soulcraft/brainy || true
+FORGE_NPM_REG="https://source.soulcraft.com/api/packages/soulcraft/npm/"
-echo -e "${GREEN}✅ Published to npm${NC}\n"
+FORGE_NPM_TOKEN_FILE="$HOME/.config/soulcraft/npm-publish-brainy.token"
-
+echo -e "${BLUE}9️⃣  Publishing to the forge registry (home)...${NC}"
-# Step 11: Create GitHub release
+if [ -f "$FORGE_NPM_TOKEN_FILE" ]; then
-echo -e "${BLUE}🔟 Creating GitHub release...${NC}"
+  TMPRC="$(mktemp)"
-if [ "$PRERELEASE" = true ]; then
+  chmod 600 "$TMPRC"
-  gh release create "v${NEW_VERSION}" --generate-notes --prerelease
+  {
+    echo "@soulcraft:registry=${FORGE_NPM_REG}"
+    echo "//source.soulcraft.com/api/packages/soulcraft/npm/:_authToken=$(cat "$FORGE_NPM_TOKEN_FILE")"
+  } > "$TMPRC"
+  if npm publish --tag "$NPM_TAG" --userconfig "$TMPRC"; then
+    echo -e "${GREEN}✅ Published to the forge${NC}\n"
   else
-  gh release create "v${NEW_VERSION}" --generate-notes
+    rm -f "$TMPRC"
+    echo -e "${RED}❌ Forge publish FAILED — aborting before npmjs so the pair never diverges. Fix and re-run.${NC}"
+    exit 1
+  fi
+  rm -f "$TMPRC"
+else
+  echo -e "${RED}❌ Forge publish token missing (${FORGE_NPM_TOKEN_FILE}) — aborting. The forge is home; publish it first or restage the token.${NC}"
+  exit 1
+fi
+
+echo -e "${BLUE}9️⃣½  Publishing to npmjs (storefront, dist-tag: ${NPM_TAG})...${NC}"
+npm publish --tag "$NPM_TAG" "--@soulcraft:registry=https://registry.npmjs.org/"
+# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
+npm access get status @soulcraft/brainy "--@soulcraft:registry=https://registry.npmjs.org/" || true
+echo -e "${GREEN}✅ Published to npmjs${NC}\n"
+
+# Step 11: Release object on the forge (presentational — the tag, CHANGELOG,
+# and RELEASES.md are the record; this just gives the forge UI a release page).
+echo -e "${BLUE}🔟 Creating forge release...${NC}"
+if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then
+  if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraft/brainy/releases" \
+    -H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \
+    -d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then
+    echo -e "${GREEN}✅ Forge release created${NC}\n"
+  else
+    echo -e "${RED}⚠️  Forge release API call failed — tag + CHANGELOG remain the record; create the release page via the forge UI if wanted${NC}\n"
+  fi
+else
+  echo -e "${RED}⚠️  FORGEJO_RELEASE_TOKEN unset — no release page created; tag + CHANGELOG remain the record${NC}\n"
+fi
+
+# Step 12: Push public docs to the soulcraft.com docs ingest door
+# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
+# DOCS_INGEST_SECRET is unset; fails loudly (without undoing the publish —
+# that already happened) when a push errors, so the docs site never
+# silently trails npm.
+echo -e "${BLUE}1️⃣2️⃣ Pushing public docs to soulcraft.com/docs...${NC}"
+if node scripts/push-docs.js; then
+  echo -e "${GREEN}✅ Docs push step done${NC}\n"
+else
+  echo -e "${RED}❌ Docs push FAILED — soulcraft.com/docs trails npm until re-run or interim sync${NC}\n"
 fi
-echo -e "${GREEN}✅ GitHub release created${NC}\n"
 
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
 echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}"
 echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
 echo ""
 echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}"
-echo -e "🐙 GitHub: ${BLUE}https://github.com/soulcraftlabs/brainy/releases/tag/v${NEW_VERSION}${NC}"
+echo -e "🏠 Forge: ${BLUE}https://source.soulcraft.com/soulcraft/brainy/releases/tag/v${NEW_VERSION}${NC}"

src/aggregation/AggregationIndex.ts

@@ -29,6 +29,7 @@ import { matchesMetadataFilter } from '../utils/metadataFilter.js'
 import { compareCodePoints } from '../utils/collation.js'
 import { bucketTimestamp } from './timeWindows.js'
 import { NounType } from '../types/graphTypes.js'
+import { prodLog } from '../utils/logger.js'
 
 /** Persistence key for aggregate definitions */
 const DEFINITIONS_KEY = '__aggregation_definitions__'
@@ -87,10 +88,18 @@ function matchesSource(entity: Record<string, unknown>, source: AggregateDefinit
     if (entity.service !== source.service) return false
   }
 
-  // Metadata where filter — match against the entity's metadata sub-object
+  // Where filter — resolve each filtered field through resolveEntityField,
+  // the SAME single source of truth groupBy uses (top-level standard fields
+  // + custom metadata). Matching only the metadata sub-object made
+  // where:{subtype}/{visibility}/… a silent no-op: reserved fields never
+  // live in the custom bag, so those filters could never match anything.
   if (source.where && Object.keys(source.where).length > 0) {
-    const metadata = (entity.metadata ?? entity) as Record<string, unknown>
+    const e = entity as unknown as HNSWNounWithMetadata
-    if (!matchesMetadataFilter(metadata, source.where)) return false
+    const resolved: Record<string, unknown> = {}
+    for (const key of Object.keys(source.where)) {
+      resolved[key] = resolveEntityField(e, key)
+    }
+    if (!matchesMetadataFilter(resolved, source.where)) return false
   }
 
   return true
@@ -343,6 +352,14 @@ export class AggregationIndex {
    */
   private pendingAdopt = new Set<string>()
 
+  /**
+   * In-flight rescan targets. While a name has a staging map, ALL
+   * contributions (the walk's and concurrent write hooks') land there instead
+   * of the live map; the live map keeps serving until {@link finishBackfill}
+   * swaps the staging map in atomically.
+   */
+  private backfillStaging = new Map<string, Map<string, AggregateGroupState>>()
+
   constructor(storage: StorageAdapter, nativeProvider?: AggregationProvider) {
     this.storage = storage
     this.nativeProvider = nativeProvider
@@ -391,10 +408,37 @@ export class AggregationIndex {
    * adopt (or init never ran / failed) — it must backfill.
    */
   private resolvePendingAdoptToBackfill(): void {
+    if (this.pendingAdopt.size > 0) {
+      prodLog.info(
+        `[Aggregation] no adoptable persisted state for: ${Array.from(this.pendingAdopt).join(', ')} — flagged for backfill`
+      )
+    }
     for (const name of this.pendingAdopt) this.needsBackfill.add(name)
     this.pendingAdopt.clear()
   }
 
+  /**
+   * May this persisted state be ADOPTED? When the store exposes its committed
+   * watermark, the state's `sourceGeneration` must EQUAL it: behind means
+   * later writes are missing from the state (unclean shutdown); ahead means
+   * it counts writes that no longer exist (e.g. a fact-log truncation on a
+   * copied store pulled the watermark back). Either way: one exact rescan,
+   * said out loud — never a silent adopt. Stores without the capability (and
+   * pre-stamp state on them) fall back to hash-only adoption.
+   */
+  private stateGenerationAdoptable(name: string, stateData: unknown): boolean {
+    const committed = this.storage.committedGeneration?.() ?? null
+    if (committed === null) return true
+    const raw = (stateData as Record<string, unknown>).sourceGeneration
+    const stamped = typeof raw === 'number' ? raw : null
+    if (stamped === committed) return true
+    prodLog.warn(
+      `[Aggregation] '${name}': persisted state is at generation ${stamped ?? 'unstamped'} ` +
+        `but the store's committed generation is ${committed} — rescanning instead of adopting`
+    )
+    return false
+  }
+
   private async loadPersisted(): Promise<void> {
     // Load persisted definitions
     const savedDefs = await this.storage.getMetadata(DEFINITIONS_KEY)
@@ -413,7 +457,11 @@ export class AggregationIndex {
           const appHash = this.definitionHashes.get(def.name) || ''
           if (appHash === savedHash && this.pendingAdopt.has(def.name)) {
             const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
-            if (stateData && stateData.groups) {
+            if (
+              stateData &&
+              stateData.groups &&
+              this.stateGenerationAdoptable(def.name, stateData)
+            ) {
               const groupMap = new Map<string, AggregateGroupState>()
               for (const group of stateData.groups as AggregateGroupState[]) {
                 groupMap.set(serializeGroupKey(group.groupKey), group)
@@ -421,6 +469,9 @@ export class AggregationIndex {
               this.states.set(def.name, groupMap)
               this.pendingAdopt.delete(def.name)
               this.needsBackfill.delete(def.name)
+              prodLog.info(
+                `[Aggregation] '${def.name}': adopted persisted state (${groupMap.size} groups) — no rescan`
+              )
             }
             // No/invalid persisted state: stays in pendingAdopt and resolves
             // to backfill when init settles.
@@ -434,7 +485,12 @@ export class AggregationIndex {
         const currentHash = hashDefinition(def)
 
         const stateData = await this.storage.getMetadata(`${STATE_KEY_PREFIX}${def.name}__`)
-        if (stateData && stateData.groups && savedHash === currentHash) {
+        if (
+          stateData &&
+          stateData.groups &&
+          savedHash === currentHash &&
+          this.stateGenerationAdoptable(def.name, stateData)
+        ) {
           // Definition unchanged — load state
           const groupMap = new Map<string, AggregateGroupState>()
           for (const group of stateData.groups as AggregateGroupState[]) {
@@ -443,6 +499,9 @@ export class AggregationIndex {
           }
           this.states.set(def.name, groupMap)
           this.needsBackfill.delete(def.name)
+          prodLog.info(
+            `[Aggregation] '${def.name}': restored definition + adopted persisted state (${groupMap.size} groups)`
+          )
         } else {
           // Definition changed or no saved state — start fresh and backfill from
           // existing entities (the owner drains needsBackfill on first query).
@@ -483,14 +542,22 @@ export class AggregationIndex {
     }))
     await this.storage.saveMetadata(DEFINITIONS_KEY, { definitions: defsToSave })
 
-    // Persist dirty states
+    // Persist dirty states, stamped with the committed generation they
+    // reflect. The stamp is what makes reopen-adoption verifiable: state at a
+    // different generation than the store's committed watermark is stale (an
+    // unclean shutdown after later writes) or over-counts (a fact-log
+    // truncation on a copied store pulled the watermark BACK below the
+    // stamp) — either way the answer is one exact rescan, never a silent
+    // adopt. Read the generation after collecting groups so any racing
+    // commit resolves toward rescan, not wrong-adopt.
     for (const name of this.dirty) {
       const stateMap = this.states.get(name)
       if (stateMap) {
         const groups = Array.from(stateMap.values())
+        const sourceGeneration = this.storage.committedGeneration?.() ?? null
         await this.storage.saveMetadata(
           `${STATE_KEY_PREFIX}${name}__`,
-          { groups }
+          sourceGeneration === null ? { groups } : { groups, sourceGeneration }
         )
       }
     }
@@ -609,9 +676,17 @@ export class AggregationIndex {
     return Array.from(this.needsBackfill)
   }
 
-  /** Clear an aggregate's state so a full rescan cannot double-count. */
+  /**
+   * Begin a rescan into a STAGING map. The live state is not touched — it
+   * keeps serving (possibly stale, but flagged pending) until the rescan
+   * completes and swaps in atomically. A mid-walk failure drops the staging
+   * map via {@link abortBackfill} and loses nothing: wiping live state before
+   * a scan that could throw was the destructive-before-durable defect.
+   * Contributions (walk + concurrent write hooks) land in staging while it
+   * exists, so the swapped-in result reflects writes that raced the walk.
+   */
   beginBackfill(name: string): void {
-    this.states.set(name, new Map())
+    this.backfillStaging.set(name, new Map())
     // Reset native provider state for this aggregate too, if present.
     const def = this.definitions.get(name)
     if (def && this.nativeProvider?.removeAggregate && this.nativeProvider?.defineAggregate) {
@@ -620,6 +695,15 @@ export class AggregationIndex {
     }
   }
 
+  /**
+   * Abandon an in-flight rescan after a failure: drop the staging map, keep
+   * the live state serving, leave the aggregate flagged as pending so a later
+   * attempt rescans. The failure itself must be surfaced loudly by the owner.
+   */
+  abortBackfill(name: string): void {
+    this.backfillStaging.delete(name)
+  }
+
   /** Feed one already-stored entity into a single aggregate during backfill. */
   backfillEntity(name: string, entity: Record<string, unknown>): void {
     if (isAggregateEntity(entity)) return
@@ -633,8 +717,13 @@ export class AggregationIndex {
     }
   }
 
-  /** Mark an aggregate's backfill complete; rebuilt state persists on next flush(). */
+  /** Swap the rebuilt staging state in atomically; persists on next flush(). */
   finishBackfill(name: string): void {
+    const staged = this.backfillStaging.get(name)
+    if (staged) {
+      this.states.set(name, staged)
+      this.backfillStaging.delete(name)
+    }
     this.needsBackfill.delete(name)
     this.dirty.add(name)
   }
@@ -873,7 +962,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: an unnest dimension makes one entity contribute to several groups.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -931,7 +1020,7 @@ export class AggregationIndex {
     def: AggregateDefinition,
     entity: Record<string, unknown>
   ): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
 
     // Fan out: reverse the entity's contribution from every group it joined.
     for (const groupKey of computeGroupKeys(entity, def.groupBy)) {
@@ -987,7 +1076,7 @@ export class AggregationIndex {
    * Apply results from native provider back into the state maps.
    */
   private applyNativeResults(aggName: string, results: AggregateGroupState[]): void {
-    const stateMap = this.states.get(aggName)!
+    const stateMap = (this.backfillStaging.get(aggName) ?? this.states.get(aggName))!
     for (const group of results) {
       const serialized = serializeGroupKey(group.groupKey)
       stateMap.set(serialized, group)

src/brainy.ts

@@ -46,8 +46,10 @@ import {
   pageRank,
   MinHeap
 } from './graph/analyticsFallback.js'
+import { runGraphAudit, type GraphAuditReport } from './graph/graphAudit.js'
 import { createPipeline } from './streaming/pipeline.js'
 import { configureLogger, LogLevel, prodLog } from './utils/logger.js'
+import { warnOnLowOsLimits } from './utils/osLimits.js'
 import { setGlobalCache } from './utils/unifiedCache.js'
 import type { UnifiedCache } from './utils/unifiedCache.js'
 import { rankIndicesByScore, reorderByIndices } from './utils/resultRanking.js'
@@ -61,7 +63,9 @@ import type {
   PathOptions,
   MetadataIndexProvider,
   OpaqueIdSet,
-  AtGenerationVectors
+  AtGenerationVectors,
+  VectorIndexProvider,
+  GraphIndexProvider
 } from './plugin.js'
 import type {
   BrainyPlugin,
@@ -70,6 +74,7 @@ import type {
 } from './plugin.js'
 import { ConnectionsCodec } from './hnsw/connectionsCodec.js'
 import { TransactionManager } from './transaction/TransactionManager.js'
+import { transactTimeoutBudget } from './transaction/Transaction.js'
 import { RevisionConflictError } from './transaction/RevisionConflictError.js'
 import { EntityNotFoundError, RelationNotFoundError } from './errors/notFound.js'
 import {
@@ -85,12 +90,12 @@ import { findCallerLocation } from './utils/callerLocation.js'
 import {
   SaveNounMetadataOperation,
   SaveNounOperation,
-  AddToHNSWOperation,
+  AddToVectorIndexOperation,
   AddToMetadataIndexOperation,
   SaveVerbMetadataOperation,
   SaveVerbOperation,
   AddToGraphIndexOperation,
-  RemoveFromHNSWOperation,
+  RemoveFromVectorIndexOperation,
   RemoveFromMetadataIndexOperation,
   RemoveFromGraphIndexOperation,
   UpdateNounMetadataOperation,
@@ -189,6 +194,7 @@ import { MemoryStorage } from './storage/adapters/memoryStorage.js'
 import type {
   CompactHistoryOptions,
   CompactHistoryResult,
+  HistoryStats,
   TransactOptions,
   TransactReceipt,
   TxLogEntry,
@@ -262,6 +268,7 @@ type ResolvedBrainyConfig = Required<
     | 'retention'
     | 'eagerEmbeddings'
     | 'migrationWaitTimeoutMs'
+    | 'transactionBudgetFloorMs'
   >
 > &
   Pick<
@@ -274,6 +281,7 @@ type ResolvedBrainyConfig = Required<
     | 'retention'
     | 'eagerEmbeddings'
     | 'migrationWaitTimeoutMs'
+    | 'transactionBudgetFloorMs'
   >
 
 /**
@@ -384,6 +392,55 @@ class InsertPreconditionExistsSignal extends Error {
  */
 export type IndexFamily = 'vector' | 'metadata' | 'graph'
 
+/**
+ * @description Honest per-surface outcome for {@link Brainy.warm}. Literal
+ * meanings — never conflate the first two:
+ * - `'warmed'` — the provider's own `warm?()` hook ran (vector/graph), or the
+ *   surface's full-hydration seam loaded EVERY shard/field/segment from
+ *   storage (metadata; graph's fallback path). The surface is genuinely at
+ *   steady-state cost for the next operation.
+ * - `'probed'` — no `warm?()` hook was available, so a best-effort read
+ *   (e.g. one `search()` call) faulted in *some* backing storage as a side
+ *   effect. Real work happened, but it is NOT the same guarantee as
+ *   `'warmed'` — never reported as `'warmed'`.
+ * - `'unavailable'` — nothing ran: no hook, no hydration seam, and (for the
+ *   vector probe fallback) nothing to probe (an empty index or unknown
+ *   vector dimension). The surface is unchanged by this `warm()` call.
+ */
+export type WarmOutcome = 'warmed' | 'probed' | 'unavailable'
+
+/**
+ * @description Result of {@link Brainy.warm}: one {@link WarmOutcome} +
+ * elapsed time per index surface, plus the total wall-clock time for the
+ * whole call. `durationMs` is measured around exactly the work described by
+ * that surface's `outcome` (e.g. the vector entry's `durationMs` times the
+ * provider `warm()` call OR the probe `search()` call — whichever ran).
+ */
+export interface WarmReport {
+  vector: { outcome: WarmOutcome; durationMs: number }
+  metadata: { outcome: WarmOutcome; durationMs: number }
+  graph: { outcome: WarmOutcome; durationMs: number }
+  /** Total wall-clock time for the whole `warm()` call (all three surfaces). */
+  totalDurationMs: number
+}
+
+/**
+ * How long a failed aggregation-backfill walk suppresses fresh walk attempts.
+ * Within the window, queries rethrow the recorded failure instantly (loud,
+ * cheap); after it, one new attempt is allowed. Bounds the damage of a
+ * caller-side tight retry loop against a deterministically-failing store.
+ */
+const AGGREGATION_BACKFILL_RETRY_COOLDOWN_MS = 30_000
+
+/**
+ * Time budget for the auto-compaction pass at close() (8.9.0). Bounds how long
+ * a clean shutdown spends reclaiming history backlog — an early stop is a
+ * consistent prefix and the next close/explicit pass resumes. Explicit
+ * `compactHistory()` calls are unbounded unless the caller passes their own
+ * `timeBudgetMs` (maintenance windows choose their own budgets).
+ */
+const CLOSE_COMPACTION_BUDGET_MS = 5_000
+
 /**
  * The main Brainy class - Clean, Beautiful, Powerful
  * REAL IMPLEMENTATION - No stubs, no mocks
@@ -599,6 +656,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
   private _pendingMigrationRunner?: MigrationRunner  // Deferred migration runner for large datasets
   private _aggregationIndex?: AggregationIndex       // Incremental aggregation engine
   private _aggregationBackfillFlight: Promise<void> | null = null  // Single-flight backfill walk
+  // A failed walk latches its error: retries within the cooldown rethrow it
+  // instantly instead of re-walking, so a tight caller-side retry loop costs
+  // one loud error per query, never a full store walk per query.
+  private _aggregationBackfillFailure: { at: number; error: Error } | null = null
   private _materializer?: AggregateMaterializer      // Debounced materialization of aggregate results
   /**
    * Fields registered via `brain.trackField()` — drives optional value validation on
@@ -921,6 +982,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       this.storage = await this.setupStorage()
       await this.storage.init()
 
+      // OS-limit detection (once per process, Linux-only, measurement-only):
+      // warn NOW about RLIMIT_NOFILE / vm.max_map_count values that will bite
+      // at pool scale, instead of letting the operator meet them as EMFILE or
+      // a failed mmap deep inside an index open. Fire-and-forget — the check
+      // never affects open.
+      void warnOnLowOsLimits()
+
       // Acquire the writer lock for filesystem (and other locking-capable) backends.
       // Skipped in reader mode and on backends that don't support multi-process locking.
       // Throws if another live writer holds the directory (unless force: true).
@@ -1182,6 +1250,22 @@ export class Brainy<T = any> implements BrainyInterface<T> {
               `(storage committed: ${committed}) — provider replays the gap per ` +
               `the post-commit applier contract`
           )
+        } else if (providerGen > committed) {
+          // The AHEAD direction is incoherence, not a replay gap: the provider's
+          // persisted index claims writes the store no longer has — the signature
+          // of a torn copy or a log truncation that pulled the committed
+          // watermark back (crash recovery, byte-copy of a live store). A replay
+          // can never converge on it and index answers may reference vanished
+          // writes. Name it loudly at open so it is never diagnosed from a
+          // silent journal; the provider's own coherence check / heal walk (or
+          // brain.repairIndex()) is the cure.
+          prodLog.warn(
+            `[Brainy] Versioned index provider is AHEAD of the store: provider ` +
+              `generation ${providerGen} vs committed ${committed}. This store was ` +
+              `likely copied from a live service or truncated during crash recovery. ` +
+              `Derived-index answers may reference rolled-back writes until the ` +
+              `provider heals from canonical (brain.repairIndex() forces it).`
+          )
         }
       }
 
@@ -1334,6 +1418,17 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         })
       }
 
+      // Eager index warm (operator opt-in, `warmOnOpen: true`). Runs AFTER
+      // every step above — index construction, crash recovery, migrations,
+      // VFS bootstrap — never in place of any of it, so `warm()` always
+      // operates on a fully-initialized brain. Blocking BY DESIGN: the
+      // operator traded a longer startup for a first-request that runs at
+      // steady-state cost instead of paying demand-load latency on the
+      // critical path. See the `warmOnOpen` JSDoc in brainy.types.ts.
+      if (this.config.warmOnOpen) {
+        await this.warm()
+      }
+
       // Resolve ready Promise - consumers awaiting brain.ready will now proceed
       if (this._readyResolve) {
         this._readyResolve()
@@ -1343,6 +1438,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       if (this._readyReject) {
         this._readyReject(error instanceof Error ? error : new Error(String(error)))
       }
+      // Machine-readable init failures pass through UNWRAPPED — the writer-lock
+      // conflict documents an err.code/err.lockInfo contract ("callers detect
+      // this case via err.code"), and wrapping in a fresh Error silently
+      // stripped both, leaving consumers only a message to regex against.
+      if (error instanceof Error && (error as Error & { code?: string }).code === 'BRAINY_WRITER_LOCKED') {
+        throw error
+      }
       throw new Error(`Failed to initialize Brainy: ${error}`)
     }
   }
@@ -1723,7 +1825,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         captureAndCheck({ nouns, verbs } as CommitBeforeImages)
       }
       await this.generationStore.runWithoutGeneration(() =>
-        this.transactionManager.executeTransaction(run)
+        this.transactionManager.executeTransaction(run, {
+          timeout: transactTimeoutBudget(
+            (touched.nouns?.length ?? 0) + (touched.verbs?.length ?? 0),
+            undefined,
+            this.config.transactionBudgetFloorMs
+          )
+        })
       )
       const timestamp = Date.now()
       // Bootstrap writes are not generation-stamped; emit without one.
@@ -1735,7 +1843,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       receipt = await this.generationStore.commitSingleOp({
         touched,
         precommit: captureAndCheck,
-        execute: () => this.transactionManager.executeTransaction(run)
+        execute: () =>
+          this.transactionManager.executeTransaction(run, {
+            timeout: transactTimeoutBudget(
+              (touched.nouns?.length ?? 0) + (touched.verbs?.length ?? 0),
+              undefined,
+              this.config.transactionBudgetFloorMs
+            )
+          })
       })
     } catch (err) {
       // A failed rollback that left the store inconsistent (a remove/update
@@ -1828,6 +1943,29 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     }
   }
 
+  /**
+   * @description Build the AGGREGATION view of an entity from a stored flat
+   * metadata record — EVERY reserved field mapped to its top-level entity
+   * name (stored `noun` → `type`), custom metadata in `metadata`. This must
+   * mirror the add-path `entityForIndexing` shape exactly: the aggregation
+   * engine resolves groupBy/where fields via `resolveEntityField`
+   * (top-level standard fields + custom metadata), so a view that drops a
+   * reserved field makes every aggregate grouped by that field decrement a
+   * group that does not exist — counts then drift upward forever after
+   * deletes (SELF-AGGREGATE-DELETE-DRIFT). Do not hand-roll subsets of this.
+   * @param record - The stored flat metadata record (before-image or pre-delete read).
+   * @returns The full-fidelity entity view for aggregation hooks.
+   */
+  private entityForAggFromRawRecord(record: Record<string, unknown>): Record<string, unknown> {
+    const { reserved, custom } = splitNounMetadataRecord(record)
+    const { noun, ...rest } = reserved
+    return {
+      type: noun,
+      ...rest,
+      metadata: custom
+    }
+  }
+
   /**
    * @description Add an entity (noun) to the brain. Embeds `data` into a vector and
    * indexes the entity across all three intelligences — vector similarity, graph
@@ -2022,7 +2160,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
 
         // Operation 3: Add to HNSW index (after entity saved)
         tx.addOperation(
-          new AddToHNSWOperation(this.index, id, vector)
+          new AddToVectorIndexOperation(this.index, id, vector)
         )
 
         // Operation 4: Add to metadata index
@@ -3011,10 +3149,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         // Operation 3-4: Update HNSW index (remove and re-add if reindexing needed)
         if (needsReindexing) {
           tx.addOperation(
-            new RemoveFromHNSWOperation(this.index, params.id, existing.vector)
+            new RemoveFromVectorIndexOperation(this.index, params.id, existing.vector)
           )
           tx.addOperation(
-            new AddToHNSWOperation(this.index, params.id, vector)
+            new AddToVectorIndexOperation(this.index, params.id, vector)
           )
         }
 
@@ -3070,15 +3208,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
           ]
         : undefined)
 
-      // Aggregation hook (outside transaction — derived data)
+      // Aggregation hook (outside transaction — derived data). `existing` is
+      // the full get() view — every reserved field top-level — and must be
+      // passed whole: a subset view makes the old-side decrement miss any
+      // reserved-field group (update would then double-count it).
       if (this._aggregationIndex) {
-        const oldEntityForAgg = {
+        this._aggregationIndex.onEntityUpdated(
-          type: existing.type,
+          params.id,
-          service: existing.service,
+          entityForIndexing,
-          data: existing.data,
+          existing as unknown as Record<string, unknown>
-          metadata: existing.metadata
+        )
-        }
-        this._aggregationIndex.onEntityUpdated(params.id, entityForIndexing, oldEntityForAgg)
       }
   }
 
@@ -3129,7 +3268,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         // Operation 1: Remove from vector index
         if (noun) {
           tx.addOperation(
-            new RemoveFromHNSWOperation(this.index, id, noun.vector)
+            new RemoveFromVectorIndexOperation(this.index, id, noun.vector)
           )
         }
 
@@ -3190,19 +3329,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
           ]
         : undefined)
 
-      // Aggregation hook (outside transaction — derived data)
+      // Aggregation hook (outside transaction — derived data). The view must
+      // carry EVERY reserved field top-level (not a subset): a groupBy on
+      // subtype/visibility/etc. otherwise decrements a nonexistent group and
+      // the real count never comes down.
       if (this._aggregationIndex && metadata) {
-        // Reconstruct entity-like object from stored metadata via the
+        this._aggregationIndex.onEntityDeleted(
-        // canonical reserved/custom split (the hand-rolled destructure here
+          id,
-        // missed subtype/_rev, leaking them into the aggregation view).
+          this.entityForAggFromRawRecord(metadata as Record<string, unknown>)
-        const { reserved, custom } = splitNounMetadataRecord(metadata)
+        )
-        const entityForAgg = {
-          type: reserved.noun,
-          service: reserved.service,
-          data: reserved.data,
-          metadata: custom
-        }
-        this._aggregationIndex.onEntityDeleted(id, entityForAgg)
       }
   }
 
@@ -6626,10 +6761,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
   /**
    * Add multiple entities in a single batch operation
    *
-   * Uses batch embedding (embedBatch) to pre-compute all vectors in a single
+   * Uses batch embedding (embedBatch) to pre-compute all vectors before any
-   * WASM forward pass instead of N individual embed() calls, providing 5-10x
+   * storage write, keeping model inference out of the per-item write path.
-   * speedup on bulk inserts. Automatically adapts batch size and parallelism
+   * (On the default WASM engine, batch throughput is comparable to sequential
-   * to the storage adapter (e.g., smaller batches for cloud storage).
+   * embed() calls — measured ~160 ms/text either way; a native embedding
+   * provider may batch faster.) Automatically adapts batch size and
+   * parallelism to the storage adapter (e.g., smaller batches for cloud
+   * storage).
    *
    * @param params - Batch add parameters
    * @param params.items - Array of AddParams (same shape as brain.add())
@@ -6827,6 +6965,30 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     this.assertWritable('removeMany')
     await this.ensureInitialized()
 
+    // Loud selector validation: a call with no usable selector used to
+    // resolve successfully having deleted NOTHING (total: 0) — the classic
+    // silent no-op being a bare array passed positionally
+    // (removeMany([id]) instead of removeMany({ ids: [id] })). The caller
+    // believes the delete happened; every count derived afterwards is "wrong"
+    // while the engine was never even asked. Refuse instead.
+    if (Array.isArray(params)) {
+      throw new Error(
+        `removeMany() takes a params object, not a bare array — use removeMany({ ids: [...] })`
+      )
+    }
+    if (!params || (!params.ids && !params.type && !params.where)) {
+      throw new Error(
+        `removeMany() requires a selector: { ids } and/or { type, where }. ` +
+          `An empty selector would silently delete nothing — refusing.`
+      )
+    }
+    if (params.ids && params.ids.length === 0) {
+      throw new Error(
+        `removeMany() received ids: [] — an empty id list deletes nothing. ` +
+          `Pass the ids to delete, or omit ids and select by { type, where }.`
+      )
+    }
+
     // Determine what to delete
     let idsToDelete: string[] = []
 
@@ -6914,7 +7076,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
               // Add delete operations to transaction
               if (noun) {
                 tx.addOperation(
-                  new RemoveFromHNSWOperation(this.index, id, noun.vector)
+                  new RemoveFromVectorIndexOperation(this.index, id, noun.vector)
                 )
               }
 
@@ -7768,11 +7930,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
         ifAtGeneration: options?.ifAtGeneration,
         precommit: casPrecommit,
         execute: async () => {
-          await this.transactionManager.executeTransaction(async (tx) => {
+          await this.transactionManager.executeTransaction(
+            async (tx) => {
               for (const operation of plan.operations) {
                 tx.addOperation(operation)
               }
-          })
+            },
+            // Budget scales with the batch (or the caller's explicit
+            // timeoutMs): a flat 30s cap silently limited honest bulk work
+            // to ~15 ops on network disks (~2s/op measured in the field).
+            {
+              timeout: transactTimeoutBudget(
+                plan.operations.length,
+                options?.timeoutMs,
+                this.config.transactionBudgetFloorMs
+              )
+            }
+          )
         }
       }))
     } catch (err) {
@@ -8148,6 +8322,72 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     return this.generationStore.compact(options)
   }
 
+  /**
+   * @description Repack cold generation history into sealed segments —
+   * re-representation, never deletion: every record and delta stays readable
+   * (`asOf()` unchanged); the physical file count drops by orders of
+   * magnitude. Runs automatically (time-bounded) at `close()`; call this for
+   * explicit maintenance windows on long-lived writers. The ONLY history
+   * transform permitted under the archival profile (`retention: 'all'`).
+   * @param options - `timeBudgetMs` bounds the pass (early stop = consistent
+   *   prefix, next pass resumes); `batchGenerations` sizes each fold.
+   * @returns Folded generation count and segments created.
+   */
+  async repackHistory(options?: {
+    timeBudgetMs?: number
+    batchGenerations?: number
+  }): Promise<{ foldedGenerations: number; segmentsCreated: number }> {
+    this.assertWritable('repackHistory')
+    await this.ensureInitialized()
+    await this.generationStore.flushPendingSingleOps()
+    return this.generationStore.repackHistory(options)
+  }
+
+  /**
+   * @description Read-only generational-history footprint for fleet audits:
+   * generation count, total on-disk bytes, generation/timestamp range, the
+   * compaction horizon, and the retention policy in force. Touches no data and
+   * changes nothing. First call pays one walk over committed deltas to seed
+   * the running byte total (subsequent calls — and every adaptive retention
+   * check — are then O(1)).
+   *
+   * A pool operator's exposure check is one call per brain:
+   * @example
+   * const stats = await brain.historyStats()
+   * console.log(`${stats.generations} generations, ${stats.bytes} bytes, mode=${stats.retentionMode}`)
+   */
+  async historyStats(): Promise<HistoryStats> {
+    await this.ensureInitialized()
+    const stats = await this.generationStore.historyStats()
+    const policy = this.resolveRetentionPolicy()
+    return {
+      ...stats,
+      retentionMode: policy.mode,
+      effectiveBudgetBytes:
+        policy.mode === 'adaptive'
+          ? this.adaptiveHistoryBudgetBytes(policy.budgetBytes)
+          : null
+    }
+  }
+
+  /**
+   * @description A deterministic content digest of the generation log through
+   * `g` (D8 — gate-to-generation provenance): identical history produces the
+   * identical digest on any machine; divergence produces a different one.
+   * Release gates and suite verdicts pin `{generation, digest}` and verify
+   * both at execution time instead of pinning a git commit. O(segments +
+   * live-tier window), never O(all generations). Throws `RangeError` out of
+   * range and `GenerationCompactedError` below the horizon — a gate can
+   * never silently pin reclaimed history.
+   * @example
+   * const gate = { generation: brain.generation(), digest: await brain.generationDigest(brain.generation()) }
+   */
+  async generationDigest(g: number): Promise<string> {
+    await this.ensureInitialized()
+    await this.generationStore.flushPendingSingleOps()
+    return this.generationStore.generationDigest(g)
+  }
+
   /**
    * @description Drive the adaptive retention byte budget at runtime — the
    * settable input a machine-level coordinator (e.g. cor's `ResourceManager`,
@@ -8227,19 +8467,24 @@ export class Brainy<T = any> implements BrainyInterface<T> {
 
   /**
    * @description Run history compaction under the resolved `retention` policy
-   * when `autoCompact` is on (the default). Invoked from `flush()` and
+   * when `autoCompact` is on (the default). Invoked from `close()` ONLY
-   * `close()` so generational record-sets cannot accumulate unbounded across a
+   * (8.9.0) — flush() is durability work and never pays maintenance costs; a
-   * long-lived writer's lifetime.
+   * production deployment measured reclaim-on-flush blocking single writes
+   * for 25-191s under memory pressure. Long-lived writers that never close
+   * accumulate history until their next explicit `compactHistory()` — the
+   * documented trade: predictable writes, explicit maintenance.
    *
    * - `'all'` → returns without reclaiming (index compaction for speed still
    *   runs elsewhere; history is decoupled and kept).
    * - `'adaptive'` → reclaim oldest-unpinned history down to the byte budget.
    * - `'explicit'` → apply the supplied `maxGenerations`/`maxAge`/`maxBytes` caps.
    *
+   * Every auto pass is TIME-BOUNDED ({@link CLOSE_COMPACTION_BUDGET_MS}) so a
+   * large backlog can never stall a clean shutdown; the next pass resumes.
    * Read-only instances and an explicit `autoCompact: false` skip silently.
    * Pinned generations are never reclaimed ({@link GenerationStore.compact}).
    * Failures are logged and swallowed — compaction is housekeeping and must
-   * never fail a flush or a clean shutdown.
+   * never fail a clean shutdown.
    */
   private async autoCompactHistory(): Promise<void> {
     // Nothing to compact on a read-only instance or before init wired up the
@@ -8255,12 +8500,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       if (policy.mode === 'adaptive') {
         const budget = this.adaptiveHistoryBudgetBytes(policy.budgetBytes)
         if (budget === Infinity) return // no pressure signal → keep everything this pass
-        await this.generationStore.compact({ maxBytes: budget })
+        await this.generationStore.compact({
+          maxBytes: budget,
+          timeBudgetMs: CLOSE_COMPACTION_BUDGET_MS
+        })
       } else {
         await this.generationStore.compact({
           maxGenerations: policy.maxGenerations,
           maxAge: policy.maxAge,
-          maxBytes: policy.maxBytes
+          maxBytes: policy.maxBytes,
+          timeBudgetMs: CLOSE_COMPACTION_BUDGET_MS
         })
       }
     } catch (error) {
@@ -9119,7 +9368,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     plan.operations.push(
       new SaveNounMetadataOperation(this.storage, id, storageMetadata, isNew),
       new SaveNounOperation(this.storage, { id, vector, connections: new Map(), level: 0 }, isNew),
-      new AddToHNSWOperation(this.index, id, vector),
+      new AddToVectorIndexOperation(this.index, id, vector),
       new AddToMetadataIndexOperation(this.metadataIndex, id, entityForIndexing)
     )
     plan.touchedNouns.push(id)
@@ -9287,8 +9536,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     )
     if (needsReindexing) {
       plan.operations.push(
-        new RemoveFromHNSWOperation(this.index, params.id, existing.vector),
+        new RemoveFromVectorIndexOperation(this.index, params.id, existing.vector),
-        new AddToHNSWOperation(this.index, params.id, vector)
+        new AddToVectorIndexOperation(this.index, params.id, vector)
       )
     }
     plan.operations.push(
@@ -9297,12 +9546,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     )
     plan.touchedNouns.push(params.id)
 
-    const oldEntityForAgg = {
+    // The full planGetEntity view, passed whole — a subset view makes the
-      type: existing.type,
+    // old-side decrement miss reserved-field groups (double-count on update).
-      service: existing.service,
+    const oldEntityForAgg = existing as unknown as Record<string, unknown>
-      data: existing.data,
-      metadata: existing.metadata
-    }
     plan.postCommit.push(() => {
       if (this._aggregationIndex) {
         this._aggregationIndex.onEntityUpdated(params.id, entityForIndexing, oldEntityForAgg)
@@ -9376,7 +9622,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     }
 
     if (noun) {
-      plan.operations.push(new RemoveFromHNSWOperation(this.index, id, noun.vector))
+      plan.operations.push(new RemoveFromVectorIndexOperation(this.index, id, noun.vector))
     }
     if (metadata) {
       plan.operations.push(new RemoveFromMetadataIndexOperation(this.metadataIndex, id, metadata))
@@ -9423,14 +9669,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     }
 
     if (metadata) {
-      // Canonical reserved/custom split — mirror of remove()'s aggregation hook.
+      // Mirror of remove()'s aggregation hook — the FULL reserved view, so
-      const { reserved, custom } = splitNounMetadataRecord(metadata)
+      // reserved-field groupBy decrements find their group.
-      const entityForAgg = {
+      const entityForAgg = this.entityForAggFromRawRecord(metadata as Record<string, unknown>)
-        type: reserved.noun,
-        service: reserved.service,
-        data: reserved.data,
-        metadata: custom
-      }
       plan.postCommit.push(() => {
         if (this._aggregationIndex) {
           this._aggregationIndex.onEntityDeleted(id, entityForAgg)
@@ -10065,6 +10306,30 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     return await coordinator.import(source as Buffer | string | object, options)
   }
 
+  /** Brain-owned background deduplicator (lazy; see getBackgroundDeduplicator). */
+  private _backgroundDedup?: import('./import/BackgroundDeduplicator.js').BackgroundDeduplicator
+
+  /**
+   * The single brain-owned BackgroundDeduplicator, lazily constructed.
+   *
+   * Ownership matters here: the post-import dedup timer must outlive the
+   * per-call ImportCoordinator but never the brain. One instance per brain
+   * restores the intended cross-import debounce (per-coordinator instances
+   * each armed their own timer, so the "debounce" never spanned imports) and
+   * gives close() a handle to cancel pending work — a delete pass must never
+   * fire against a closed brain.
+   * @internal
+   */
+  async getBackgroundDeduplicator(): Promise<
+    import('./import/BackgroundDeduplicator.js').BackgroundDeduplicator
+  > {
+    if (!this._backgroundDedup) {
+      const { BackgroundDeduplicator } = await import('./import/BackgroundDeduplicator.js')
+      this._backgroundDedup = new BackgroundDeduplicator(this)
+    }
+    return this._backgroundDedup
+  }
+
   /**
    * Virtual File System API - Knowledge Operating System
    *
@@ -10389,12 +10654,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       this.generationStore.persistCounterNow()
     ])
 
-    // 6. Auto-compact generational history per config.retention (default on).
+    // NOTE (8.9.0): flush() no longer compacts history. Flush is DURABILITY
-    //    Runs after the flush so durable state is in place; respects live
+    // work — it must cost what this window's deltas cost, never what the
-    //    Db pins and an explicit autoCompact: false.
+    // history backlog costs. Auto-compaction (a MAINTENANCE concern) runs at
-    await this.autoCompactHistory()
+    // close() and via explicit compactHistory(); a production deployment
+    // measured reclaim-on-flush stalling single writes for 25-191s under
+    // memory pressure, which is exactly the class this separation ends.
 
-    // 7. Stamp the entity tree: which source generation the canonical tree
+    // 6. Stamp the entity tree: which source generation the canonical tree
     //    reflects + the rollup invariants that verify it whole (the counters
     //    persisted in step 1). Written at flush boundaries — the tree tracks
     //    every commit by construction, so the stamp is a durable checkpoint,
@@ -14070,6 +14337,118 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     return this.embedder(textToEmbed)
   }
 
+  /**
+   * Eagerly load/fault-in the vector index, metadata index, and graph
+   * adjacency so the FIRST real operation after this call runs at
+   * steady-state cost — no page-cache-miss / demand-load latency on the
+   * critical path. Complements {@link warmupEmbeddings} (which warms the
+   * embedding engine, not the storage indexes).
+   *
+   * Sequence per surface:
+   * - **Vector**: calls the provider's own `warm?()` when the active
+   *   `'vector'` provider implements it (`'warmed'`). Otherwise runs a
+   *   best-effort probe — one `search()` with a deterministic unit vector of
+   *   the index's known dimension, `k = min(100, size())` — which faults in
+   *   *some* backing storage as a side effect but is reported honestly as
+   *   `'probed'`, never `'warmed'`. An empty index or unknown dimension has
+   *   nothing to probe (`'unavailable'`).
+   * - **Metadata**: full hydration — every persisted field's sparse index is
+   *   loaded from storage (`MetadataIndexManager.hydrateAll()`), not just the
+   *   heuristic common-fields subset `init()` warms.
+   * - **Graph**: calls the provider's own `warm?()` when the active graph
+   *   provider implements it; otherwise re-runs its existing eager cold-load
+   *   `init()` seam (idempotent — the JS adjacency index's `init()` already
+   *   loads the full LSM manifests + SSTables, so re-invoking it here is a
+   *   genuine full hydration, not a stat call).
+   *
+   * A single surface's `'unavailable'`/probe outcome never fails the whole
+   * call — `warm()` is a best-effort readiness step, not a correctness gate;
+   * queries still demand-load normally regardless of what `warm()` achieved.
+   *
+   * @returns A {@link WarmReport}: honest per-surface outcome + timing.
+   * @example
+   * ```typescript
+   * const brain = new Brainy({ storage: { path: '/data' } })
+   * await brain.init()
+   * const report = await brain.warm() // blocking, explicit control over timing
+   * console.log(report.vector.outcome, report.vector.durationMs)
+   *
+   * // Or opt into the same work automatically during init():
+   * const eager = new Brainy({ storage: { path: '/data' }, warmOnOpen: true })
+   * await eager.init() // warm() already ran before this resolves
+   * ```
+   */
+  async warm(): Promise<WarmReport> {
+    await this.ensureInitialized({ needs: ['vector', 'metadata', 'graph'] })
+
+    const totalStart = Date.now()
+
+    // --- Vector ---------------------------------------------------------
+    const vectorStart = Date.now()
+    let vectorOutcome: WarmOutcome
+    const vectorProvider = this.index as VectorIndexProvider & { warm?: () => Promise<void> }
+    if (typeof vectorProvider.warm === 'function') {
+      await vectorProvider.warm()
+      vectorOutcome = 'warmed'
+    } else {
+      const size = this.index.size()
+      const dimension = this.dimensions
+      if (size > 0 && dimension && dimension > 0) {
+        // A deterministic unit vector (L2 norm 1) — same probe every call,
+        // no reliance on stored data shape.
+        const probeVector = new Array(dimension).fill(1 / Math.sqrt(dimension))
+        await this.index.search(probeVector, Math.min(100, size))
+        vectorOutcome = 'probed'
+      } else {
+        // Nothing to probe: an empty index, or the vector dimension is not
+        // yet known (no entity has ever been added).
+        vectorOutcome = 'unavailable'
+      }
+    }
+    const vectorDurationMs = Date.now() - vectorStart
+
+    // --- Metadata --------------------------------------------------------
+    const metadataStart = Date.now()
+    let metadataOutcome: WarmOutcome
+    const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise<void> }
+    if (typeof metadataWithHydrate.hydrateAll === 'function') {
+      await metadataWithHydrate.hydrateAll()
+      metadataOutcome = 'warmed'
+    } else {
+      // No hydration seam on this metadata provider — nothing to run.
+      metadataOutcome = 'unavailable'
+    }
+    const metadataDurationMs = Date.now() - metadataStart
+
+    // --- Graph -------------------------------------------------------------
+    const graphStart = Date.now()
+    let graphOutcome: WarmOutcome
+    const graphProvider = this.graphIndex as GraphIndexProvider & {
+      warm?: () => Promise<void>
+      init?: () => Promise<void>
+    }
+    if (typeof graphProvider.warm === 'function') {
+      await graphProvider.warm()
+      graphOutcome = 'warmed'
+    } else if (typeof graphProvider.init === 'function') {
+      // Existing eager cold-load seam (readiness contract): idempotent, and
+      // for the JS adjacency index it's already a genuine full load of the
+      // LSM manifests + SSTables — a real hydration, not a stat call.
+      await graphProvider.init()
+      graphOutcome = 'warmed'
+    } else {
+      graphOutcome = 'unavailable'
+    }
+    const graphDurationMs = Date.now() - graphStart
+
+    return {
+      vector: { outcome: vectorOutcome, durationMs: vectorDurationMs },
+      metadata: { outcome: metadataOutcome, durationMs: metadataDurationMs },
+      graph: { outcome: graphOutcome, durationMs: graphDurationMs },
+      totalDurationMs: Date.now() - totalStart
+    }
+  }
+
   /**
    * Explicitly warm up the embedding engine
    *
@@ -14588,6 +14967,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       // Migration LOCK wait budget — left undefined when omitted so
       // awaitMigrationLock() applies its 30 s default at the read site.
       migrationWaitTimeoutMs: config?.migrationWaitTimeoutMs ?? undefined,
+      // Apply-phase transaction budget floor — left undefined when omitted so
+      // transactTimeoutBudget() applies its 30 s default at the read site.
+      transactionBudgetFloorMs: config?.transactionBudgetFloorMs ?? undefined,
       // Pre-upgrade backup — default-on; opt out with `migrationBackup: false`.
       migrationBackup: config?.migrationBackup ?? true,
       // Vector index configuration (8.0) — algorithm-neutral surface with
@@ -14603,6 +14985,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       // is the active one on a non-reader writer outside unit tests). Explicit
       // true/false always wins. See the eagerEmbeddings JSDoc in brainy.types.ts.
       eagerEmbeddings: config?.eagerEmbeddings ?? undefined,
+      // Eager index warm at init() — default off (lazy demand-load), the
+      // pre-8.9 behavior. See the warmOnOpen JSDoc in brainy.types.ts.
+      warmOnOpen: config?.warmOnOpen ?? false,
       // Plugin configuration - undefined = auto-detect
       plugins: config?.plugins ?? undefined,
       // Integration Hub - undefined/false = disabled
@@ -15461,6 +15846,88 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     )
   }
 
+  /**
+   * Read-only graph-truth audit — proves (or disproves) that relationship
+   * reads return canonical truth on THIS brain, and classifies every
+   * discrepancy into its failure family:
+   *
+   * - `missingFromReads` — canonical verb records the read path omits
+   *   (PRESENT BUT INVISIBLE: adjacency/membership staleness)
+   * - `danglingEndpoints` — verbs whose endpoint entity is gone (SCAR class)
+   * - `readOnlyVerbIds` — read-path edges with no canonical record (GHOSTS)
+   *
+   * Design-hidden edges (internal/system visibility) are counted separately —
+   * the audit reads with every visibility tier included, so intentional
+   * hiding is never misclassified as index loss. Mutates nothing; safe on a
+   * live brain (cost: one canonical walk + one indexed read per source).
+   * Run it after any engine upgrade, restore, or migration; a `coherent`
+   * report is the verified statement that `related()`/`readdir` can be
+   * trusted. If it reports discrepancies, `repairIndex()` is the sanctioned
+   * heal — re-run the audit afterwards to prove the repair.
+   *
+   * @param options.maxExamples - Cap per example list (counts stay exact). Default 100.
+   * @returns The full audit report; also narrated via logs (loud on incoherence).
+   * @since 8.6.0
+   */
+  async auditGraph(options: { maxExamples?: number } = {}): Promise<GraphAuditReport> {
+    await this.ensureInitialized({ needs: ['graph'] })
+
+    const PAGE = 1000
+    return runGraphAudit(
+      {
+        eachNounId: async (consume) => {
+          let cursor: string | undefined
+          let offset = 0
+          for (;;) {
+            const page = await (this.storage as unknown as {
+              getNounIdsWithPagination(o: {
+                limit: number
+                offset?: number
+                cursor?: string
+              }): Promise<{ ids: string[]; hasMore: boolean; nextCursor?: string }>
+            }).getNounIdsWithPagination(
+              cursor ? { limit: PAGE, cursor } : { limit: PAGE, offset }
+            )
+            for (const id of page.ids) consume(id)
+            if (!page.hasMore || page.ids.length === 0) break
+            if (page.nextCursor) cursor = page.nextCursor
+            else offset += page.ids.length
+          }
+        },
+        eachVerb: async (consume) => {
+          let cursor: string | undefined
+          let offset = 0
+          for (;;) {
+            const page = await this.storage.getVerbs({
+              pagination: cursor ? { limit: PAGE, cursor } : { limit: PAGE, offset }
+            })
+            for (const verb of page.items) {
+              const v = verb as unknown as Record<string, unknown>
+              consume({
+                id: String(v.id),
+                type: String(v.verb ?? 'unknown'),
+                sourceId: String(v.sourceId),
+                targetId: String(v.targetId),
+                visibility: typeof v.visibility === 'string' ? v.visibility : undefined
+              })
+            }
+            if (!page.hasMore || page.items.length === 0) break
+            if (page.nextCursor) cursor = page.nextCursor
+            else offset += page.items.length
+          }
+        },
+        readRelationsFrom: async (sourceId) =>
+          this.related({
+            from: sourceId,
+            includeInternal: true,
+            includeSystem: true,
+            limit: 100000
+          })
+      },
+      options
+    )
+  }
+
   async repairIndex(): Promise<void> {
     await this.ensureInitialized()
 
@@ -15828,6 +16295,18 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     // the in-flight walk snapshotted its batch before `name` became pending —
     // the next iteration starts a fresh walk that includes it.
     while (index.getPendingBackfills().includes(name)) {
+      // Failure latch: a deterministically-failing walk must not be re-run at
+      // the caller's retry rate — that is a silent CPU loop wearing a retry
+      // loop's clothes. Within the cooldown, rethrow the recorded failure
+      // immediately; after it, one fresh attempt is allowed.
+      const failure = this._aggregationBackfillFailure
+      if (failure && Date.now() - failure.at < AGGREGATION_BACKFILL_RETRY_COOLDOWN_MS) {
+        throw new Error(
+          `Aggregation backfill for '${name}' is in failure cooldown (retry in ` +
+            `${Math.ceil((AGGREGATION_BACKFILL_RETRY_COOLDOWN_MS - (Date.now() - failure.at)) / 1000)}s). ` +
+            `Last failure: ${failure.error.message}`
+        )
+      }
       if (!this._aggregationBackfillFlight) {
         this._aggregationBackfillFlight = this.runAggregationBackfillWalk()
           .finally(() => {
@@ -15850,8 +16329,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     const names = index.getPendingBackfills()
     if (names.length === 0) return
 
+    prodLog.info(`[Aggregation] backfill walk starting for: ${names.join(', ')}`)
+    const startedAt = Date.now()
     for (const n of names) index.beginBackfill(n)
 
+    let scanned = 0
+    try {
       const PAGE = 500
       let offset = 0
       let cursor: string | undefined
@@ -15865,15 +16348,41 @@ export class Brainy<T = any> implements BrainyInterface<T> {
             index.backfillEntity(n, record)
           }
         }
+        scanned += page.items.length
         if (!page.hasMore || page.items.length === 0) break
         if (page.nextCursor) {
+          if (page.nextCursor === cursor) {
+            // A non-advancing cursor with hasMore=true would loop this walk at
+            // CPU speed forever, silently. That is a storage pagination defect —
+            // fail the waiting queries loudly instead of spinning.
+            throw new Error(
+              `Aggregation backfill aborted: storage pagination returned a non-advancing cursor ` +
+                `after ${scanned} entities with hasMore=true — the storage adapter's getNouns cursor is broken.`
+            )
+          }
           cursor = page.nextCursor
         } else {
           offset += page.items.length
         }
       }
+    } catch (err) {
+      // Non-destructive failure: drop the staging maps (live state keeps
+      // serving), keep the aggregates flagged pending, latch the error so
+      // retries within the cooldown fail fast, and say all of it out loud.
+      for (const n of names) index.abortBackfill(n)
+      this._aggregationBackfillFailure = { at: Date.now(), error: err as Error }
+      prodLog.warn(
+        `[Aggregation] backfill walk FAILED after ${scanned} entities: ${(err as Error).message} — ` +
+          `prior aggregate state preserved; retries suppressed for ${AGGREGATION_BACKFILL_RETRY_COOLDOWN_MS / 1000}s`
+      )
+      throw err
+    }
 
     for (const n of names) index.finishBackfill(n)
+    this._aggregationBackfillFailure = null
+    prodLog.info(
+      `[Aggregation] backfill walk finished: ${scanned} entities → ${names.length} aggregate(s) in ${Date.now() - startedAt}ms`
+    )
   }
 
   /**
@@ -15883,6 +16392,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
    * This ensures deferred persistence mode data is saved
    */
   async close(): Promise<void> {
+    // Cancel any pending post-import background deduplication FIRST — it is a
+    // writer (merge-deletes), and no delete pass may start mid- or post-close.
+    this._backgroundDedup?.cancelPending()
+
     // Change-feed teardown: no events are delivered for or after close().
     this._changeFeed.close()
 
@@ -15893,9 +16406,21 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       await this.generationStore.flushPendingSingleOps()
     }
 
-    // Phase 0b: Auto-compact generational history per config.retention (default
+    // Phase 0b: REPACK cold history into sealed segments (D1+D3 —
-    // on) BEFORE the generation store closes below. Respects live Db pins and
+    // re-representation, never deletion; the only history transform under the
-    // an explicit autoCompact: false; no-op on read-only instances.
+    // archival profile), then auto-compact per config.retention. Repack runs
+    // FIRST so bounded-retention reclaim can drop whole segments. Both are
+    // time-bounded maintenance passes (8.9.0 law: flush() never pays these);
+    // both are housekeeping — failures warn, never fail a clean shutdown.
+    if (!this.isReadOnly && this.generationStore) {
+      try {
+        await this.generationStore.repackHistory({ timeBudgetMs: 5_000 })
+      } catch (error) {
+        console.warn(
+          `History repacking failed (non-fatal): ${error instanceof Error ? error.message : String(error)}`
+        )
+      }
+    }
     await this.autoCompactHistory()
 
     // Phase 1: Flush ALL components in parallel to persist buffered data

src/db/factLog.ts

@@ -102,12 +102,26 @@ export interface FactScanBatch {
   segmentId: string
 }
 
+/**
+ * Liveness bound on a scan's FIRST batch (Stage-2 co-freeze, D1 contract):
+ * `batches()` must yield its first batch — or fail loudly — within this many
+ * ms of the first pull. A backlogged or damaged store may be SLOW, but it may
+ * never be SILENT: a consumer awaiting the first batch is otherwise
+ * indistinguishable from a wedge (the exact failure shape a production heal
+ * hit against a generations-backlogged brain).
+ */
+export const SCANFACTS_FIRST_BATCH_MS = 10_000
+
 /** The telemetry a scan OPEN returns (frozen shape). */
 export interface FactScanHandle {
   headGeneration: number
   segmentCount: number
   approxFactCount: number
-  /** Ordered batches; a detected gap aborts LOUDLY, never a silent skip. */
+  /**
+   * Ordered batches; a detected gap aborts LOUDLY, never a silent skip.
+   * Liveness contract: the FIRST batch resolves or rejects within
+   * {@link SCANFACTS_FIRST_BATCH_MS} of the first pull — never a silent hang.
+   */
   batches: () => AsyncGenerator<FactScanBatch>
   /** Close telemetry — the invariant cross-check, valid after iteration ends. */
   summary: () => { factsYielded: number; segmentsRead: number }
@@ -440,6 +454,8 @@ export class FactLog {
     toGeneration?: number
     kinds?: Array<'noun' | 'verb'>
     batchSize?: number
+    /** Test override for the first-batch liveness bound (default {@link SCANFACTS_FIRST_BATCH_MS}). */
+    firstBatchTimeoutMs?: number
   }): FactScanHandle {
     const from = options?.fromGeneration ?? 1
     const to = options?.toGeneration ?? this.head
@@ -514,11 +530,43 @@ export class FactLog {
       }
     }
 
+    // Liveness wrapper: the FIRST pull races the contract deadline. Only the
+    // first — the bound is time-to-first-batch (proof the producer is alive),
+    // not per-batch pacing; and it runs only while a pull is actually pending,
+    // so consumer think-time between pulls never counts against the producer.
+    const firstBatchTimeoutMs = options?.firstBatchTimeoutMs ?? SCANFACTS_FIRST_BATCH_MS
+    async function* batchesWithLiveness(this: void): AsyncGenerator<FactScanBatch> {
+      const inner = batches()
+      let timer: NodeJS.Timeout | undefined
+      try {
+        const deadline = new Promise<never>((_, reject) => {
+          timer = setTimeout(
+            () =>
+              reject(
+                new Error(
+                  `fact log: scanFacts produced no first batch within ${firstBatchTimeoutMs}ms ` +
+                    `(liveness contract) — the store is wedged or unreadably slow; aborting scan LOUDLY ` +
+                    `instead of hanging the consumer.`
+                )
+              ),
+            firstBatchTimeoutMs
+          )
+          timer.unref?.()
+        })
+        const first = await Promise.race([inner.next(), deadline])
+        if (first.done) return
+        yield first.value
+      } finally {
+        clearTimeout(timer)
+      }
+      yield* inner
+    }
+
     return {
       headGeneration: this.head,
       segmentCount: segments.length + (tailSnapshot.length > 0 ? 1 : 0),
       approxFactCount,
-      batches,
+      batches: batchesWithLiveness,
       summary: () => ({ factsYielded, segmentsRead })
     }
   }

src/db/generationSegments.ts

@@ -0,0 +1,459 @@
+/**
+ * @module db/generationSegments
+ * @description The generation-segment store — Stage-2 D1+D3+repacking's file
+ * format (co-frozen 2026-07-19; design: the d1-d3-repacking spec).
+ *
+ * Packs CONSECUTIVE cold generations' record-sets (before-images + delta)
+ * into append-once segment files with derived sidecar indexes, so history
+ * scales in SEGMENTS (tens) instead of FILES-PER-GENERATION (hundreds of
+ * thousands), and cold-open reads ONE manifest instead of listing the
+ * backlog. Layout under `_generations/segments/`:
+ *
+ * - `seg-<firstGen, zero-padded 20>.bgs` — magic "BGS1", then one frame per
+ *   generation: `u32 payloadLen | u32 crc32c | msgpack payload`. Payload is
+ *   POSITIONAL: `[generation, timestamp, delta, records[], flags]` with
+ *   records `[kindByte, id, record]`. `flags` reserves encoding evolution
+ *   (bit 0 = compressed payload — v1 always 0; a future writer upgrade,
+ *   never a format break). Sealed segments are IMMUTABLE — the fact log's
+ *   own law, generalized.
+ * - `seg-<firstGen>.idx` — DERIVED sidecar (msgpack): per-generation frame
+ *   offsets (point reads = one ranged read, never a listing) + per-id
+ *   generation postings (per-id chain rebuilds read only what they need).
+ *   Corrupt/missing → rebuilt from its segment in one sequential read,
+ *   loudly.
+ * - `manifest.json` — the segment catalogue + `compactedBelow` (D3's
+ *   horizon marker). Cold-open reads THIS; the packed backlog is never
+ *   listed.
+ *
+ * D3 semantics carried here: bounded-retention reclaim drops WHOLE segments
+ * at boundaries (O(1) per segment, no rewrite); under the archival profile
+ * (`retention: 'all'`) nothing here is ever dropped — folding is the only
+ * transform (re-representation, never deletion).
+ */
+
+import { encode as msgpackEncode, decode as msgpackDecode } from '@msgpack/msgpack'
+import { crc32c } from '../utils/crc32c.js'
+import type { FactLogStorage } from './factLog.js'
+import { prodLog } from '../utils/logger.js'
+
+/** Directory for segment files + manifest, under the generations prefix. */
+export const SEGMENTS_PREFIX = '_generations/segments'
+
+/** Target sealed-segment size (co-freeze proposal; tunable on evidence). */
+export const SEGMENT_TARGET_BYTES = 64 * 1024 * 1024
+
+const MAGIC = new TextEncoder().encode('BGS1')
+const FRAME_PREFIX_BYTES = 8 // u32 payloadLen + u32 crc32c
+const MANIFEST_PATH = `${SEGMENTS_PREFIX}/manifest.json`
+
+/** One generation's fold input — exactly what the live tier holds for it. */
+export interface FoldGeneration {
+  generation: number
+  timestamp: number
+  /** The tx.json delta object, carried verbatim. */
+  delta: unknown
+  /** The before-image record-set (empty for record-less generations). */
+  records: Array<{ kind: 'noun' | 'verb'; id: string; record: unknown }>
+}
+
+/** Manifest entry for one sealed segment. */
+export interface SegmentMeta {
+  file: string
+  firstGeneration: number
+  lastGeneration: number
+  frames: number
+  bytes: number
+  /** crc32c of the full segment byte stream — the digest chain's link. */
+  checksum: number
+}
+
+interface SegmentManifest {
+  version: 1
+  compactedBelow: number
+  segments: SegmentMeta[]
+}
+
+interface SidecarIndex {
+  version: 1
+  /** [generation, frameOffset, frameLen] ascending by generation. */
+  generations: Array<[number, number, number]>
+  /** `${kindByte}:${id}` → ascending generations holding a record for it. */
+  ids: Record<string, number[]>
+}
+
+const segmentFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.bgs`
+const sidecarFileName = (firstGeneration: number): string =>
+  `seg-${String(firstGeneration).padStart(20, '0')}.idx`
+
+/**
+ * The generation-segment store. Owns the packed tier ONLY — the live
+ * per-generation tier and the routing between tiers belong to
+ * `GenerationStore`. All mutating entry points here are called under the
+ * generation store's commit mutex.
+ */
+export class GenerationSegmentStore {
+  private readonly storage: FactLogStorage
+  private manifest: SegmentManifest = { version: 1, compactedBelow: 0, segments: [] }
+  /** Sidecar cache — segments are immutable, so entries never invalidate. */
+  private readonly sidecars = new Map<string, SidecarIndex>()
+
+  constructor(storage: FactLogStorage) {
+    this.storage = storage
+  }
+
+  /** Load the manifest (ONE read — never a directory listing). */
+  async open(): Promise<void> {
+    const raw = (await this.storage.readRawObject(MANIFEST_PATH)) as SegmentManifest | null
+    if (raw) {
+      if (raw.version !== 1) {
+        throw new Error(
+          `[GenerationSegments] manifest version ${String(raw.version)} is newer than this ` +
+            `engine understands — refusing to serve partial history. Upgrade the engine.`
+        )
+      }
+      this.manifest = raw
+    }
+  }
+
+  /** The packed tier's catalogue (ascending, immutable snapshot). */
+  segments(): readonly SegmentMeta[] {
+    return this.manifest.segments
+  }
+
+  /** D3's horizon marker: generations below this were reclaimed (bounded profiles only). */
+  compactedBelow(): number {
+    return this.manifest.compactedBelow
+  }
+
+  /** The covering sealed segment for `gen`, or null if it lives outside the packed tier. */
+  private coveringSegment(gen: number): SegmentMeta | null {
+    // Manifest is ascending and ranges never overlap — binary search.
+    const segs = this.manifest.segments
+    let lo = 0
+    let hi = segs.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      const s = segs[mid]
+      if (gen < s.firstGeneration) hi = mid - 1
+      else if (gen > s.lastGeneration) lo = mid + 1
+      else return s
+    }
+    return null
+  }
+
+  /** True when `gen` is packed (readable from this tier). */
+  hasGeneration(gen: number): boolean {
+    return this.coveringSegment(gen) !== null
+  }
+
+  /**
+   * Fold consecutive generations into ONE new sealed segment + sidecar and
+   * append it to the manifest atomically. Caller guarantees: `gens` is
+   * ascending, contiguous with the packed tier (first = last packed + 1 when
+   * segments exist), and already durable in the live tier. Crash between the
+   * segment write and the caller's live-tier delete leaves a DUPLICATE
+   * representation — resolved live-tier-wins by the reader; never a gap.
+   */
+  async fold(gens: FoldGeneration[]): Promise<SegmentMeta> {
+    if (gens.length === 0) {
+      throw new Error('[GenerationSegments] fold() requires at least one generation')
+    }
+    for (let i = 1; i < gens.length; i++) {
+      if (gens[i].generation <= gens[i - 1].generation) {
+        throw new Error('[GenerationSegments] fold() input must be strictly ascending')
+      }
+    }
+    const last = this.manifest.segments[this.manifest.segments.length - 1]
+    if (last && gens[0].generation <= last.lastGeneration) {
+      throw new Error(
+        `[GenerationSegments] fold() overlaps the packed tier: ${gens[0].generation} ≤ ` +
+          `sealed ${last.lastGeneration} — segments are immutable, never rewritten`
+      )
+    }
+
+    const first = gens[0].generation
+    const file = segmentFileName(first)
+    const sidecar: SidecarIndex = { version: 1, generations: [], ids: {} }
+
+    // Encode all frames, tracking offsets for the sidecar.
+    const parts: Uint8Array[] = [MAGIC]
+    let offset = MAGIC.length
+    for (const g of gens) {
+      const payload = msgpackEncode([
+        g.generation,
+        g.timestamp,
+        g.delta,
+        g.records.map((r) => [r.kind === 'noun' ? 0 : 1, r.id, r.record]),
+        0 // flags: v1 = uncompressed
+      ])
+      const frame = new Uint8Array(FRAME_PREFIX_BYTES + payload.length)
+      const view = new DataView(frame.buffer)
+      view.setUint32(0, payload.length, true)
+      view.setUint32(4, crc32c(payload), true)
+      frame.set(payload, FRAME_PREFIX_BYTES)
+      sidecar.generations.push([g.generation, offset, frame.length])
+      for (const r of g.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(sidecar.ids[key] ??= []).push(g.generation)
+      }
+      parts.push(frame)
+      offset += frame.length
+    }
+    const total = parts.reduce((n, p) => n + p.length, 0)
+    const bytes = new Uint8Array(total)
+    let at = 0
+    for (const p of parts) {
+      bytes.set(p, at)
+      at += p.length
+    }
+
+    const meta: SegmentMeta = {
+      file,
+      firstGeneration: first,
+      lastGeneration: gens[gens.length - 1].generation,
+      frames: gens.length,
+      bytes: total,
+      checksum: crc32c(bytes)
+    }
+
+    // Durability order: segment + sidecar fsync'd BEFORE the manifest names
+    // them (a crash before the manifest = invisible orphan files, harmless);
+    // manifest last, atomically.
+    const segPath = `${SEGMENTS_PREFIX}/${file}`
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(first)}`
+    await this.storage.writeRawBytes(segPath, bytes)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(sidecar))
+    await this.storage.syncRawObjects([segPath, idxPath])
+    const next: SegmentManifest = {
+      ...this.manifest,
+      segments: [...this.manifest.segments, meta]
+    }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    this.sidecars.set(file, sidecar)
+    return meta
+  }
+
+  /** Load (or rebuild, loudly) a segment's sidecar. */
+  private async sidecarFor(meta: SegmentMeta): Promise<SidecarIndex> {
+    const cached = this.sidecars.get(meta.file)
+    if (cached) return cached
+    const idxPath = `${SEGMENTS_PREFIX}/${sidecarFileName(meta.firstGeneration)}`
+    const raw = await this.storage.readRawBytes(idxPath)
+    if (raw) {
+      try {
+        const idx = msgpackDecode(raw) as SidecarIndex
+        if (idx.version === 1) {
+          this.sidecars.set(meta.file, idx)
+          return idx
+        }
+      } catch {
+        // fall through to rebuild
+      }
+    }
+    // Sidecars are DERIVED: rebuild from the segment, loudly — never serve
+    // wrong offsets silently.
+    prodLog.warn(
+      `[GenerationSegments] sidecar for ${meta.file} missing or unreadable — rebuilding from the segment`
+    )
+    const rebuilt = await this.rebuildSidecar(meta)
+    await this.storage.writeRawBytes(idxPath, msgpackEncode(rebuilt))
+    this.sidecars.set(meta.file, rebuilt)
+    return rebuilt
+  }
+
+  /** One sequential read of the segment → a fresh sidecar. Verifies every frame CRC. */
+  private async rebuildSidecar(meta: SegmentMeta): Promise<SidecarIndex> {
+    const frames = await this.readAllFrames(meta)
+    const idx: SidecarIndex = { version: 1, generations: [], ids: {} }
+    for (const f of frames) {
+      idx.generations.push([f.generation, f.offset, f.frameLen])
+      for (const r of f.records) {
+        const key = `${r.kind === 'noun' ? 0 : 1}:${r.id}`
+        ;(idx.ids[key] ??= []).push(f.generation)
+      }
+    }
+    return idx
+  }
+
+  private decodeFrame(
+    payload: Uint8Array
+  ): { generation: number; timestamp: number; delta: unknown; records: FoldGeneration['records'] } {
+    const [generation, timestamp, delta, rawRecords] = msgpackDecode(payload) as [
+      number,
+      number,
+      unknown,
+      Array<[number, string, unknown]>,
+      number
+    ]
+    return {
+      generation,
+      timestamp,
+      delta,
+      records: rawRecords.map(([kindByte, id, record]) => ({
+        kind: kindByte === 0 ? ('noun' as const) : ('verb' as const),
+        id,
+        record
+      }))
+    }
+  }
+
+  private async readAllFrames(meta: SegmentMeta): Promise<
+    Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }>
+  > {
+    const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+    if (!bytes) {
+      throw new Error(
+        `[GenerationSegments] sealed segment ${meta.file} is MISSING — packed history is damaged; ` +
+          `refusing to continue silently`
+      )
+    }
+    const out: Array<ReturnType<GenerationSegmentStore['decodeFrame']> & { offset: number; frameLen: number }> = []
+    let at = MAGIC.length
+    const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength)
+    while (at + FRAME_PREFIX_BYTES <= bytes.length) {
+      const payloadLen = view.getUint32(at, true)
+      const crc = view.getUint32(at + 4, true)
+      const payload = bytes.subarray(at + FRAME_PREFIX_BYTES, at + FRAME_PREFIX_BYTES + payloadLen)
+      if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+        throw new Error(
+          `[GenerationSegments] frame CRC mismatch in ${meta.file} at offset ${at} — ` +
+            `packed history is damaged; refusing to serve it`
+        )
+      }
+      out.push({ ...this.decodeFrame(payload), offset: at, frameLen: FRAME_PREFIX_BYTES + payloadLen })
+      at += FRAME_PREFIX_BYTES + payloadLen
+    }
+    return out
+  }
+
+  /** Read one packed generation's frame via its sidecar offset (one ranged read). */
+  private async readFrame(
+    gen: number
+  ): Promise<ReturnType<GenerationSegmentStore['decodeFrame']> | null> {
+    const meta = this.coveringSegment(gen)
+    if (!meta) return null
+    const idx = await this.sidecarFor(meta)
+    // generations ascending → binary search.
+    const gens = idx.generations
+    let lo = 0
+    let hi = gens.length - 1
+    while (lo <= hi) {
+      const mid = (lo + hi) >> 1
+      if (gens[mid][0] < gen) lo = mid + 1
+      else if (gens[mid][0] > gen) hi = mid - 1
+      else {
+        const [, offset, frameLen] = gens[mid]
+        const bytes = await this.storage.readRawBytes(`${SEGMENTS_PREFIX}/${meta.file}`)
+        if (!bytes) {
+          throw new Error(`[GenerationSegments] sealed segment ${meta.file} is MISSING`)
+        }
+        const frame = bytes.subarray(offset, offset + frameLen)
+        const view = new DataView(frame.buffer, frame.byteOffset, frame.byteLength)
+        const payloadLen = view.getUint32(0, true)
+        const crc = view.getUint32(4, true)
+        const payload = frame.subarray(FRAME_PREFIX_BYTES, FRAME_PREFIX_BYTES + payloadLen)
+        if (payload.length !== payloadLen || crc32c(payload) !== crc) {
+          throw new Error(
+            `[GenerationSegments] frame CRC mismatch for generation ${gen} in ${meta.file} — ` +
+              `packed history is damaged; refusing to serve it`
+          )
+        }
+        return this.decodeFrame(payload)
+      }
+    }
+    // In the covering range but not present: the packed tier is dense by
+    // construction (fold packs every generation it is handed, including
+    // record-less ones) — absence inside a sealed range is damage.
+    throw new Error(
+      `[GenerationSegments] generation ${gen} is inside sealed segment ${meta.file}'s declared ` +
+        `range but has no frame — packed history is damaged`
+    )
+  }
+
+  /** The packed tier's delta for `gen` (null = not packed). */
+  async readDelta(gen: number): Promise<{ delta: unknown; timestamp: number } | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? { delta: frame.delta, timestamp: frame.timestamp } : null
+  }
+
+  /** The packed tier's full record-set for `gen` (null = not packed). */
+  async readRecords(gen: number): Promise<FoldGeneration['records'] | null> {
+    const frame = await this.readFrame(gen)
+    return frame ? frame.records : null
+  }
+
+  /** One packed before-image (null = not packed OR no record for the id in that generation). */
+  async readRecord(gen: number, kind: 'noun' | 'verb', id: string): Promise<unknown | null> {
+    const frame = await this.readFrame(gen)
+    if (!frame) return null
+    const hit = frame.records.find((r) => r.kind === kind && r.id === id)
+    return hit ? hit.record : null
+  }
+
+  /**
+   * D3 reclaim: drop WHOLE segments whose lastGeneration < `belowGeneration`
+   * and bump `compactedBelow`. Partial segments are never dropped — the
+   * boundary waits. NEVER called under the archival profile (the caller
+   * enforces retention semantics; this method only executes boundary drops).
+   */
+  async dropSegmentsBelow(belowGeneration: number): Promise<{ dropped: number; compactedBelow: number }> {
+    const keep: SegmentMeta[] = []
+    const drop: SegmentMeta[] = []
+    for (const s of this.manifest.segments) {
+      ;(s.lastGeneration < belowGeneration ? drop : keep).push(s)
+    }
+    if (drop.length === 0) {
+      return { dropped: 0, compactedBelow: this.manifest.compactedBelow }
+    }
+    const compactedBelow = Math.max(
+      this.manifest.compactedBelow,
+      drop[drop.length - 1].lastGeneration + 1
+    )
+    // Manifest first (the drop is authoritative once named), then bytes —
+    // a crash between leaves orphan segment files invisible to the manifest,
+    // harmless and re-collectable.
+    const next: SegmentManifest = { ...this.manifest, compactedBelow, segments: keep }
+    await this.storage.writeRawObject(MANIFEST_PATH, next)
+    await this.storage.syncRawObjects([MANIFEST_PATH])
+    this.manifest = next
+    for (const s of drop) {
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${s.file}`)
+      await this.storage.deleteRawObject(`${SEGMENTS_PREFIX}/${sidecarFileName(s.firstGeneration)}`)
+      this.sidecars.delete(s.file)
+    }
+    return { dropped: drop.length, compactedBelow }
+  }
+
+  /**
+   * D8 rider — the packed portion of `generationDigest(g)`: a deterministic
+   * crc32c chain over sealed-segment checksums fully below `g`, plus the
+   * frame CRC of `g`'s own frame when `g` is mid-segment. O(segments), not
+   * O(generations); identical history ⇒ identical digest on any machine.
+   * The live-tier portion is composed by the caller.
+   */
+  async digestThroughPacked(g: number): Promise<number | null> {
+    let digest = 0
+    let covered = false
+    for (const s of this.manifest.segments) {
+      if (s.lastGeneration <= g) {
+        digest = crc32c(new TextEncoder().encode(`${digest}:${s.checksum}`))
+        if (s.lastGeneration === g) covered = true
+      } else if (s.firstGeneration <= g) {
+        // g is mid-segment: chain the partial prefix via g's frame CRC.
+        const frame = await this.readFrame(g)
+        if (frame === null) return null
+        const idx = await this.sidecarFor(s)
+        const upTo = idx.generations.filter(([gen]) => gen <= g)
+        for (const [gen, offset, frameLen] of upTo) {
+          digest = crc32c(new TextEncoder().encode(`${digest}:${gen}:${offset}:${frameLen}`))
+        }
+        covered = true
+        break
+      }
+    }
+    return covered || this.manifest.segments.length > 0 ? digest : null
+  }
+}

src/db/generationStore.ts

@@ -46,6 +46,8 @@ import type {
   TxLogEntry
 } from './types.js'
 import { FactLog, storageSupportsFactLog, type CommitFact, type FactOp } from './factLog.js'
+import { GenerationSegmentStore, type FoldGeneration } from './generationSegments.js'
+import { crc32c } from '../utils/crc32c.js'
 
 /**
  * The byte-identical before-images of every id a commit touches, read UNDER
@@ -257,6 +259,30 @@ export class GenerationStore {
    */
   private deltaCacheMax = 4096
 
+  /**
+   * Running total of on-disk history bytes across committed generations —
+   * `null` until {@link historyBytes} pays its one seeding walk. Maintained
+   * incrementally at commit/reclaim so the adaptive retention check on every
+   * flush() is O(1), never a tail re-walk. Never updated by cache re-reads
+   * ({@link setDelta} inserts are cache population, not new history).
+   */
+  private historyBytesTotal: number | null = null
+
+  /**
+   * The packed tier (D1+D3): sealed segments holding folded cold
+   * generations. Null until {@link open} wires it (and on storage adapters
+   * without raw-byte primitives — the live tier then carries everything,
+   * exactly as before the packed tier existed).
+   */
+  private segments: GenerationSegmentStore | null = null
+
+  /**
+   * Live-tier window: generations newer than `committed - REPACK_LIVE_WINDOW`
+   * are never folded — the hot tail stays in the per-generation layout the
+   * write path owns. Matches the resident chain window's scale.
+   */
+  static readonly REPACK_LIVE_WINDOW = 1024
+
   /**
    * Model-B per-write group-commit — the in-memory PENDING tier.
    *
@@ -424,6 +450,33 @@ export class GenerationStore {
       this.factLog = null
     }
 
+    // PACKED TIER (D1+D3): same capability gate as the fact log. Opening
+    // reads ONE manifest — never a listing of the packed backlog — and seeds
+    // committedRanges with the sealed ranges so packed generations resolve
+    // exactly like live ones.
+    if (storageSupportsFactLog(this.storage)) {
+      this.segments = new GenerationSegmentStore(this.storage)
+      await this.segments.open()
+      const packedRanges = this.segments
+        .segments()
+        .map((s): [number, number] => [s.firstGeneration, Math.min(s.lastGeneration, this.committed)])
+        .filter(([lo, hi]) => lo <= hi)
+      if (packedRanges.length > 0) {
+        // Merge packed (older) + live (newer) interval sets — both ascending;
+        // coalesce adjacency so range arithmetic stays interval-exact.
+        const merged: Array<[number, number]> = []
+        for (const r of [...packedRanges, ...this.committedRanges].sort((a, b) => a[0] - b[0])) {
+          const last = merged[merged.length - 1]
+          if (last && r[0] <= last[1] + 1) last[1] = Math.max(last[1], r[1])
+          else merged.push([r[0], r[1]])
+        }
+        this.committedRanges = merged
+      }
+      this.horizonGen = Math.max(this.horizonGen, this.segments.compactedBelow() - 1)
+    } else {
+      this.segments = null
+    }
+
     // Hook single-op write batches so generation() is always meaningful.
     // Suppressed while a transact batch executes (the batch is ONE generation).
     if (!options?.readOnly) {
@@ -483,6 +536,85 @@ export class GenerationStore {
     return this.horizonGen
   }
 
+  /**
+   * @description Read-only history footprint for fleet audits: how much
+   * generational history this store holds on disk. `bytes` pays (and seeds)
+   * the one-time {@link historyBytes} walk on first call — subsequent calls
+   * are O(1). The oldest/newest timestamps come from those generations'
+   * deltas (cache-bounded reads).
+   * @returns Counts, bytes, generation range, and the compaction horizon.
+   */
+  /**
+   * @description D8 (gate-to-generation provenance): a deterministic content
+   * digest of the generation log THROUGH `g` — identical history ⇒ identical
+   * digest on any machine; any divergence (different records, different
+   * order, reclaimed range) ⇒ different digest. Composed from the packed
+   * tier's sealed-segment checksum chain (O(segments)) plus the live tier's
+   * per-generation delta digests (O(live window at most)). Release gates pin
+   * {generation, digest} and verify both at execution time.
+   * @param g - The generation to digest through (≤ committed).
+   * @returns A hex digest string, stable across reopen and repacking states
+   *   ONLY for fully-packed prefixes — repacking changes representation, so
+   *   the composed digest is defined over CONTENT: live-tier gens hash their
+   *   delta + record ids, packed gens hash via frame CRCs. A gate should pin
+   *   after a repack pass for long-term stability, or re-pin on repack.
+   */
+  async generationDigest(g: number): Promise<string> {
+    if (!Number.isInteger(g) || g < 1 || g > this.committed) {
+      throw new RangeError(
+        `generationDigest(): generation ${g} is out of range [1, ${this.committed}]`
+      )
+    }
+    if (g <= this.horizonGen) {
+      throw new GenerationCompactedError(g, this.horizonGen)
+    }
+    let digest = 0
+    const enc = new TextEncoder()
+    if (this.segments) {
+      const packed = await this.segments.digestThroughPacked(g)
+      if (packed !== null) digest = packed
+    }
+    // Live-tier composition: every committed gen ≤ g not covered by a sealed
+    // segment hashes its delta content in ascending order.
+    for (const gen of this.committedGensAsc()) {
+      if (gen > g) break
+      if (this.segments?.hasGeneration(gen)) continue
+      const delta = await this.getDelta(gen)
+      digest = crc32c(
+        enc.encode(
+          `${digest}:${gen}:${delta.timestamp}:${[...delta.nouns].sort().join(',')}:${[...delta.verbs].sort().join(',')}`
+        )
+      )
+    }
+    return digest.toString(16).padStart(8, '0')
+  }
+
+  async historyStats(): Promise<{
+    generations: number
+    bytes: number
+    oldestGeneration: number | null
+    newestGeneration: number | null
+    oldestTimestamp: number | null
+    newestTimestamp: number | null
+    horizon: number
+  }> {
+    let oldest: number | null = null
+    let newest: number | null = null
+    for (const gen of this.committedGensAsc()) {
+      if (oldest === null) oldest = gen
+      newest = gen
+    }
+    return {
+      generations: this.committedCount(),
+      bytes: await this.historyBytes(),
+      oldestGeneration: oldest,
+      newestGeneration: newest,
+      oldestTimestamp: oldest !== null ? (await this.getDelta(oldest)).timestamp : null,
+      newestTimestamp: newest !== null ? (await this.getDelta(newest)).timestamp : null,
+      horizon: this.horizonGen
+    }
+  }
+
   /**
    * @description Read one generation's persisted before-image records — the
    * compaction fallback for generations written before deltas carried
@@ -495,14 +627,17 @@ export class GenerationStore {
     try {
       paths = await this.storage.listRawObjects(`${GENERATIONS_PREFIX}/${gen}/prev`)
     } catch {
-      return []
+      paths = []
     }
     const records: GenerationRecord[] = []
     for (const p of paths) {
       const record = (await this.storage.readRawObject(p)) as GenerationRecord | null
       if (record) records.push(record)
     }
-    return records
+    if (records.length > 0) return records
+    // Two-tier: folded generations serve their record-set from the segment.
+    const packed = await this.segments?.readRecords(gen)
+    return packed ? (packed.map((r) => r.record) as GenerationRecord[]) : []
   }
 
   /**
@@ -849,6 +984,9 @@ export class GenerationStore {
           timestamp,
           bytes: delta.bytes ?? 0
         })
+        if (this.historyBytesTotal !== null) {
+          this.historyBytesTotal += delta.bytes ?? 0
+        }
         this.extendChains(gen, nouns, verbs)
         const logEntry: TxLogEntry = { generation: gen, timestamp, ...(args.meta && { meta: args.meta }) }
         await this.storage.appendTxLogLine(JSON.stringify(logEntry))
@@ -1302,6 +1440,9 @@ export class GenerationStore {
           timestamp: buf.timestamp,
           bytes: genBytes.get(gen) ?? 0
         })
+        if (this.historyBytesTotal !== null) {
+          this.historyBytesTotal += genBytes.get(gen) ?? 0
+        }
         this.pendingBuffer.delete(gen)
       }
       this.pendingGens = []
@@ -1734,9 +1875,15 @@ export class GenerationStore {
     if (pending) {
       return (kind === 'noun' ? pending.nouns : pending.verbs).get(id) ?? null
     }
-    return (await this.storage.readRawObject(
+    const live = (await this.storage.readRawObject(
       `${GENERATIONS_PREFIX}/${gen}/prev/${id}.json`
     )) as GenerationRecord | null
+    if (live) return live
+    // Two-tier: the packed tier serves folded generations (live-tier-wins).
+    if (this.segments?.hasGeneration(gen)) {
+      return (await this.segments.readRecord(gen, kind, id)) as GenerationRecord | null
+    }
+    return null
   }
 
   /**
@@ -2083,6 +2230,21 @@ export class GenerationStore {
       `${GENERATIONS_PREFIX}/${gen}/tx.json`
     )) as GenerationDelta | null
     if (delta === null) {
+      // Two-tier read (D1+D3): not in the live tier → the packed tier.
+      // Live-tier-wins ordering (a crash mid-fold leaves a duplicate, never
+      // a gap), so the segment lookup runs only after the live miss.
+      const packed = await this.segments?.readDelta(gen)
+      if (packed) {
+        const d = packed.delta as GenerationDelta
+        const entry = {
+          nouns: new Set(d.nouns),
+          verbs: new Set(d.verbs),
+          timestamp: packed.timestamp,
+          bytes: d.bytes ?? 0
+        }
+        this.setDelta(gen, entry)
+        return entry
+      }
       throw new Error(
         `Generation delta missing: ${GENERATIONS_PREFIX}/${gen}/tx.json ` +
           `(store corrupted or records removed outside compactHistory())`
@@ -2122,17 +2284,26 @@ export class GenerationStore {
   /**
    * @description Total serialized bytes of the ON-DISK generational history —
    * the sum of every committed generation's recorded `bytes`. Backs the
-   * `maxBytes` and adaptive retention caps. Reads each committed generation's
+   * `maxBytes` and adaptive retention caps. O(1) after the first call: the
-   * delta (cached; a re-read only for cache-evicted ones) — O(committed
+   * total is computed by ONE walk over committed deltas, then maintained
-   * generations), bounded by retention itself and invoked only at compaction
+   * incrementally at every commit (+bytes) and reclaim (−bytes) and dropped on
-   * time. Pending (un-flushed) generations are excluded (they are not on disk).
+   * a wholesale state replacement (restore). Without the running total, the
+   * adaptive auto-compaction on every flush() re-walked the ENTIRE history —
+   * O(committed generations) file reads per flush past the delta-cache bound —
+   * which is how a 70k-generation production brain turned every write into a
+   * full-tail scan (SELF-GENERATIONS-GROWTH). Pending (un-flushed) generations
+   * are excluded (they are not on disk).
    * @returns The total on-disk history byte count.
    */
   async historyBytes(): Promise<number> {
+    if (this.historyBytesTotal !== null) {
+      return this.historyBytesTotal
+    }
     let total = 0
     for (const gen of this.committedGensAsc()) {
       total += (await this.getDelta(gen)).bytes
     }
+    this.historyBytesTotal = total
     return total
   }
 
@@ -2155,6 +2326,94 @@ export class GenerationStore {
    * @param options - Retention caps (see {@link CompactHistoryOptions}).
    * @returns Count of removed record-sets and the new horizon.
    */
+  /**
+   * @description The REPACKER (D1+D3+repacking): fold cold live-tier
+   * generations into sealed segments — re-representation, never deletion.
+   * Every record and delta stays readable (asOf/chains unchanged); the
+   * per-generation directories are deleted only AFTER their segment is
+   * durable (crash between = duplicate representation, resolved
+   * live-tier-wins by every reader; never a gap). This is the transform that
+   * takes a 70k-file history to tens of segment files, and the ONLY history
+   * transform permitted under the archival profile.
+   *
+   * Folds oldest-first, contiguous from the packed boundary, in batches, and
+   * stops at the live window ({@link GenerationStore.REPACK_LIVE_WINDOW})
+   * or when `timeBudgetMs` is spent — an early stop is a consistent prefix;
+   * the next pass resumes.
+   */
+  async repackHistory(options?: { timeBudgetMs?: number; batchGenerations?: number }): Promise<{
+    foldedGenerations: number
+    segmentsCreated: number
+  }> {
+    if (!this.segments) return { foldedGenerations: 0, segmentsCreated: 0 }
+    const segments = this.segments
+    return this.withMutex(async () => {
+      const deadline =
+        options?.timeBudgetMs !== undefined ? Date.now() + options.timeBudgetMs : undefined
+      const batchSize = options?.batchGenerations ?? 512
+      const coldCeiling = this.committed - GenerationStore.REPACK_LIVE_WINDOW
+      const packedThrough =
+        segments.segments().length > 0
+          ? segments.segments()[segments.segments().length - 1].lastGeneration
+          : 0
+
+      // Cold, unpacked, committed generations — ascending, contiguous scan.
+      const eligible: number[] = []
+      for (const gen of this.committedGensAsc()) {
+        if (gen > coldCeiling) break
+        if (gen <= packedThrough) continue // already packed (dup fold barred)
+        if (this.pendingBuffer.has(gen)) continue // un-flushed = live by definition
+        eligible.push(gen)
+      }
+
+      let folded = 0
+      let segmentsCreated = 0
+      for (let i = 0; i < eligible.length; i += batchSize) {
+        if (deadline !== undefined && Date.now() >= deadline) break
+        const batch = eligible.slice(i, i + batchSize)
+        const foldInput: FoldGeneration[] = []
+        for (const gen of batch) {
+          const delta = (await this.storage.readRawObject(
+            `${GENERATIONS_PREFIX}/${gen}/tx.json`
+          )) as GenerationDelta | null
+          if (delta === null) {
+            // Already folded by a prior crashed pass whose dirs were removed,
+            // or damage — getDelta's two-tier read decides which, loudly,
+            // when someone asks. Skip; never fold a generation we cannot read.
+            continue
+          }
+          const records: FoldGeneration['records'] = []
+          for (const [kind, ids] of [
+            ['noun', delta.nouns] as const,
+            ['verb', delta.verbs] as const
+          ]) {
+            for (const id of ids) {
+              const record = await this.storage.readRawObject(
+                `${GENERATIONS_PREFIX}/${gen}/prev/${id}.json`
+              )
+              if (record) records.push({ kind, id, record })
+            }
+          }
+          foldInput.push({ generation: gen, timestamp: delta.timestamp, delta, records })
+        }
+        if (foldInput.length === 0) continue
+        await segments.fold(foldInput)
+        segmentsCreated++
+        // Segment + manifest durable → the live copies retire.
+        for (const g of foldInput) {
+          await this.storage.removeRawPrefix(`${GENERATIONS_PREFIX}/${g.generation}`)
+        }
+        folded += foldInput.length
+      }
+      if (folded > 0) {
+        prodLog.info(
+          `[GenerationStore] repacked ${folded} cold generation(s) into ${segmentsCreated} segment(s) — history preserved, file count reduced`
+        )
+      }
+      return { foldedGenerations: folded, segmentsCreated }
+    })
+  }
+
   async compact(options?: CompactHistoryOptions): Promise<CompactHistoryResult> {
     return this.withMutex(async () => {
       const minPinned = this.minPinnedGeneration()
@@ -2162,6 +2421,11 @@ export class GenerationStore {
       const maxAge = options?.maxAge
       const maxBytes = options?.maxBytes
       const ageCutoff = maxAge !== undefined ? Date.now() - maxAge : undefined
+      // Bounded maintenance pass (8.9.0): stop reclaiming once the budget is
+      // spent. Safe mid-loop — reclamation is oldest-first, so an early stop
+      // leaves a consistent contiguous prefix and the next pass resumes.
+      const deadline =
+        options?.timeBudgetMs !== undefined ? Date.now() + options.timeBudgetMs : undefined
       const noCaps =
         maxGenerations === undefined && maxAge === undefined && maxBytes === undefined
 
@@ -2175,6 +2439,7 @@ export class GenerationStore {
       for (const gen of [...this.committedGensAsc()]) {
         // Pins are always exempt: never reclaim a generation a live pin needs.
         if (gen > minPinned) break // committedGensAsc ascending → nothing newer is eligible either
+        if (deadline !== undefined && Date.now() >= deadline) break // budget spent — resume next pass
         const delta = await this.getDelta(gen)
         if (!noCaps) {
           const violatesCount = maxGenerations !== undefined && remainingCount > maxGenerations
@@ -2206,6 +2471,9 @@ export class GenerationStore {
 
         await this.storage.removeRawPrefix(`${GENERATIONS_PREFIX}/${gen}`)
         this.deltaCache.delete(gen)
+        if (this.historyBytesTotal !== null) {
+          this.historyBytesTotal -= delta.bytes
+        }
 
         // AFTER the record-set is gone (over-count-only crash ordering):
         // release its history references and reclaim any blob left with zero
@@ -2237,6 +2505,16 @@ export class GenerationStore {
         // Reclaimed generations leave the per-id chains stale → rebuild on next read.
         this.invalidateChains()
         this.horizonGen = Math.max(this.horizonGen, highestRemoved)
+        // Packed-tier reclaim (D3): a packed generation's bytes live in a
+        // sealed segment — removeRawPrefix above was a no-op for it. Drop
+        // WHOLE segments now fully below the horizon; a partially-reclaimed
+        // segment keeps its bytes until the boundary passes it (the frozen
+        // partial-segments-wait rule; logical reclamation above still holds —
+        // the generations left committedRanges and asOf below the horizon
+        // throws regardless).
+        if (this.segments) {
+          await this.segments.dropSegmentsBelow(this.horizonGen + 1)
+        }
         const manifest: GenerationManifest = {
           version: 1,
           generation: this.committed,
@@ -2266,6 +2544,9 @@ export class GenerationStore {
   async reopenAfterRestore(floorGeneration: number): Promise<void> {
     await this.withMutex(async () => {
       this.deltaCache.clear()
+      // The running history-byte total describes the REPLACED store — drop it;
+      // the next historyBytes() re-seeds with one walk over the new state.
+      this.historyBytesTotal = null
       // A wholesale state replacement invalidates any buffered single-op
       // history — discard the pending tier (its live writes are gone with the
       // replaced store).

src/db/types.ts

@@ -116,6 +116,16 @@ export interface TransactOptions {
    * record is staged.
    */
   ifAtGeneration?: number
+  /**
+   * Budget (ms) for the atomic apply phase. When omitted, the budget SCALES
+   * with the batch: `max(30 000, opCount × 2 000)` — production imports on
+   * network-attached disks measure ~2 s per operation, so a flat 30 s budget
+   * silently capped honest bulk work at ~15 operations. A tripped budget
+   * rolls the whole batch back and throws a retryable
+   * `TransactionTimeoutError` naming the operation it stopped at, the batch
+   * size, and the elapsed/budget times.
+   */
+  timeoutMs?: number
 }
 
 /**
@@ -166,6 +176,15 @@ export interface CompactHistoryOptions {
    * of each surviving generation's serialized record set (`GenerationDelta.bytes`).
    */
   maxBytes?: number
+  /**
+   * Stop reclaiming after this many milliseconds even if caps are still
+   * exceeded (8.9.0). Compaction is maintenance — a bounded pass keeps
+   * `close()` (and any explicit maintenance window) from stalling on a large
+   * backlog; the next pass resumes where this one stopped (reclamation is
+   * oldest-first, so an early stop is always a consistent prefix). Unset =
+   * run to completion.
+   */
+  timeBudgetMs?: number
 }
 
 /**
@@ -183,6 +202,36 @@ export interface CompactHistoryResult {
   horizon: number
 }
 
+/**
+ * @description Result of `brain.historyStats()` — the read-only generational
+ * history footprint, for fleet audits and ops doors. A pool operator runs this
+ * per brain to size retention exposure (how much MVCC history each brain
+ * carries and under which policy) without touching any data.
+ */
+export interface HistoryStats {
+  /** Committed generation record-sets currently on disk. */
+  generations: number
+  /** Total on-disk history bytes across those record-sets. */
+  bytes: number
+  /** Oldest committed generation still on disk (null when history is empty). */
+  oldestGeneration: number | null
+  /** Newest committed generation (null when history is empty). */
+  newestGeneration: number | null
+  /** Commit timestamp (ms) of the oldest on-disk generation. */
+  oldestTimestamp: number | null
+  /** Commit timestamp (ms) of the newest on-disk generation. */
+  newestTimestamp: number | null
+  /** Compaction horizon — generations below it were reclaimed. */
+  horizon: number
+  /** The effective retention mode this brain runs under. */
+  retentionMode: 'all' | 'adaptive' | 'explicit'
+  /**
+   * The adaptive byte budget in force (coordinator-driven or the local
+   * free-memory probe); null under 'all' or explicit caps.
+   */
+  effectiveBudgetBytes: number | null
+}
+
 // ============================================================================
 // Db surfaces
 // ============================================================================

src/graph/graphAdjacencyIndex.ts

@@ -282,6 +282,16 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
       }
 
       hasMore = result.hasMore
+      if (hasMore && (!result.nextCursor || result.nextCursor === cursor)) {
+        // A stalled cursor with hasMore=true would re-read the same page
+        // forever — a silent full-CPU loop at cold open. Abort loudly; a
+        // graph read failing beats a process that spins without a log line.
+        throw new Error(
+          `GraphAdjacencyIndex: verb walk stalled after ${count} verbs — storage returned ` +
+            `hasMore=true with ${result.nextCursor ? 'a non-advancing' : 'no'} cursor. ` +
+            `Aborting the cold-load; run brain.repairIndex() if this persists.`
+        )
+      }
       cursor = result.nextCursor
     }
 

src/graph/graphAudit.ts

@@ -0,0 +1,214 @@
+/**
+ * @module graph/graphAudit
+ * @description Read-only graph-truth audit — the graph sibling of `repairIndex()`'s
+ * diagnosis half. Verifies three layers against each other without mutating anything:
+ *
+ *   1. CANONICAL verb records (the storage walk — the source of truth)
+ *   2. the RELATIONSHIP READ PATH (`related()` with all visibility tiers — exactly
+ *      what application reads like a VFS `readdir` consult)
+ *   3. ENTITY ENDPOINTS (does each verb's source/target still exist?)
+ *
+ * and classifies every discrepancy into the three failure families production
+ * incidents have shown:
+ *
+ *   - `missingFromReads` — a canonical verb record the read path does NOT return
+ *     for its source: PRESENT BUT INVISIBLE (adjacency/membership staleness).
+ *   - `danglingEndpoints` — a canonical verb whose endpoint entity is gone:
+ *     the SCAR class (write-path loss / partial delete).
+ *   - `readOnlyVerbIds` — the read path returns an edge with NO canonical
+ *     record: GHOST edges (stale index entries).
+ *
+ * `visibilityHiddenCount` is reported separately: an internal/system edge that is
+ * indexed and present but hidden from DEFAULT reads is working as designed — the
+ * audit reads with all tiers included so design-hiding is never misclassified as
+ * index loss.
+ *
+ * Full counts are always exact; only the example LISTS are capped (`maxExamples`)
+ * — a capped report says so via `truncatedExamples`, never silently.
+ */
+
+import { prodLog } from '../utils/logger.js'
+
+/** One discrepant relationship, identified fully enough to inspect by hand. */
+export interface GraphAuditDiscrepancy {
+  verbId: string
+  from: string
+  to: string
+  type: string
+}
+
+export interface GraphAuditReport {
+  /** True iff every discrepancy count is zero — `related()` returns canonical truth. */
+  coherent: boolean
+  verbsInCanonical: number
+  entitiesInCanonical: number
+  /** Distinct source entities whose read path was actually consulted (coverage honesty). */
+  sourcesChecked: number
+
+  /** PRESENT BUT INVISIBLE: canonical records the read path omits. */
+  missingFromReadsCount: number
+  missingFromReads: GraphAuditDiscrepancy[]
+
+  /** SCAR CLASS: canonical verbs with a missing endpoint entity. */
+  danglingEndpointsCount: number
+  danglingEndpoints: Array<GraphAuditDiscrepancy & { missingEnd: 'from' | 'to' | 'both' }>
+
+  /** GHOST EDGES: read-path verb ids with no canonical record. */
+  readOnlyCount: number
+  readOnlyVerbIds: string[]
+
+  /** Canonical verbs hidden from DEFAULT reads by design (internal/system visibility). */
+  visibilityHiddenCount: number
+
+  /** Example lists above were capped at maxExamples; counts remain exact. */
+  truncatedExamples: boolean
+  durationMs: number
+}
+
+/** A canonical verb record, as the audit needs it. */
+export interface AuditVerbRecord {
+  id: string
+  type: string
+  sourceId: string
+  targetId: string
+  visibility?: string
+}
+
+/** The seams the audit runs over — injected so the walk is testable in isolation. */
+export interface GraphAuditDeps {
+  /** Stream every canonical entity id (id-only; no per-entity reads needed). */
+  eachNounId(consume: (id: string) => void): Promise<void>
+  /** Stream every canonical verb record. */
+  eachVerb(consume: (verb: AuditVerbRecord) => void): Promise<void>
+  /**
+   * The END-TO-END relationship read for one source, ALL visibility tiers
+   * included — must be the same path application reads consult.
+   */
+  readRelationsFrom(sourceId: string): Promise<Array<{ id: string }>>
+}
+
+export interface GraphAuditOptions {
+  /** Cap on entries per example list (counts stay exact). Default 100. */
+  maxExamples?: number
+}
+
+export async function runGraphAudit(
+  deps: GraphAuditDeps,
+  options: GraphAuditOptions = {}
+): Promise<GraphAuditReport> {
+  const maxExamples = options.maxExamples ?? 100
+  const started = Date.now()
+
+  // 1. Canonical entity ids — endpoint existence oracle.
+  const entityIds = new Set<string>()
+  await deps.eachNounId((id) => entityIds.add(id))
+
+  // 2. Canonical verb walk: group by source, check endpoints, note visibility.
+  const canonicalVerbIds = new Set<string>()
+  const bySource = new Map<string, AuditVerbRecord[]>()
+  let verbsInCanonical = 0
+  let visibilityHiddenCount = 0
+  let danglingEndpointsCount = 0
+  const danglingEndpoints: GraphAuditReport['danglingEndpoints'] = []
+
+  await deps.eachVerb((verb) => {
+    verbsInCanonical++
+    canonicalVerbIds.add(verb.id)
+    const list = bySource.get(verb.sourceId)
+    if (list) list.push(verb)
+    else bySource.set(verb.sourceId, [verb])
+
+    if (verb.visibility === 'internal' || verb.visibility === 'system') {
+      visibilityHiddenCount++
+    }
+
+    const fromMissing = !entityIds.has(verb.sourceId)
+    const toMissing = !entityIds.has(verb.targetId)
+    if (fromMissing || toMissing) {
+      danglingEndpointsCount++
+      if (danglingEndpoints.length < maxExamples) {
+        danglingEndpoints.push({
+          verbId: verb.id,
+          from: verb.sourceId,
+          to: verb.targetId,
+          type: verb.type,
+          missingEnd: fromMissing && toMissing ? 'both' : fromMissing ? 'from' : 'to'
+        })
+      }
+    }
+  })
+
+  // 3. Per-source read-path comparison. A verb must be returned by the read
+  //    path of ITS OWN source — the exact consult a readdir/traversal makes.
+  let missingFromReadsCount = 0
+  const missingFromReads: GraphAuditDiscrepancy[] = []
+  let readOnlyCount = 0
+  const readOnlyVerbIds: string[] = []
+  const readOnlySeen = new Set<string>()
+
+  for (const [sourceId, verbs] of bySource) {
+    const readIds = new Set((await deps.readRelationsFrom(sourceId)).map((r) => r.id))
+
+    for (const verb of verbs) {
+      if (!readIds.has(verb.id)) {
+        missingFromReadsCount++
+        if (missingFromReads.length < maxExamples) {
+          missingFromReads.push({
+            verbId: verb.id,
+            from: verb.sourceId,
+            to: verb.targetId,
+            type: verb.type
+          })
+        }
+      }
+    }
+
+    for (const readId of readIds) {
+      if (!canonicalVerbIds.has(readId) && !readOnlySeen.has(readId)) {
+        readOnlySeen.add(readId)
+        readOnlyCount++
+        if (readOnlyVerbIds.length < maxExamples) {
+          readOnlyVerbIds.push(readId)
+        }
+      }
+    }
+  }
+
+  const coherent =
+    missingFromReadsCount === 0 && danglingEndpointsCount === 0 && readOnlyCount === 0
+
+  const report: GraphAuditReport = {
+    coherent,
+    verbsInCanonical,
+    entitiesInCanonical: entityIds.size,
+    sourcesChecked: bySource.size,
+    missingFromReadsCount,
+    missingFromReads,
+    danglingEndpointsCount,
+    danglingEndpoints,
+    readOnlyCount,
+    readOnlyVerbIds,
+    visibilityHiddenCount,
+    truncatedExamples:
+      missingFromReadsCount > missingFromReads.length ||
+      danglingEndpointsCount > danglingEndpoints.length ||
+      readOnlyCount > readOnlyVerbIds.length,
+    durationMs: Date.now() - started
+  }
+
+  if (coherent) {
+    prodLog.info(
+      `[GraphAudit] coherent: ${verbsInCanonical} verbs across ${bySource.size} sources — ` +
+        `the read path returns canonical truth (${report.durationMs}ms)`
+    )
+  } else {
+    prodLog.warn(
+      `[GraphAudit] INCOHERENT: ${missingFromReadsCount} present-but-invisible, ` +
+        `${danglingEndpointsCount} dangling-endpoint, ${readOnlyCount} ghost ` +
+        `(of ${verbsInCanonical} canonical verbs, ${bySource.size} sources, ` +
+        `${visibilityHiddenCount} visibility-hidden by design) — ${report.durationMs}ms`
+    )
+  }
+
+  return report
+}

src/hnsw/hnswIndex.ts

@@ -54,6 +54,9 @@ export class HnswFlushError extends Error {
  * acceleration provider).
  */
 export class JsHnswVectorIndex implements VectorIndexProvider {
+  /** Self-identifies as the built-in JS fallback engine — see {@link VectorIndexProvider.name}. */
+  readonly name = 'hnsw-js'
+
   private nouns: Map<string, HNSWNoun> = new Map()
   /**
    * Reverse adjacency: `target id → (level → set of node ids that link TO target)`.

src/import/BackgroundDeduplicator.ts

@@ -41,6 +41,14 @@ export interface DeduplicationStats {
  * - Import-scoped deduplication (no cross-contamination)
  * - 3-tier strategy (ID → Name → Similarity)
  * - Uses existing indexes (EntityIdMapper, MetadataIndexManager, TypeAware HNSW)
+ *
+ * Lifecycle: ONE instance per brain, owned by Brainy (getBackgroundDeduplicator)
+ * so the debounce genuinely spans imports and brain.close() cancels pending
+ * work via cancelPending() — this pass merge-DELETES duplicate entities, so it
+ * must never fire against a closed brain. The enableDeduplication gate lives
+ * at the scheduling call site (ImportCoordinator); scheduleDedup itself is
+ * unconditional. The timer is unref'd — a pending pass never holds the
+ * process open.
  */
 export class BackgroundDeduplicator {
   private brain: Brainy
@@ -67,12 +75,15 @@ export class BackgroundDeduplicator {
       clearTimeout(this.debounceTimer)
     }
 
-    // Schedule for 5 minutes from now
+    // Schedule for 5 minutes from now. unref'd: a pending dedup pass must
+    // never hold the process open (exit-hang class) — if the process exits
+    // first, the pass simply never runs; imports are already durable.
     this.debounceTimer = setTimeout(() => {
       this.runBatchDedup().catch(error => {
         prodLog.error('[BackgroundDedup] Batch dedup failed:', error)
       })
     }, 5 * 60 * 1000)
+    this.debounceTimer.unref?.()
   }
 
   /**

src/import/ImportCoordinator.ts

@@ -13,7 +13,6 @@
 import { Brainy } from '../brainy.js'
 import { FormatDetector, SupportedFormat } from './FormatDetector.js'
 import { ImportHistory, type ImportHistoryEntry } from './ImportHistory.js'
-import { BackgroundDeduplicator } from './BackgroundDeduplicator.js'
 import { SmartExcelImporter } from '../importers/SmartExcelImporter.js'
 import { SmartPDFImporter } from '../importers/SmartPDFImporter.js'
 import { SmartCSVImporter } from '../importers/SmartCSVImporter.js'
@@ -112,7 +111,12 @@ export interface ValidImportOptions {
   /** Confidence threshold for entities */
   confidenceThreshold?: number
 
-  /** Enable entity deduplication across imports */
+  /**
+   * Enable entity deduplication (default: true). Gates BOTH passes: the
+   * inline merge during import AND the debounced background pass that runs
+   * ~5 minutes after the last import (which merge-DELETES duplicate entities).
+   * Set false for deployments that must never auto-remove records.
+   */
   enableDeduplication?: boolean
 
   /** Similarity threshold for deduplication (0-1) */
@@ -286,7 +290,6 @@ export class ImportCoordinator {
   private brain: Brainy
   private detector: FormatDetector
   private history: ImportHistory
-  private backgroundDedup: BackgroundDeduplicator
   private excelImporter: SmartExcelImporter
   private pdfImporter: SmartPDFImporter
   private csvImporter: SmartCSVImporter
@@ -300,7 +303,6 @@ export class ImportCoordinator {
     this.brain = brain
     this.detector = new FormatDetector()
     this.history = new ImportHistory(brain)
-    this.backgroundDedup = new BackgroundDeduplicator(brain)
     this.excelImporter = new SmartExcelImporter(brain)
     this.pdfImporter = new SmartPDFImporter(brain)
     this.csvImporter = new SmartCSVImporter(brain)
@@ -1459,9 +1461,16 @@ export class ImportCoordinator {
       }
     }
 
-    // Schedule background deduplication (debounced 5 minutes)
+    // Schedule background deduplication (debounced 5 minutes, brain-owned so
-    if (trackingContext && trackingContext.importId) {
+    // close() can cancel it). Honors the same enableDeduplication gate as the
-      this.backgroundDedup.scheduleDedup(trackingContext.importId)
+    // inline pass — false means NO dedup, inline or background.
+    if (
+      trackingContext &&
+      trackingContext.importId &&
+      options.enableDeduplication !== false
+    ) {
+      const backgroundDedup = await this.brain.getBackgroundDeduplicator()
+      backgroundDedup.scheduleDedup(trackingContext.importId)
     }
 
     return {

src/index.ts

@@ -28,6 +28,19 @@ export type { FileVersion } from './vfs/types.js'
 
 // Export diagnostics result type
 export type { DiagnosticsResult } from './brainy.js'
+// brain.warm() — eager index/storage readiness report (per-surface honest
+// outcome + timing). See the WarmReport JSDoc in brainy.ts.
+export type { WarmReport, WarmOutcome } from './brainy.js'
+export type {
+  GraphAuditReport,
+  GraphAuditDiscrepancy
+} from './graph/graphAudit.js'
+export {
+  checkOsLimits,
+  NOFILE_POOL_FLOOR,
+  MAX_MAP_COUNT_POOL_FLOOR
+} from './utils/osLimits.js'
+export type { OsLimitsReport } from './utils/osLimits.js'
 
 // Export Brainy configuration and types
 export type {
@@ -191,6 +204,7 @@ export type {
   TxLogEntry,
   CompactHistoryOptions,
   CompactHistoryResult,
+  HistoryStats,
   ChangedIds,
   DiffResult,
   HistoryVersion,
@@ -202,6 +216,7 @@ export type {
   CommitFact,
   FactOp,
   FactScanBatch,
+  SCANFACTS_FIRST_BATCH_MS,
   FactScanHandle
 } from './db/factLog.js'
 // The generalized family stamp — which source generation a projection

src/plugin.ts

@@ -381,6 +381,20 @@ export interface GraphIndexProvider {
    */
   init?(): Promise<void>
 
+  /**
+   * @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap
+   * pretouch) so first operations run at steady-state cost. Optional;
+   * absence means the provider demand-loads. Distinct from `init?()`: `init`
+   * is called once automatically during brain startup as part of the
+   * readiness contract (cold-load + rebuild gating); `warm` is an explicit,
+   * separate readiness step a caller opts into via `brain.warm()` (or
+   * `warmOnOpen`) specifically to pre-pay demand-load cost that `init` left
+   * lazy. A provider that already loads everything eagerly in `init?()` may
+   * implement `warm` as a no-op or omit it — `brain.warm()` falls back to a
+   * best-effort read-through probe when absent.
+   */
+  warm?(): Promise<void>
+
   /**
    * @description OPTIONAL. A native provider returns true from the moment its
    * `init()` detects a large epoch-drift until its background
@@ -947,6 +961,26 @@ export interface AtGenerationVectors {
  * are retired and never looked up.
  */
 export interface VectorIndexProvider {
+  /**
+   * @description REQUIRED self-reported implementation identity, rendered as
+   * `[vector-index:<name>]` in prose log lines and stamped into the
+   * transaction op-name strings that surface in journals/timings (e.g.
+   * `AddToVectorIndex(hnsw-js)`) — see
+   * `src/transaction/operations/IndexOperations.ts`. A provider must name
+   * itself TRUTHFULLY (its own algorithm/engine, e.g. its plugin/package
+   * name) and must never inherit a default — guessing a backend name would
+   * resurrect the exact bug this field exists to prevent: an operator
+   * hunting an index that isn't the one actually running. The built-in JS
+   * index always sets this to `'hnsw-js'`; a native provider picks its own
+   * string. At the TypeScript level this field is required; a runtime
+   * instance from an older provider compiled against the previous optional
+   * `providerId` contract is tolerated (never crashes, never silently
+   * mislabeled) — see `resolveVectorProviderId` in
+   * `src/transaction/operations/IndexOperations.ts`, which stamps
+   * `'unknown-provider'` and emits one loud warning for that case.
+   */
+  readonly name: string
+
   addItem(item: VectorDocument): Promise<string>
   removeItem(id: string): Promise<boolean>
   search(
@@ -1009,6 +1043,20 @@ export interface VectorIndexProvider {
    */
   init?(): Promise<void>
 
+  /**
+   * @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap
+   * pretouch) so first operations run at steady-state cost. Optional;
+   * absence means the provider demand-loads. Distinct from `init?()`: `init`
+   * runs automatically once during brain startup (the cold-load + rebuild
+   * readiness contract above); `warm` is a separate, explicit step a caller
+   * opts into via `brain.warm()` (or `warmOnOpen`) to pre-pay demand-load
+   * cost `init` left lazy — e.g. touching every mmap page rather than just
+   * opening the file. A provider that already loads everything eagerly in
+   * `init?()` may implement `warm` as a no-op or omit it — `brain.warm()`
+   * falls back to a best-effort probe `search()` when absent.
+   */
+  warm?(): Promise<void>
+
   /**
    * @description OPTIONAL honest durability signal (readiness contract,
    * mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted

src/storage/adapters/fileSystemStorage.ts

@@ -96,6 +96,14 @@ export class FileSystemStorage extends BaseStorage {
   private static readonly WRITER_STALE_THRESHOLD_MS = 60_000
   private writerLockHeartbeat?: NodeJS.Timeout
   private writerLockInfo?: WriterLockInfo
+  /**
+   * The currently-executing heartbeat refresh, if any. `releaseWriterLock()`
+   * awaits it before unlinking: clearInterval() stops FUTURE ticks but not a
+   * tick already in flight, and a straggler landing after the unlink would
+   * RE-CREATE the lock file — a phantom lock blocking the next writer until
+   * the stale TTL expires (the pool-eviction reopen case).
+   */
+  private writerHeartbeatInFlight?: Promise<void>
 
   // Flush-request RPC state. The writer polls `locks/_flush_requests/` for
   // new `.req` files and emits `.ack` files in `locks/_flush_responses/` after
@@ -1764,27 +1772,144 @@ export class FileSystemStorage extends BaseStorage {
     const lockFile = path.join(this.lockDir, FileSystemStorage.WRITER_LOCK_FILE)
     const os = await import('node:os')
     const hostname = os.hostname()
-    const now = new Date().toISOString()
+    const myPid = typeof process !== 'undefined' && process.pid ? process.pid : 0
 
+    // Bounded acquire loop. The CLAIM itself is an atomic create-exclusive
+    // write (O_EXCL) — two processes racing an ABSENT lock can never both
+    // succeed, which closes the read-then-write window where the loser used
+    // to keep running unlocked, silently. An EEXIST loser loops, re-reads,
+    // and handles whatever it finds honestly (fresh foreign lock → loud
+    // throw; stale/forced → verified takeover).
+    const MAX_ATTEMPTS = 3
+    for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) {
+      const now = new Date().toISOString()
       const existing = await this.readWriterLock()
-    if (existing && !options?.force) {
+
+      if (existing) {
         // Same-process re-open: a second Brainy instance in this Node process
         // (e.g. test "simulate server restart" patterns, or a consumer that
         // explicitly re-instantiates without closing first). This isn't the
         // dangerous cross-process case the lock exists to prevent — the two
         // instances share a memory space and can't silently diverge from each
-      // other beyond what their callers already see. Warn and take over the lock.
+        // other beyond what their callers already see. Warn and take over.
-      if (existing.pid === (typeof process !== 'undefined' ? process.pid : 0) &&
+        if (existing.pid === myPid && existing.hostname === hostname && !options?.force) {
-          existing.hostname === hostname) {
           console.warn(
             `[brainy] Re-acquiring writer lock for ${this.rootDir} held by the same process (PID ${existing.pid}). ` +
             `If you intended to keep the previous Brainy instance alive, this is a bug — close it first.`
           )
-      } else {
+          const info: WriterLockInfo = {
-        const stale = await this.isWriterLockStale(existing)
+            pid: myPid,
-        if (!stale) {
+            hostname,
+            startedAt: now,
+            lastHeartbeat: now,
+            version: getBrainyVersion(),
+            rootDir: this.rootDir
+          }
+          await this.writeFileAtomic(lockFile, JSON.stringify(info, null, 2))
+          this.installWriterLock(info)
+          return info
+        }
+
+        const stale = !options?.force && (await this.isWriterLockStale(existing))
+        if (!options?.force && !stale) {
           // Consumer-facing error contract: callers detect this case via
           // err.code and read the holder's details from err.lockInfo.
+          throw this.writerLockedError(existing)
+        }
+
+        console.warn(
+          options?.force
+            ? `[brainy] Force-overwriting writer lock for ${this.rootDir} ` +
+              `(was held by PID ${existing.pid} on ${existing.hostname}).`
+            : `[brainy] Overwriting stale writer lock for ${this.rootDir} ` +
+              `(PID ${existing.pid} on ${existing.hostname} appears dead).`
+        )
+        // Takeover: verify the file still holds the lock we judged (a live
+        // successor may have claimed meanwhile), then remove it and fall
+        // through to the atomic claim below. A racing claimer who beats us to
+        // the create simply wins — our next loop iteration reads their fresh
+        // lock and throws honestly. (Advisory file locking has no
+        // compare-and-delete; staleness requiring a 60s-old heartbeat keeps
+        // the residual verify-to-unlink window practically unreachable.)
+        const recheck = await this.readWriterLock()
+        if (
+          recheck &&
+          (recheck.pid !== existing.pid ||
+            recheck.startedAt !== existing.startedAt ||
+            recheck.lastHeartbeat !== existing.lastHeartbeat)
+        ) {
+          continue // the lock changed hands while we deliberated — re-evaluate
+        }
+        try {
+          await fs.promises.unlink(lockFile)
+        } catch (err: any) {
+          if (err.code !== 'ENOENT') throw err
+        }
+      }
+
+      const info: WriterLockInfo = {
+        pid: myPid,
+        hostname,
+        startedAt: existing && options?.force ? existing.startedAt : now,
+        lastHeartbeat: now,
+        version: getBrainyVersion(),
+        rootDir: this.rootDir
+      }
+
+      // The atomic claim: create-exclusive, so exactly ONE racer wins.
+      try {
+        await fs.promises.writeFile(lockFile, JSON.stringify(info, null, 2), { flag: 'wx' })
+      } catch (err: any) {
+        if (err.code === 'EEXIST') {
+          continue // someone else claimed between our read and create — re-evaluate
+        }
+        throw err
+      }
+
+      this.installWriterLock(info)
+      return info
+    }
+
+    // Attempts exhausted: something is claiming this directory faster than we
+    // can evaluate it. Read whoever holds it now and fail loudly with their
+    // details rather than degrading into a lockless open.
+    const holder = await this.readWriterLock()
+    if (holder) throw this.writerLockedError(holder)
+    throw new Error(
+      `Failed to acquire the writer lock for ${this.rootDir} after ${MAX_ATTEMPTS} attempts — ` +
+      `the lock file is being contended. Retry, or inspect ${lockFile}.`
+    )
+  }
+
+  /** Record lock ownership + start the unref'd heartbeat. */
+  private installWriterLock(info: WriterLockInfo): void {
+    this.writerLockInfo = info
+
+    // Heartbeat — rewrite lastHeartbeat every WRITER_HEARTBEAT_MS so other
+    // processes can tell a live writer from one that crashed without releasing.
+    this.writerLockHeartbeat = setInterval(() => {
+      const tick = this.refreshWriterLockHeartbeat().catch((err) => {
+        // ENOENT = the lock (or its directory) vanished mid-refresh — the
+        // store was released or removed under us; the next acquire recreates
+        // it. Benign by construction; anything else stays loud.
+        if ((err as NodeJS.ErrnoException)?.code !== 'ENOENT') {
+          console.warn('[brainy] Failed to refresh writer lock heartbeat:', err)
+        }
+      })
+      this.writerHeartbeatInFlight = tick.finally(() => {
+        if (this.writerHeartbeatInFlight === tick) {
+          this.writerHeartbeatInFlight = undefined
+        }
+      })
+    }, FileSystemStorage.WRITER_HEARTBEAT_MS)
+    if (typeof this.writerLockHeartbeat.unref === 'function') {
+      // Don't keep the event loop alive just for the heartbeat.
+      this.writerLockHeartbeat.unref()
+    }
+  }
+
+  /** The consumer-facing BRAINY_WRITER_LOCKED error, holder details attached. */
+  private writerLockedError(existing: WriterLockInfo): Error {
     const err = new Error(
       `Another writer holds this Brainy directory.\n` +
       `  PID:        ${existing.pid} on host ${existing.hostname}\n` +
@@ -1798,45 +1923,7 @@ export class FileSystemStorage extends BaseStorage {
     ) as Error & { code: string; lockInfo: WriterLockInfo }
     err.code = 'BRAINY_WRITER_LOCKED'
     err.lockInfo = existing
-          throw err
+    return err
-        }
-        console.warn(
-          `[brainy] Overwriting stale writer lock for ${this.rootDir} ` +
-          `(PID ${existing.pid} on ${existing.hostname} appears dead).`
-        )
-      }
-    } else if (existing && options?.force) {
-      console.warn(
-        `[brainy] Force-overwriting writer lock for ${this.rootDir} ` +
-        `(was held by PID ${existing.pid} on ${existing.hostname}).`
-      )
-    }
-
-    const info: WriterLockInfo = {
-      pid: typeof process !== 'undefined' && process.pid ? process.pid : 0,
-      hostname,
-      startedAt: existing && options?.force ? existing.startedAt : now,
-      lastHeartbeat: now,
-      version: getBrainyVersion(),
-      rootDir: this.rootDir
-    }
-
-    await this.writeFileAtomic(lockFile, JSON.stringify(info, null, 2))
-    this.writerLockInfo = info
-
-    // Heartbeat — rewrite lastHeartbeat every WRITER_HEARTBEAT_MS so other
-    // processes can tell a live writer from one that crashed without releasing.
-    this.writerLockHeartbeat = setInterval(() => {
-      this.refreshWriterLockHeartbeat().catch((err) => {
-        console.warn('[brainy] Failed to refresh writer lock heartbeat:', err)
-      })
-    }, FileSystemStorage.WRITER_HEARTBEAT_MS)
-    if (typeof this.writerLockHeartbeat.unref === 'function') {
-      // Don't keep the event loop alive just for the heartbeat.
-      this.writerLockHeartbeat.unref()
-    }
-
-    return info
   }
 
   public override async releaseWriterLock(): Promise<void> {
@@ -1844,6 +1931,15 @@ export class FileSystemStorage extends BaseStorage {
       clearInterval(this.writerLockHeartbeat)
       this.writerLockHeartbeat = undefined
     }
+    // Drain an in-flight heartbeat tick BEFORE unlinking: clearInterval stops
+    // future ticks only, and a straggler write landing after the unlink would
+    // re-create the lock as a phantom (blocking the next writer until the
+    // stale TTL). After the drain, any refresh is either fully landed (we
+    // unlink its output below) or not started (it sees writerLockInfo
+    // undefined and returns).
+    if (this.writerHeartbeatInFlight) {
+      await this.writerHeartbeatInFlight
+    }
     if (!this.writerLockInfo) {
       return
     }

src/storage/adapters/memoryStorage.ts

@@ -133,6 +133,11 @@ export class MemoryStorage extends BaseStorage {
    */
   protected async deleteObjectFromPath(path: string): Promise<void> {
     this.objectStore.delete(path)
+    // Filesystem parity: on disk, objects and raw BYTE files are both just
+    // files — unlink removes whichever exists. Without this, deleteRawObject
+    // on a raw-bytes path (fact-log/generation segments) silently no-ops on
+    // memory storage: the delete "succeeds" and the bytes remain.
+    this.rawBytesStore.delete(path)
   }
 
   /**

src/storage/baseStorage.ts

@@ -2068,11 +2068,20 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // Cursor (8.0): resume token carrying the (shard, nounId) of the last returned
     // noun — the noun mirror of getVerbsWithPagination. When present it supersedes
     // `offset` and resumes the shard walk immediately AFTER that position, so a full
-    // walk is O(N) instead of the O(N²) of offset paging. Malformed/foreign tokens
+    // walk is O(N) instead of the O(N²) of offset paging. (Previously the cursor was
-    // decode to null → offset fallback. (Previously the cursor was ignored, which
+    // ignored, which was latent — the only multi-page consumer used a single big
-    // was latent — the only multi-page consumer used a single big page — until small
+    // page — until small chunk sizes needed page 2 and an offset-0-on-every-call
-    // chunk sizes needed page 2 and an offset-0-on-every-call walk never terminated.)
+    // walk never terminated.)
     const cursor = this.decodeNounWalkCursor(options.cursor)
+    if (options.cursor && cursor === null) {
+      // A supplied-but-undecodable resume token must FAIL, not silently restart
+      // at offset 0 — to a while(hasMore) caller the silent fallback re-serves
+      // page 1 forever: an unbounded CPU loop wearing pagination's clothes.
+      throw BrainyError.storage(
+        `getNouns: invalid pagination cursor '${options.cursor}' — cannot resume this walk. ` +
+          `Restart it without a cursor.`
+      )
+    }
     const collected: Array<{ noun: HNSWNounWithMetadata; shard: number }> = []
 
     // Peek one past the window so `hasMore` is decidable. Cursor mode collects one
@@ -2221,6 +2230,14 @@ export abstract class BaseStorage extends BaseStorageAdapter {
 
     const { limit, offset = 0, filter } = options
     const cursor = this.decodeNounWalkCursor(options.cursor)
+    if (options.cursor && cursor === null) {
+      // Same law as getNouns/getVerbs: an undecodable resume token FAILS
+      // instead of silently restarting the walk at offset 0.
+      throw BrainyError.storage(
+        `getNounIds: invalid pagination cursor '${options.cursor}' — cannot resume this walk. ` +
+          `Restart it without a cursor.`
+      )
+    }
     const collected: Array<{ id: string; shard: number }> = []
     const peekCount = cursor ? limit + 1 : offset + limit + 1
     const startShard = cursor ? cursor.shard : 0
@@ -2366,8 +2383,17 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // (shard, verbId) of the last returned verb. When present it SUPERSEDES `offset`
     // and resumes the shard walk immediately AFTER that position, so a full walk is
     // O(N) total instead of the O(N²) of offset paging (which re-scans from shard 0
-    // every page). Malformed / foreign tokens decode to null → offset fallback.
+    // every page).
     const cursor = this.decodeVerbWalkCursor(options.cursor)
+    if (options.cursor && cursor === null) {
+      // A supplied-but-undecodable resume token must FAIL, not silently restart
+      // at offset 0 — to a while(hasMore) caller the silent fallback re-serves
+      // page 1 forever: an unbounded CPU loop wearing pagination's clothes.
+      throw BrainyError.storage(
+        `getVerbs: invalid pagination cursor '${options.cursor}' — cannot resume this walk. ` +
+          `Restart it without a cursor.`
+      )
+    }
 
     // Each collected entry remembers its shard so nextCursor can point at the exact
     // (shard, id) resume position.

src/transaction/Transaction.ts

@@ -37,6 +37,49 @@ const DEFAULT_OPTIONS: Required<TransactionOptions> = {
   maxRollbackRetries: 3
 }
 
+/**
+ * The floor term of {@link transactTimeoutBudget}'s scaling formula when the
+ * caller supplies neither an explicit override nor `config.transactionBudgetFloorMs`.
+ */
+const DEFAULT_BUDGET_FLOOR_MS = 30_000
+
+/**
+ * The apply-phase budget for a batch of `opCount` operations.
+ *
+ * An explicit `override` wins untouched (a caller-specified deadline for this
+ * one batch). Otherwise the budget SCALES with the batch:
+ * `max(floorMs, opCount × 2 000)`, where `floorMs` defaults to `30 000` and
+ * is overridable via `BrainyConfig.transactionBudgetFloorMs` for the whole
+ * brain. The per-op term is calibrated from field data — bulk imports on
+ * network-attached disks measure ~2 s per operation (each op pays canonical
+ * writes + fsync + index maintenance) — so a flat 30 s budget silently capped
+ * honest work at ~15 operations while looking generous for small batches. A
+ * 16-op batch, for example, scales to `max(30 000, 16 × 2 000) = 32 000`.
+ * Scaling keeps small transacts fast-failing and gives bulk ones a budget
+ * proportional to the work they actually asked for.
+ *
+ * This is a **mid-batch** budget only: it governs whether the transaction's
+ * NEXT operation may start (see {@link Transaction.execute}), never whether
+ * already-completed work is rolled back after the fact. A trip mid-batch
+ * still rolls back every operation applied so far, atomically, and throws a
+ * retryable, fully-labeled TransactionTimeoutError — that zero-loss guarantee
+ * doesn't change; only the point at which the clock stops mattering does (at
+ * the last operation, not one check later).
+ *
+ * @param opCount - Number of operations in the batch.
+ * @param override - A full override for this call; wins over everything else.
+ * @param floorMs - The scaling floor for this call (e.g. `config.transactionBudgetFloorMs`).
+ *   Ignored when `override` is set. Defaults to `30 000` when omitted.
+ */
+export function transactTimeoutBudget(
+  opCount: number,
+  override?: number,
+  floorMs?: number
+): number {
+  if (override !== undefined) return override
+  return Math.max(floorMs ?? DEFAULT_BUDGET_FLOOR_MS, opCount * 2_000)
+}
+
 /**
  * Transaction class
  */
@@ -80,7 +123,20 @@ export class Transaction implements TransactionContext {
   }
 
   /**
-   * Execute all operations atomically
+   * Execute all operations atomically.
+   *
+   * Budget semantics: the configured budget gates starting the next
+   * operation; completed work is never rolled back for elapsed time. Elapsed
+   * time is checked ONLY before starting operation `i` for `i ≥ 1` — never
+   * before operation 0 (an operation always gets to start) and never again
+   * after the final operation completes. Concretely: a single-op transaction
+   * can never time out post-hoc — it either runs (and its result stands, no
+   * matter how long it took) or a `TransactionTimeoutError` is thrown before
+   * it starts, which cannot happen since there is no operation before it. A
+   * multi-op transaction whose op `i` overruns the budget stops op `i+1` from
+   * starting, rolls back operations `0..i` (reverse order), and throws
+   * `TransactionTimeoutError` — that zero-loss rollback guarantee for
+   * mid-batch trips is unchanged; only the post-completion check is gone.
    */
   async execute(): Promise<void> {
     if (this.state !== 'pending') {
@@ -110,11 +166,19 @@ export class Transaction implements TransactionContext {
       // already-applied writes as torn, generation-less state in canonical
       // storage.
       for (let i = 0; i < this.operations.length; i++) {
-        // Budget check BEFORE starting the next operation. A trip here throws
+        // Budget gates STARTING the next operation; completed work is never
-        // into the catch below and rolls back like any other failure — it must
+        // rolled back for elapsed time. Skipped for i === 0 (operation 0
-        // never bypass rollback.
+        // always gets to start — nothing has run yet to have overrun) and
-        if (Date.now() - this.startTime > this.options.timeout) {
+        // never re-checked after the final operation completes (there is no
-          throw new TransactionTimeoutError(this.options.timeout, i)
+        // "next operation" left to gate). A trip here throws into the catch
+        // below and rolls back like any other failure — it must never bypass
+        // rollback.
+        if (i > 0 && Date.now() - this.startTime > this.options.timeout) {
+          throw new TransactionTimeoutError(this.options.timeout, i, {
+            elapsedMs: Date.now() - this.startTime,
+            totalOperations: this.operations.length,
+            operationName: this.operations[i]?.name
+          })
         }
 
         const operation = this.operations[i]

src/transaction/errors.ts

@@ -77,11 +77,24 @@ export class InvalidTransactionStateError extends TransactionError {
 export class TransactionTimeoutError extends TransactionError {
   constructor(
     timeoutMs: number,
-    operationIndex: number
+    operationIndex: number,
+    telemetry?: {
+      elapsedMs?: number
+      totalOperations?: number
+      operationName?: string
+    }
   ) {
+    const progress =
+      telemetry?.totalOperations !== undefined
+        ? `${operationIndex}/${telemetry.totalOperations}`
+        : String(operationIndex)
+    const name = telemetry?.operationName ? ` ('${telemetry.operationName}')` : ''
+    const elapsed =
+      telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
     super(
-      `Transaction timed out after ${timeoutMs}ms at operation ${operationIndex}`,
+      `Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
-      { timeoutMs, operationIndex }
+        `The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`,
+      { timeoutMs, operationIndex, ...telemetry }
     )
     this.name = 'TransactionTimeoutError'
   }

src/transaction/operations/IndexOperations.ts

@@ -2,34 +2,73 @@
  * Index Operations with Rollback Support
  *
  * Provides transactional operations for all indexes:
- * - JsHnswVectorIndex (unified vector index)
+ * - VectorIndexProvider (the JS HNSW fallback, or a native acceleration provider)
  * - MetadataIndexManager (roaring bitmap filtering)
  * - GraphAdjacencyIndex (LSM-tree graph storage)
  *
  * Each operation can be executed and rolled back atomically.
  */
 
-import type { JsHnswVectorIndex } from '../../hnsw/hnswIndex.js'
+import type { VectorIndexProvider, GraphIndexProvider } from '../../plugin.js'
 import type { MetadataIndexManager } from '../../utils/metadataIndex.js'
-import type { GraphIndexProvider } from '../../plugin.js'
 import type { GraphVerb } from '../../coreTypes.js'
 import type { Operation, RollbackAction } from '../types.js'
 
 /**
- * Add to HNSW index with rollback support
+ * Backend identity stamped into an operation's emitted `name` string (e.g.
+ * `AddToVectorIndex(hnsw-js)`), resolved from the provider's own
+ * {@link VectorIndexProvider.name} self-report.
+ *
+ * These operation classes are backend-neutral (the vector index they wrap may
+ * be Brainy's own JS HNSW fallback OR a native acceleration provider), but
+ * their names surface directly in consumer-visible transaction journals and
+ * timings. `name` is REQUIRED at the TypeScript level — but a native provider
+ * instance compiled against the previous (pre-required) contract can still
+ * reach this function at runtime without it. That legacy case is tolerated,
+ * never crashed on and never silently mislabeled: it resolves to
+ * `'unknown-provider'` and emits ONE loud warning naming the missing contract
+ * field, so the fix (implement `name`) is discoverable rather than a silent
+ * fossil label in every journal line thereafter.
+ */
+const warnedMissingName = new WeakSet<VectorIndexProvider>()
+
+function resolveVectorProviderId(index: VectorIndexProvider): string {
+  const name = (index as { name?: unknown }).name
+  if (typeof name === 'string') return name
+  if (!warnedMissingName.has(index)) {
+    warnedMissingName.add(index)
+    console.warn(
+      '[vector-index] provider is missing the required `name` field (VectorIndexProvider.name, ' +
+        'required since 8.10.0) — stamping "unknown-provider" in transaction op names until the ' +
+        'provider declares its own identity.'
+    )
+  }
+  return 'unknown-provider'
+}
+
+/**
+ * Add to the vector index with rollback support.
+ *
+ * Backend-neutral: `index` is whatever the `'vector'` provider factory
+ * returns — Brainy's own JS HNSW fallback, or a native acceleration provider
+ * (e.g. DiskANN). The emitted `name` stamps the active backend (see
+ * {@link resolveVectorProviderId}) so operators reading a transaction journal
+ * or timing trace see which engine actually ran, never a fossil name from
+ * whichever engine happened to be active when this op class was written.
  *
  * Rollback strategy:
  * - Remove item from index
-
  */
-export class AddToHNSWOperation implements Operation {
+export class AddToVectorIndexOperation implements Operation {
-  readonly name = 'AddToHNSW'
+  readonly name: string
 
   constructor(
-    private readonly index: JsHnswVectorIndex,
+    private readonly index: VectorIndexProvider,
     private readonly id: string,
     private readonly vector: number[]
-  ) {}
+  ) {
+    this.name = `AddToVectorIndex(${resolveVectorProviderId(index)})`
+  }
 
   async execute(): Promise<RollbackAction> {
     // Check if item already exists (for rollback decision)
@@ -59,11 +98,12 @@ export class AddToHNSWOperation implements Operation {
    * pre-existence as "existed" made every rollback skip removeItem, leaving
    * phantom entries in the index after a failed transaction. The safe default
    * is to remove what this operation added — update flows pair this op with a
-   * RemoveFromHNSWOperation whose own rollback restores the prior vector, so
+   * RemoveFromVectorIndexOperation whose own rollback restores the prior
-   * reverse-order rollback reconstructs the original state either way.
+   * vector, so reverse-order rollback reconstructs the original state either
+   * way.
    */
   private async itemExists(id: string): Promise<boolean> {
-    const index = this.index as JsHnswVectorIndex & {
+    const index = this.index as VectorIndexProvider & {
       getItem?: (id: string) => Promise<unknown>
     }
     if (typeof index.getItem !== 'function') return false
@@ -77,21 +117,27 @@ export class AddToHNSWOperation implements Operation {
 }
 
 /**
- * Remove from HNSW index with rollback support
+ * Remove from the vector index with rollback support.
+ *
+ * Backend-neutral: see {@link AddToVectorIndexOperation} — `index` may be the
+ * JS HNSW fallback or a native acceleration provider; the emitted `name`
+ * stamps the active backend.
  *
  * Rollback strategy:
  * - Re-add item to index with original vector
  *
  * Note: Requires storing the vector for rollback
  */
-export class RemoveFromHNSWOperation implements Operation {
+export class RemoveFromVectorIndexOperation implements Operation {
-  readonly name = 'RemoveFromHNSW'
+  readonly name: string
 
   constructor(
-    private readonly index: JsHnswVectorIndex,
+    private readonly index: VectorIndexProvider,
     private readonly id: string,
     private readonly vector: number[] // Required for rollback
-  ) {}
+  ) {
+    this.name = `RemoveFromVectorIndex(${resolveVectorProviderId(index)})`
+  }
 
   async execute(): Promise<RollbackAction> {
     // Remove from index
@@ -285,22 +331,24 @@ export class RemoveFromGraphIndexOperation implements Operation {
 }
 
 /**
- * Batch operation: Add multiple items to HNSW index
+ * Batch operation: Add multiple items to the vector index (backend-neutral —
+ * see {@link AddToVectorIndexOperation}).
  *
  * Useful for bulk imports with transaction support.
  * Rolls back all items if any fail.
  */
-export class BatchAddToHNSWOperation implements Operation {
+export class BatchAddToVectorIndexOperation implements Operation {
-  readonly name = 'BatchAddToHNSW'
+  readonly name: string
 
-  private operations: AddToHNSWOperation[]
+  private operations: AddToVectorIndexOperation[]
 
   constructor(
-    index: JsHnswVectorIndex,
+    index: VectorIndexProvider,
     items: Array<{ id: string; vector: number[] }>
   ) {
+    this.name = `BatchAddToVectorIndex(${resolveVectorProviderId(index)})`
     this.operations = items.map(
-      item => new AddToHNSWOperation(index, item.id, item.vector)
+      item => new AddToVectorIndexOperation(index, item.id, item.vector)
     )
   }
 

src/transaction/operations/index.ts

@@ -21,12 +21,12 @@ export {
 
 // Index Operations
 export {
-  AddToHNSWOperation,
+  AddToVectorIndexOperation,
-  RemoveFromHNSWOperation,
+  RemoveFromVectorIndexOperation,
   AddToMetadataIndexOperation,
   RemoveFromMetadataIndexOperation,
   AddToGraphIndexOperation,
   RemoveFromGraphIndexOperation,
-  BatchAddToHNSWOperation,
+  BatchAddToVectorIndexOperation,
   BatchAddToMetadataIndexOperation
 } from './IndexOperations.js'

src/types/brainy.types.ts

@@ -1651,6 +1651,25 @@ export interface BrainyConfig {
    */
   disableAutoRebuild?: boolean
 
+  /**
+   * The floor (ms) of the apply-phase transaction budget's scaling formula:
+   * `max(transactionBudgetFloorMs, opCount × 2 000)`. The budget governs
+   * whether a `transact()` / single-op write's NEXT internal operation may
+   * **start** — never whether already-completed work gets rolled back after
+   * the fact (a single-op write can never time out post-hoc: it either runs
+   * or it commits). A trip mid-batch still rolls back every applied operation
+   * atomically and throws a retryable `TransactionTimeoutError`; only the
+   * floor of the formula is configurable here.
+   *
+   * Raise this when a cold store's first writes after a restart legitimately
+   * take longer than 30s per operation (e.g. page-cache-cold canonical writes
+   * on a large brain) so a bulk `transact()` batch gets a proportionally
+   * larger runway instead of tripping mid-batch. Lower it to fail faster on a
+   * latency-sensitive write path. Default: `30000` (30 s) — unchanged from
+   * pre-8.9 behavior.
+   */
+  transactionBudgetFloorMs?: number
+
   /**
    * How long (ms) an operation **waits** on the coordinated 7.x → 8.0 migration
    * before throwing a retryable `MigrationInProgressError`.
@@ -1737,7 +1756,10 @@ export interface BrainyConfig {
    * Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
    * `remove`/`relate`) produces an immutable generation record-set serving
    * historical reads (`asOf()`, pinned `Db` values). Without compaction those
-   * accumulate, so Brainy **auto-compacts on every `flush()` and `close()`**.
+   * accumulate, so Brainy **auto-compacts at `close()`** (time-bounded per
+   * pass; 8.9.0 removed compaction from `flush()` — flush is durability work
+   * and never pays maintenance costs). A long-lived writer that never closes
+   * accumulates history until its next explicit `compactHistory()` call.
    * Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
    *
    * Modes:
@@ -1754,7 +1776,7 @@ export interface BrainyConfig {
    *   the oldest unpinned generations while ANY supplied cap is exceeded
    *   (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
    *
-   * `autoCompact: false` disables the automatic flush/close compaction (manage
+   * `autoCompact: false` disables the automatic close() compaction (manage
    * manually via `brain.compactHistory()`). `budgetBytes` is the settable
    * adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
    * Long-term archives belong in `db.persist(path)` snapshots, which compaction
@@ -1772,7 +1794,7 @@ export interface BrainyConfig {
         maxBytes?: number
         /** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
         budgetBytes?: number
-        /** Run compaction automatically on flush()/close() (default: true). */
+        /** Run compaction automatically at close() (default: true; 8.9.0 — flush() never compacts). */
         autoCompact?: boolean
       }
 
@@ -1803,6 +1825,23 @@ export interface BrainyConfig {
    */
   eagerEmbeddings?: boolean
 
+  /**
+   * When `true`, `init()` **awaits `warm()`** (see {@link Brainy.warm}) before
+   * it resolves — blocking startup until the vector index, metadata index,
+   * and graph adjacency have all faulted their backing storage in. This is a
+   * deliberate operator opt-in trade: startup takes longer, but the FIRST
+   * request after a cold restart runs at steady-state cost instead of paying
+   * page-cache-miss / demand-load latency on the critical path.
+   *
+   * Runs AFTER `init()`'s own sequence completes (index construction, crash
+   * recovery, migrations, VFS bootstrap) — never in place of any of it — so
+   * `warm()` always operates on a fully-initialized brain.
+   *
+   * Default: `false` (lazy — the pre-8.9 behavior: indexes demand-load on
+   * first access).
+   */
+  warmOnOpen?: boolean
+
   // Plugin configuration
   // Controls which plugins are loaded during init().
   // - undefined (default): guarded auto-detection of the first-party

src/utils/metadataIndex.ts

@@ -423,6 +423,46 @@ export class MetadataIndexManager implements MetadataIndexProvider {
     prodLog.debug('✅ Type-aware cache warming completed')
   }
 
+  /**
+   * Full hydration — the {@link Brainy.warm} readiness seam for the metadata
+   * index. Unlike {@link warmCache} / {@link warmCacheForTopTypes} (which
+   * warm only a heuristic subset: common fields plus the top-N types' top
+   * fields), this loads EVERY field's sparse index the field registry knows
+   * about — a real read through {@link loadSparseIndex} into the unified
+   * cache for each field, not a stat/existence check. Idempotent: an
+   * already-cached field's `loadSparseIndex` call is a cheap cache hit.
+   *
+   * Re-reads the field registry first when `fieldIndexes` is empty (a warm()
+   * call issued before `init()` populated it would otherwise hydrate
+   * nothing), then loads every discovered field in parallel.
+   */
+  async hydrateAll(): Promise<void> {
+    if (this.fieldIndexes.size === 0) {
+      await this.loadFieldRegistry()
+    }
+
+    const fields = Array.from(this.fieldIndexes.keys())
+    if (fields.length === 0) {
+      prodLog.debug('[MetadataIndex] hydrateAll: no persisted fields to hydrate')
+      return
+    }
+
+    prodLog.debug(`[MetadataIndex] hydrateAll: loading ${fields.length} field(s) — ${fields.join(', ')}`)
+
+    await Promise.all(
+      fields.map(async field => {
+        try {
+          await this.loadSparseIndex(field)
+        } catch (error) {
+          // A single field's load failure doesn't abort the rest of the
+          // hydration — warm() is a best-effort readiness step, never a
+          // correctness gate (queries still demand-load on miss).
+          prodLog.debug(`[MetadataIndex] hydrateAll: field '${field}' failed to load:`, error)
+        }
+      })
+    )
+  }
+
   /**
    * Acquire an in-memory lock for coordinating concurrent metadata index writes
    * Uses in-memory locks since MetadataIndexManager doesn't have direct file system access
@@ -1867,9 +1907,26 @@ export class MetadataIndexManager implements MetadataIndexProvider {
     // not once per AND-clause inside it.
     const unindexedFields: string[] = []
 
-    for (const [field, condition] of Object.entries(filter)) {
+    for (const [rawField, condition] of Object.entries(filter)) {
       // Skip logical operators
-      if (field === 'allOf' || field === 'anyOf' || field === 'not') continue
+      if (rawField === 'allOf' || rawField === 'anyOf' || rawField === 'not') continue
+
+      // Metadata is FLATTENED at index time (metadata.entry.title indexes as
+      // entry.title), so a `metadata.`-prefixed where key is almost always
+      // the caller spelling the STORAGE shape rather than the index shape.
+      // Accept both spellings: when the key as spelled is unindexed but its
+      // stripped spelling is, query the stripped one. A literal nested
+      // custom key named `metadata` still wins when indexed as spelled
+      // (checked first), so that rare shape keeps working.
+      let field = rawField
+      if (
+        rawField.startsWith('metadata.') &&
+        this.columnStore &&
+        !this.columnStore.hasField(rawField) &&
+        this.columnStore.hasField(rawField.slice('metadata.'.length))
+      ) {
+        field = rawField.slice('metadata.'.length)
+      }
 
       let fieldResults: string[] = []
 

src/utils/osLimits.ts

@@ -0,0 +1,141 @@
+/**
+ * @module utils/osLimits
+ * @description Detect-and-warn for OS resource limits that bite at POOL scale.
+ *
+ * A single brain rarely notices them, but a pool of brains — especially with a
+ * native accelerator memory-mapping many index files per brain — consumes file
+ * descriptors and memory mappings multiplicatively. On stock Linux defaults
+ * (RLIMIT_NOFILE soft 1024, vm.max_map_count 65530) the failure arrives as
+ * EMFILE or a failed mmap deep inside an index open, long after the real cause
+ * (the limit) stopped being visible. This module reads the limits at open and
+ * WARNS ONCE per process with the exact raise commands, so the operator learns
+ * the fix before the incident instead of from it.
+ *
+ * Read-only and Linux-only by construction: both sources are `/proc` files.
+ * On platforms where they are absent the check reports nulls and stays silent —
+ * no limit read means no claim made, never a guessed warning.
+ */
+
+import * as fs from 'node:fs'
+import { prodLog } from './logger.js'
+
+/** Soft-NOFILE floor below which pool-scale use is at EMFILE risk. */
+export const NOFILE_POOL_FLOOR = 65536
+
+/** vm.max_map_count floor below which mmap-heavy native indexes are at risk. */
+export const MAX_MAP_COUNT_POOL_FLOOR = 262144
+
+export interface OsLimitsReport {
+  /** RLIMIT_NOFILE soft limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileSoft: number | null
+  /** RLIMIT_NOFILE hard limit (null when unreadable; Infinity for 'unlimited'). */
+  nofileHard: number | null
+  /** vm.max_map_count (null when unreadable). */
+  maxMapCount: number | null
+  /** Human-actionable warnings for limits below the pool floors. Empty = fine. */
+  warnings: string[]
+}
+
+/**
+ * Parse the `Max open files` row of a `/proc/<pid>/limits` document into
+ * soft/hard values. Returns nulls when the row is absent or malformed.
+ */
+export function parseProcLimits(content: string): { soft: number | null; hard: number | null } {
+  const line = content.split('\n').find((l) => l.startsWith('Max open files'))
+  if (!line) return { soft: null, hard: null }
+  const m = line.match(/^Max open files\s+(\S+)\s+(\S+)/)
+  if (!m) return { soft: null, hard: null }
+  const parse = (v: string): number | null => {
+    if (v === 'unlimited') return Infinity
+    const n = Number.parseInt(v, 10)
+    return Number.isNaN(n) ? null : n
+  }
+  return { soft: parse(m[1]), hard: parse(m[2]) }
+}
+
+/**
+ * Assess readable limits against the pool floors. Pure — feed it any values.
+ * A null (unreadable) limit produces NO warning: no measurement, no claim.
+ */
+export function assessOsLimits(limits: {
+  nofileSoft: number | null
+  nofileHard: number | null
+  maxMapCount: number | null
+}): string[] {
+  const warnings: string[] = []
+
+  if (limits.nofileSoft !== null && limits.nofileSoft < NOFILE_POOL_FLOOR) {
+    const hardNote =
+      limits.nofileHard !== null && limits.nofileHard >= NOFILE_POOL_FLOOR
+        ? ` (the hard limit ${limits.nofileHard === Infinity ? 'unlimited' : limits.nofileHard} already allows it — raise the soft limit only)`
+        : ''
+    warnings.push(
+      `RLIMIT_NOFILE soft limit is ${limits.nofileSoft} — below the ${NOFILE_POOL_FLOOR} recommended ` +
+        `for pool-scale use (a pool of brains with a native accelerator opens many index files per brain; ` +
+        `the failure mode is EMFILE deep inside an index open). Raise with \`ulimit -n ${NOFILE_POOL_FLOOR}\` ` +
+        `or LimitNOFILE=${NOFILE_POOL_FLOOR} in the service unit${hardNote}.`
+    )
+  }
+
+  if (limits.maxMapCount !== null && limits.maxMapCount < MAX_MAP_COUNT_POOL_FLOOR) {
+    warnings.push(
+      `vm.max_map_count is ${limits.maxMapCount} — below the ${MAX_MAP_COUNT_POOL_FLOOR} recommended ` +
+        `for mmap-heavy native indexes at pool scale (each mapped index segment consumes map entries; ` +
+        `the failure mode is a failed mmap mid-heal). Raise with ` +
+        `\`sysctl -w vm.max_map_count=${MAX_MAP_COUNT_POOL_FLOOR}\` (persist in /etc/sysctl.d/).`
+    )
+  }
+
+  return warnings
+}
+
+/**
+ * Read the limits from /proc and assess them. `readFile` is injectable for
+ * tests; absent/unreadable sources yield nulls (and therefore no warnings).
+ */
+export async function checkOsLimits(
+  readFile: (path: string) => Promise<string> = async (p) => fs.promises.readFile(p, 'utf-8')
+): Promise<OsLimitsReport> {
+  let nofileSoft: number | null = null
+  let nofileHard: number | null = null
+  let maxMapCount: number | null = null
+
+  try {
+    const parsed = parseProcLimits(await readFile('/proc/self/limits'))
+    nofileSoft = parsed.soft
+    nofileHard = parsed.hard
+  } catch {
+    // Not Linux (or /proc unavailable) — no measurement, no claim.
+  }
+
+  try {
+    const raw = (await readFile('/proc/sys/vm/max_map_count')).trim()
+    const n = Number.parseInt(raw, 10)
+    maxMapCount = Number.isNaN(n) ? null : n
+  } catch {
+    // Not Linux — same rule.
+  }
+
+  const warnings = assessOsLimits({ nofileSoft, nofileHard, maxMapCount })
+  return { nofileSoft, nofileHard, maxMapCount, warnings }
+}
+
+/** Once-per-process latch so a brain pool warns once, not once per brain. */
+let osLimitsWarned = false
+
+/**
+ * Run the check and warn (once per process) about limits below the pool
+ * floors. Called from brain open; safe everywhere (silent off-Linux).
+ */
+export async function warnOnLowOsLimits(): Promise<void> {
+  if (osLimitsWarned) return
+  osLimitsWarned = true
+  try {
+    const report = await checkOsLimits()
+    for (const warning of report.warnings) {
+      prodLog.warn(`[Brainy] OS limit check: ${warning}`)
+    }
+  } catch {
+    // The check must never affect open — measurement-only.
+  }
+}

tests/integration/aggregate-reserved-fields.test.ts

@@ -0,0 +1,169 @@
+/**
+ * @module tests/integration/aggregate-reserved-fields
+ * @description One field-resolution law across the whole aggregation + query
+ * surface (SELF-AGGREGATE-DELETE-DRIFT). Laws:
+ * (1) aggregates grouped by a RESERVED field (subtype) decrement on delete —
+ *     the delete-side entity view carries every reserved field, so the
+ *     decrement finds its group (counts must never drift from ground truth);
+ * (2) same for update: moving an entity between reserved-field groups
+ *     decrements the old group and increments the new one (no double-count);
+ * (3) aggregation source.where on a reserved field (subtype) FILTERS instead
+ *     of silently matching nothing;
+ * (4) removeMany refuses empty/invalid selectors loudly (bare array, empty
+ *     object, ids: []) instead of resolving as a silent no-op;
+ * (5) find() accepts both where spellings: flattened (entry.title) and
+ *     storage-shaped (metadata.entry.title) resolve to the same rows.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+
+const stubEmbedding = async (text: string): Promise<number[]> => {
+  const hash = text.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0)
+  return new Array(384).fill(0).map((_, i) => Math.sin(hash + i))
+}
+
+describe('aggregation + query field-resolution law', () => {
+  let brain: Brainy
+
+  beforeEach(async () => {
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' as const },
+      embeddingFunction: stubEmbedding
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+  })
+
+  it('reserved-field groupBy decrements on delete (the drift bug)', async () => {
+    brain.defineAggregate({
+      name: 'by_subtype',
+      source: { type: NounType.Document },
+      groupBy: ['subtype'],
+      metrics: { count: { op: 'count' } }
+    })
+
+    const ids: string[] = []
+    for (let i = 0; i < 5; i++) {
+      ids.push(
+        await brain.add({
+          data: `doc-${i}`,
+          type: NounType.Document,
+          subtype: 'note',
+          metadata: { team: 'alpha' }
+        })
+      )
+    }
+    let groups = await brain.queryAggregate('by_subtype')
+    expect(groups).toHaveLength(1)
+    expect(groups[0].groupKey).toEqual({ subtype: 'note' })
+    expect(groups[0].metrics.count).toBe(5)
+
+    await brain.remove(ids[0])
+    await brain.flush()
+
+    groups = await brain.queryAggregate('by_subtype')
+    expect(groups[0].metrics.count).toBe(4)
+    const live = await brain.find({ type: NounType.Document, limit: 100 })
+    expect(groups[0].metrics.count).toBe(live.length)
+  })
+
+  it('reserved-field groupBy moves between groups on update (no double-count)', async () => {
+    brain.defineAggregate({
+      name: 'by_subtype',
+      source: { type: NounType.Document },
+      groupBy: ['subtype'],
+      metrics: { count: { op: 'count' } }
+    })
+    const id = await brain.add({
+      data: 'doc-move',
+      type: NounType.Document,
+      subtype: 'draft'
+    })
+    await brain.update({ id, subtype: 'published' })
+
+    const groups = await brain.queryAggregate('by_subtype')
+    const byKey = Object.fromEntries(
+      groups.map((g) => [String(g.groupKey.subtype), g.metrics.count])
+    )
+    expect(byKey['published']).toBe(1)
+    // The old group must be gone or zero — never still counting the entity.
+    expect(byKey['draft'] ?? 0).toBe(0)
+  })
+
+  it('source.where on a reserved field filters instead of matching nothing', async () => {
+    brain.defineAggregate({
+      name: 'notes_only',
+      source: { type: NounType.Document, where: { subtype: 'note' } },
+      groupBy: ['team'],
+      metrics: { count: { op: 'count' } }
+    })
+    await brain.add({
+      data: 'n1',
+      type: NounType.Document,
+      subtype: 'note',
+      metadata: { team: 'alpha' }
+    })
+    await brain.add({
+      data: 'd1',
+      type: NounType.Document,
+      subtype: 'draft',
+      metadata: { team: 'alpha' }
+    })
+
+    const groups = await brain.queryAggregate('notes_only')
+    expect(groups).toHaveLength(1)
+    expect(groups[0].metrics.count).toBe(1) // the note, never the draft
+  })
+
+  it('removeMany refuses empty/invalid selectors loudly', async () => {
+    const id = await brain.add({ data: 'keep-me', type: NounType.Document })
+
+    // Bare array passed positionally — the classic silent no-op.
+    await expect(
+      brain.removeMany([id] as unknown as Parameters<typeof brain.removeMany>[0])
+    ).rejects.toThrow(/bare array/)
+    // Empty selector object.
+    await expect(
+      brain.removeMany({} as Parameters<typeof brain.removeMany>[0])
+    ).rejects.toThrow(/requires a selector/)
+    // Explicit empty id list.
+    await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
+
+    // Nothing was deleted by any of the refused calls.
+    expect(await brain.get(id)).toBeTruthy()
+  })
+
+  it('find() accepts both flattened and metadata.-prefixed where spellings', async () => {
+    await brain.add({
+      data: 'nested-doc',
+      type: NounType.Document,
+      metadata: { entry: { title: 'T1' }, classifier: { contextHints: { vfsPath: '/n/a.md' } } }
+    })
+    await brain.flush()
+
+    const flat = await brain.find({
+      type: NounType.Document,
+      where: { 'entry.title': 'T1' },
+      limit: 10
+    })
+    const prefixed = await brain.find({
+      type: NounType.Document,
+      where: { 'metadata.entry.title': 'T1' },
+      limit: 10
+    })
+    const deepPrefixed = await brain.find({
+      type: NounType.Document,
+      where: { 'metadata.classifier.contextHints.vfsPath': '/n/a.md' },
+      limit: 10
+    })
+    expect(flat).toHaveLength(1)
+    expect(prefixed).toHaveLength(1)
+    expect(prefixed[0].id).toBe(flat[0].id)
+    expect(deepPrefixed).toHaveLength(1)
+  })
+})

tests/integration/aggregation-state-persistence.test.ts

@@ -209,6 +209,82 @@ describe('aggregation state persistence — boot-order contract', () => {
     await brain2.close()
   })
 
+  it('generation-mismatched persisted state is rescanned once, loudly — never adopted', async () => {
+    const brain1 = await open()
+    brain1.defineAggregate(SPENDING)
+    await seed(brain1)
+    await brain1.queryAggregate('spending')
+    await brain1.close()
+
+    // Simulate the copied-store incident class: a fact-log truncation (or an
+    // unclean shutdown) leaves the committed watermark different from the
+    // generation the flushed state was stamped with.
+    const tamper: any = await open()
+    const key = '__aggregation_state_spending__'
+    const stored = await tamper.storage.getMetadata(key)
+    expect(typeof stored.sourceGeneration).toBe('number') // the stamp is really persisted
+    await tamper.storage.saveMetadata(key, {
+      ...stored,
+      sourceGeneration: stored.sourceGeneration + 5
+    })
+    await tamper.close()
+
+    const warnSpy = vi.spyOn(prodLog, 'warn')
+    const brain2 = await open()
+    brain2.defineAggregate(SPENDING)
+    await brain2.getNounCount()
+    const walks = countWalks(brain2)
+
+    const rows = await brain2.queryAggregate('spending')
+
+    expect(walks.count()).toBe(1) // exactly ONE rescan — no silent adopt, no spin
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(6) // rescan produced exact results
+    expect(
+      warnSpy.mock.calls.some(args => String(args[0]).includes('rescanning instead of adopting'))
+    ).toBe(true) // and it said so out loud
+    warnSpy.mockRestore()
+    await brain2.close()
+  })
+
+  it('a failing walk is loud, non-destructive, and latched — never a silent retry loop', async () => {
+    // Fresh define + seeded writes: the write hooks have populated LIVE state,
+    // and the first-query rescan is still pending. The incident shape
+    // (wipe-before-scan + no try/catch + per-query re-walk) would have wiped
+    // that live state and silently re-walked on every query.
+    const brain: any = await open()
+    brain.defineAggregate(SPENDING)
+    await seed(brain)
+    expect(brain._aggregationIndex.queryAggregate({ name: 'spending' }).length).toBe(2)
+
+    const storage = brain.storage
+    const origGetNouns = storage.getNouns.bind(storage)
+    let walkAttempts = 0
+    storage.getNouns = async () => {
+      walkAttempts++
+      throw new Error('injected storage failure')
+    }
+
+    // First query: the walk fails LOUDLY with the storage error.
+    await expect(brain.queryAggregate('spending')).rejects.toThrow('injected storage failure')
+    expect(walkAttempts).toBe(1)
+
+    // Live state was NOT destroyed by the failed walk (staging was dropped).
+    expect(brain._aggregationIndex.queryAggregate({ name: 'spending' }).length).toBe(2)
+
+    // Second query inside the cooldown: instant loud failure, NO new walk.
+    await expect(brain.queryAggregate('spending')).rejects.toThrow('failure cooldown')
+    expect(walkAttempts).toBe(1)
+
+    // Heal the storage + expire the cooldown: one fresh walk succeeds exactly.
+    storage.getNouns = origGetNouns
+    brain._aggregationBackfillFailure.at = Date.now() - 60_000
+    const rows = await brain.queryAggregate('spending')
+    const food = rows.find((r: any) => r.groupKey.category === 'food')
+    expect(food.metrics.count).toBe(6)
+    await brain.close()
+  })
+
   it('aggregation persistence keys never log "Unknown key format"', async () => {
     const warnSpy = vi.spyOn(prodLog, 'warn')
     const brain1 = await open()

tests/integration/background-dedup-lifecycle.test.ts

@@ -0,0 +1,85 @@
+/**
+ * @module tests/integration/background-dedup-lifecycle
+ * @description The post-import background deduplication pass (a merge-DELETE
+ * writer) obeys the same contract as the inline pass. Laws:
+ * (1) enableDeduplication:false schedules NO background pass — the brain-owned
+ *     deduplicator is never even constructed;
+ * (2) by default the pass IS scheduled, brain-owned, with an unref'd timer
+ *     (a pending pass never holds the process open);
+ * (3) repeated imports debounce into ONE pending batch on ONE instance
+ *     (per-coordinator instances used to arm one timer per import);
+ * (4) close() cancels pending work — no delete pass can fire after close.
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import { Brainy } from '../../src/brainy.js'
+
+const ROWS = [
+  { name: 'Alice Zephyr', role: 'engineer' },
+  { name: 'Bob Quill', role: 'writer' }
+]
+
+// Keep imports fast and deterministic — dedup scheduling is what's under test.
+const FAST = {
+  enableNeuralExtraction: false,
+  enableRelationshipInference: false,
+  enableConceptExtraction: false
+} as const
+
+// Deterministic stub embedder (hnsw-rebuild.test.ts pattern) — dedup
+// scheduling never inspects vector CONTENT, so skip the WASM model load.
+const stubEmbedding = async (text: string): Promise<number[]> => {
+  const hash = text.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0)
+  const vector = new Array(384).fill(0).map((_, i) => Math.sin(hash + i))
+  return vector
+}
+
+describe('background dedup lifecycle', () => {
+  let brain: Brainy
+
+  beforeEach(async () => {
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' as const },
+      embeddingFunction: stubEmbedding
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close()
+  })
+
+  it('enableDeduplication:false schedules no background pass at all', async () => {
+    await brain.import(ROWS, { ...FAST, enableDeduplication: false })
+    expect((brain as any)._backgroundDedup).toBeUndefined()
+  })
+
+  it('default schedules a brain-owned pass with an unref-ed timer', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const dedup = (brain as any)._backgroundDedup
+    expect(dedup).toBeDefined()
+    expect(dedup.pendingImports.size).toBe(1)
+    const timer = dedup.debounceTimer
+    expect(timer).toBeDefined()
+    // Node timers expose hasRef(); an unref'd timer must not hold the process.
+    expect(typeof timer.hasRef).toBe('function')
+    expect(timer.hasRef()).toBe(false)
+  })
+
+  it('imports debounce into one pending batch on one brain-owned instance', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const first = (brain as any)._backgroundDedup
+    await brain.import([{ name: 'Cara Vex', role: 'analyst' }], { ...FAST })
+    expect((brain as any)._backgroundDedup).toBe(first)
+    expect(first.pendingImports.size).toBe(2)
+  })
+
+  it('close() cancels pending background dedup', async () => {
+    await brain.import(ROWS, { ...FAST })
+    const dedup = (brain as any)._backgroundDedup
+    expect(dedup.debounceTimer).toBeDefined()
+    await brain.close()
+    expect(dedup.debounceTimer).toBeUndefined()
+    expect(dedup.pendingImports.size).toBe(0)
+  })
+})

tests/integration/db-mvcc.test.ts

@@ -1280,24 +1280,32 @@ describe('8.0 Db API — generational MVCC', () => {
     await expect(reopened.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
   })
 
-  it('Model-B retention — setRetentionBudget drives adaptive reclaim on flush; live data intact', async () => {
+  it('Model-B retention — flush() NEVER compacts (8.9.0); adaptive reclaim runs at close()', async () => {
-    // Default brain → ADAPTIVE retention. A coordinator (e.g. cor's ResourceManager)
+    // Default brain → ADAPTIVE retention with a driven byte budget far below
-    // pushes a byte budget via setRetentionBudget(); auto-compaction on flush() reclaims
+    // the accumulated history (~13 generations of full-vector before-images).
-    // oldest history down toward it. Each update's before-image carries the full prior
+    // The 8.9.0 law: flush() is durability-only — it must not reclaim even
-    // 384-dim vector (~KBs), so ~13 generations far exceed a few-KB budget.
+    // when the budget is exceeded (reclaim-on-flush blocked production writes
-    const { brain } = await openFsBrain()
+    // for 25-191s). Maintenance runs at close(), time-bounded.
+    const { brain, dir } = await openFsBrain()
     const a = uid('ret-budget')
     await brain.add({ id: a, type: NounType.Document, data: 'v0', vector: vec(1), metadata: { v: 0 } })
     for (let v = 1; v <= 12; v++) await brain.update({ id: a, metadata: { v } })
 
     brain.setRetentionBudget(6000) // ~6 KB — well below the accumulated history
-    await brain.flush() // group-commit + adaptive auto-compaction under the budget
+    await brain.flush()
 
-    // History was reclaimed (the horizon advanced past the oldest generations)…
+    // flush() paid durability only: nothing reclaimed, all history readable.
-    expect(generationStoreOf(brain).horizon()).toBeGreaterThan(0)
+    expect(generationStoreOf(brain).horizon()).toBe(0)
-    await expect(brain.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    const probe = await brain.asOf(1) // readable proves nothing was reclaimed…
-    // …but the budget reclaims HISTORY only — the live record is untouched.
+    await probe.release() // …and MUST be released: a held pin would (correctly)
-    expect((await brain.get(a))?.metadata?.v).toBe(12)
+    // protect every newer generation through the close() compaction below.
+    await brain.close() // ← THE auto-compaction site now
+
+    // close() reclaimed under the budget; live record intact; horizon durable.
+    const { brain: reopened } = await openFsBrain(dir)
+    expect(generationStoreOf(reopened).horizon()).toBeGreaterThan(0)
+    await expect(reopened.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    expect((await reopened.get(a))?.metadata?.v).toBe(12)
   })
 
   // ==========================================================================

tests/integration/graph-audit.test.ts

@@ -0,0 +1,164 @@
+/**
+ * @module tests/integration/graph-audit
+ * @description brain.auditGraph() — the read-only graph-truth instrument.
+ * Laws under test:
+ * (1) a healthy brain audits COHERENT: every canonical verb is returned by the
+ *     read path of its source, endpoints exist, no ghosts;
+ * (2) design-hidden edges (internal/system visibility) are counted separately
+ *     and never misclassified as index loss;
+ * (3) a verb whose endpoint entity was destroyed at the storage layer (the
+ *     scar class) is flagged as a dangling endpoint, loudly;
+ * (4) the classification core flags present-but-invisible and ghost edges
+ *     exactly (exercised via injected seams — manufacturing a genuinely stale
+ *     adjacency index end-to-end would require corrupting internals the
+ *     public API rightly refuses to corrupt).
+ */
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../src/index.js'
+import { NounType, VerbType } from '../../src/types/graphTypes.js'
+import { runGraphAudit, type AuditVerbRecord } from '../../src/graph/graphAudit.js'
+
+describe('brain.auditGraph() — graph-truth audit', () => {
+  let dir: string
+  let brain: any
+
+  beforeEach(async () => {
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-graph-audit-'))
+    brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true
+    })
+    await brain.init()
+  })
+
+  afterEach(async () => {
+    await brain.close().catch(() => {})
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  async function seedGraph(): Promise<{ ids: string[]; verbIds: string[] }> {
+    const ids: string[] = []
+    for (let i = 0; i < 5; i++) {
+      ids.push(
+        await brain.add({
+          data: `entity ${i}`,
+          type: NounType.Concept,
+          metadata: { n: i }
+        })
+      )
+    }
+    const verbIds: string[] = []
+    verbIds.push(await brain.relate({ from: ids[0], to: ids[1], type: VerbType.RelatedTo }))
+    verbIds.push(await brain.relate({ from: ids[0], to: ids[2], type: VerbType.Contains }))
+    verbIds.push(await brain.relate({ from: ids[1], to: ids[3], type: VerbType.DependsOn }))
+    verbIds.push(await brain.relate({ from: ids[3], to: ids[4], type: VerbType.RelatedTo }))
+    return { ids, verbIds }
+  }
+
+  it('audits a healthy brain as coherent, with exact counts', async () => {
+    const { ids } = await seedGraph()
+    const report = await brain.auditGraph()
+
+    expect(report.coherent).toBe(true)
+    expect(report.verbsInCanonical).toBe(4)
+    expect(report.entitiesInCanonical).toBeGreaterThanOrEqual(ids.length) // VFS root etc. may add system nouns
+    expect(report.sourcesChecked).toBe(3) // ids[0], ids[1], ids[3]
+    expect(report.missingFromReadsCount).toBe(0)
+    expect(report.danglingEndpointsCount).toBe(0)
+    expect(report.readOnlyCount).toBe(0)
+    expect(report.truncatedExamples).toBe(false)
+  })
+
+  it('counts design-hidden edges separately and stays coherent', async () => {
+    const { ids } = await seedGraph()
+    await brain.relate({
+      from: ids[2],
+      to: ids[4],
+      type: VerbType.RelatedTo,
+      visibility: 'internal'
+    })
+
+    const report = await brain.auditGraph()
+    expect(report.coherent).toBe(true) // hidden-by-design is NOT a discrepancy
+    expect(report.verbsInCanonical).toBe(5)
+    expect(report.visibilityHiddenCount).toBeGreaterThanOrEqual(1)
+  })
+
+  it('flags a destroyed endpoint as a dangling verb (the scar class)', async () => {
+    const { ids } = await seedGraph()
+    // Destroy ids[4] at the STORAGE layer (bypassing remove(), which would
+    // also delete its verbs) — the historical partial-delete scar shape.
+    await brain.storage.deleteNoun(ids[4])
+
+    const report = await brain.auditGraph()
+    expect(report.coherent).toBe(false)
+    expect(report.danglingEndpointsCount).toBe(1)
+    expect(report.danglingEndpoints[0].to).toBe(ids[4])
+    expect(report.danglingEndpoints[0].missingEnd).toBe('to')
+  })
+})
+
+describe('runGraphAudit classification core (injected seams)', () => {
+  const verb = (id: string, from: string, to: string): AuditVerbRecord => ({
+    id,
+    type: 'relatedTo',
+    sourceId: from,
+    targetId: to
+  })
+
+  const deps = (opts: {
+    nouns: string[]
+    verbs: AuditVerbRecord[]
+    reads: Record<string, string[]> // sourceId -> verb ids the read path returns
+  }) => ({
+    eachNounId: async (consume: (id: string) => void) => {
+      for (const id of opts.nouns) consume(id)
+    },
+    eachVerb: async (consume: (v: AuditVerbRecord) => void) => {
+      for (const v of opts.verbs) consume(v)
+    },
+    readRelationsFrom: async (sourceId: string) =>
+      (opts.reads[sourceId] ?? []).map((id) => ({ id }))
+  })
+
+  it('flags a canonical verb the read path omits — present but invisible', async () => {
+    const report = await runGraphAudit(
+      deps({
+        nouns: ['A', 'B', 'C'],
+        verbs: [verb('v1', 'A', 'B'), verb('v2', 'A', 'C')],
+        reads: { A: ['v1'] } // v2 exists canonically but reads miss it
+      })
+    )
+    expect(report.coherent).toBe(false)
+    expect(report.missingFromReadsCount).toBe(1)
+    expect(report.missingFromReads[0].verbId).toBe('v2')
+  })
+
+  it('flags a read-path edge with no canonical record — a ghost', async () => {
+    const report = await runGraphAudit(
+      deps({
+        nouns: ['A', 'B'],
+        verbs: [verb('v1', 'A', 'B')],
+        reads: { A: ['v1', 'ghost-9'] }
+      })
+    )
+    expect(report.coherent).toBe(false)
+    expect(report.readOnlyCount).toBe(1)
+    expect(report.readOnlyVerbIds).toEqual(['ghost-9'])
+  })
+
+  it('caps example lists but keeps counts exact, and says so', async () => {
+    const verbs = Array.from({ length: 10 }, (_, i) => verb(`v${i}`, 'A', 'B'))
+    const report = await runGraphAudit(
+      deps({ nouns: ['A', 'B'], verbs, reads: { A: [] } }),
+      { maxExamples: 3 }
+    )
+    expect(report.missingFromReadsCount).toBe(10)
+    expect(report.missingFromReads.length).toBe(3)
+    expect(report.truncatedExamples).toBe(true)
+  })
+})

tests/integration/history-repacking.test.ts

@@ -0,0 +1,186 @@
+/**
+ * @module tests/integration/history-repacking
+ * @description The D1+D3 two-tier history lifecycle end-to-end on a real
+ * brain. Laws: (1) repacking is RE-REPRESENTATION — after folding, every
+ * asOf() read below the fold boundary answers exactly as before, across a
+ * cold reopen; (2) folded per-generation directories are physically gone
+ * (the file-count cure is real, not cosmetic); (3) repack + reclaim compose:
+ * bounded retention after repacking drops whole segments and asOf below the
+ * horizon throws GenerationCompactedError; (4) repackHistory is explicit
+ * API and time-bounded (spent budget = consistent no-op).
+ *
+ * Uses a tiny REPACK_LIVE_WINDOW override so a small history has a cold
+ * tier at all (the production window is 1024).
+ */
+import { describe, it, expect, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as path from 'node:path'
+import * as os from 'node:os'
+import { Brainy } from '../../src/brainy.js'
+import { NounType } from '../../src/types/graphTypes.js'
+import { GenerationStore } from '../../src/db/generationStore.js'
+import { GenerationCompactedError } from '../../src/db/errors.js'
+import { SEGMENTS_PREFIX } from '../../src/db/generationSegments.js'
+
+const stub = async (text: string): Promise<number[]> => {
+  const h = text.split('').reduce((a, c) => a + c.charCodeAt(0), 0)
+  return new Array(384).fill(0).map((_, i) => Math.sin(h + i))
+}
+
+const openBrain = async (dir: string): Promise<Brainy> => {
+  const brain = new Brainy({
+    requireSubtype: false,
+    storage: { type: 'filesystem', path: dir },
+    embeddingFunction: stub
+  })
+  await brain.init()
+  return brain
+}
+
+describe('history repacking — the two-tier lifecycle', () => {
+  const dirs: string[] = []
+  const tempDir = (): string => {
+    const d = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-repack-'))
+    dirs.push(d)
+    return d
+  }
+  const originalWindow = GenerationStore.REPACK_LIVE_WINDOW
+
+  afterEach(() => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = originalWindow
+    for (const d of dirs.splice(0)) {
+      try {
+        fs.rmSync(d, { recursive: true, force: true })
+      } catch {
+        /* best effort */
+      }
+    }
+  })
+
+  it('repack preserves every historical read across cold reopen; folded dirs are gone', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 3
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+
+    const id = await brain.add({
+      data: 'versioned-entity',
+      type: NounType.Document,
+      metadata: { v: 0 }
+    })
+    for (let v = 1; v <= 10; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+
+    // Ground truth BEFORE repacking: capture asOf views for early generations.
+    const before: Record<number, number> = {}
+    for (const g of [2, 4, 6]) {
+      const db = await brain.asOf(g)
+      before[g] = (await db.get(id))?.metadata?.v as number
+      await db.release()
+    }
+
+    const result = await brain.repackHistory()
+    expect(result.foldedGenerations).toBeGreaterThan(0)
+    expect(result.segmentsCreated).toBeGreaterThan(0)
+
+    // The folded per-generation directories are PHYSICALLY gone…
+    const genDirs = fs
+      .readdirSync(path.join(dir, '_generations'), { withFileTypes: true })
+      .filter((e) => e.isDirectory() && /^\d+$/.test(e.name)).length
+    expect(genDirs).toBeLessThanOrEqual(4) // live window (3) + at most the newest
+    // …and the segment tier exists (the filesystem adapter stores objects
+    // gzipped, so the manifest may live at either spelling).
+    const segDir = path.join(dir, SEGMENTS_PREFIX)
+    expect(
+      fs.existsSync(path.join(segDir, 'manifest.json')) ||
+        fs.existsSync(path.join(segDir, 'manifest.json.gz'))
+    ).toBe(true)
+    expect(fs.readdirSync(segDir).some((f) => f.endsWith('.bgs'))).toBe(true)
+
+    // Same asOf answers from the packed tier, same process…
+    for (const g of [2, 4, 6]) {
+      const db = await brain.asOf(g)
+      expect((await db.get(id))?.metadata?.v).toBe(before[g])
+      await db.release()
+    }
+    await brain.close()
+
+    // …and across a COLD REOPEN (manifest discovery, no live dirs to list).
+    const reopened = await openBrain(dir)
+    for (const g of [2, 4, 6]) {
+      const db = await reopened.asOf(g)
+      expect((await db.get(id))?.metadata?.v).toBe(before[g])
+      await db.release()
+    }
+    expect((await reopened.get(id))?.metadata?.v).toBe(10) // live state untouched
+    await reopened.close()
+  })
+
+  it('repack + bounded reclaim compose: whole segments drop, horizon is loud', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'reclaim-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 8; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+    await brain.repackHistory()
+
+    // Reclaim down to the 3 newest generations — packed segments below the
+    // horizon drop whole; asOf below throws loudly.
+    const res = await brain.compactHistory({ maxGenerations: 3 })
+    expect(res.removedGenerations).toBeGreaterThan(0)
+    await expect(brain.asOf(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    expect((await brain.get(id))?.metadata?.v).toBe(8)
+    await brain.close()
+  })
+
+  it('generationDigest: reopen-stable, divergence-sensitive, loud below the horizon', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'digest-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 6; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+    await brain.repackHistory()
+
+    const gen = brain.generation()
+    const atHead = await brain.generationDigest(gen)
+    const atMid = await brain.generationDigest(3)
+    expect(atHead).toMatch(/^[0-9a-f]{8}$/)
+    expect(atMid).not.toBe(atHead) // more history ⇒ different digest
+    await brain.close()
+
+    // Reopen-stable: same history, same digests (packed prefix stability).
+    const reopened = await openBrain(dir)
+    expect(await reopened.generationDigest(gen)).toBe(atHead)
+    expect(await reopened.generationDigest(3)).toBe(atMid)
+
+    // New history diverges the head digest.
+    await reopened.update({ id, metadata: { v: 7 } })
+    await reopened.flush()
+    expect(await reopened.generationDigest(reopened.generation())).not.toBe(atHead)
+
+    // Below the horizon: LOUD, never a silent pin of reclaimed history.
+    await reopened.compactHistory({ maxGenerations: 2 })
+    await expect(reopened.generationDigest(1)).rejects.toBeInstanceOf(GenerationCompactedError)
+    await reopened.close()
+  })
+
+  it('a spent time budget is a consistent no-op; the next pass resumes', async () => {
+    ;(GenerationStore as any).REPACK_LIVE_WINDOW = 2
+    const dir = tempDir()
+    const brain = await openBrain(dir)
+    const id = await brain.add({ data: 'budget-probe', type: NounType.Document, metadata: { v: 0 } })
+    for (let v = 1; v <= 6; v++) await brain.update({ id, metadata: { v } })
+    await brain.flush()
+
+    const bounded = await brain.repackHistory({ timeBudgetMs: 0 })
+    expect(bounded).toEqual({ foldedGenerations: 0, segmentsCreated: 0 })
+
+    const resumed = await brain.repackHistory()
+    expect(resumed.foldedGenerations).toBeGreaterThan(0)
+    const db = await brain.asOf(3)
+    expect((await db.get(id))?.metadata?.v).toBeDefined()
+    await db.release()
+    await brain.close()
+  })
+})

tests/integration/multi-process-safety.test.ts

@@ -110,6 +110,89 @@ describe('Multi-process safety + read-only mode', () => {
       // Don't track `blocked` for afterEach cleanup since init failed.
     })
 
+    it('takes over a STALE foreign lock (dead PID + old heartbeat) and claims atomically', async () => {
+      const { mkdirSync, writeFileSync, readFileSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      const os = await import('node:os')
+      mkdirSync(join(dir, 'locks'), { recursive: true })
+      const tenMinutesAgo = new Date(Date.now() - 10 * 60 * 1000).toISOString()
+      writeFileSync(join(dir, 'locks', '_writer.lock'), JSON.stringify({
+        pid: 999999999, // no such process — provably dead
+        hostname: os.hostname(),
+        startedAt: tenMinutesAgo,
+        lastHeartbeat: tenMinutesAgo,
+        version: '8.0.0',
+        rootDir: dir
+      }))
+
+      writer = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await writer.init() // stale takeover must succeed
+
+      const lock = JSON.parse(readFileSync(join(dir, 'locks', '_writer.lock'), 'utf-8'))
+      expect(lock.pid).toBe(process.pid) // the atomic wx claim installed OUR lock
+    })
+
+    it('the writer-locked error carries the machine-readable contract (code + lockInfo)', async () => {
+      const { mkdirSync, writeFileSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      const os = await import('node:os')
+      mkdirSync(join(dir, 'locks'), { recursive: true })
+      const otherPid = (process as any).ppid || 1
+      writeFileSync(join(dir, 'locks', '_writer.lock'), JSON.stringify({
+        pid: otherPid,
+        hostname: os.hostname(),
+        startedAt: new Date().toISOString(),
+        lastHeartbeat: new Date().toISOString(),
+        version: '8.7.0',
+        rootDir: dir
+      }))
+
+      const blocked = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      const err: any = await blocked.init().catch((e) => e)
+      expect(err.code).toBe('BRAINY_WRITER_LOCKED')
+      expect(err.lockInfo?.pid).toBe(otherPid)
+    })
+
+    it('release drains an in-flight heartbeat — no phantom lock re-created after unlink', async () => {
+      // The race (8.9.0): clearInterval stops FUTURE heartbeat ticks, but a
+      // tick already in flight could land its lock rewrite AFTER release's
+      // unlink — re-creating the lock as a phantom that blocks the next
+      // writer until the stale TTL. Simulate the in-flight tick explicitly
+      // and prove release waits for it.
+      writer = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await writer.init()
+      const storage: any = (writer as any).storage
+
+      // An in-flight refresh that is ALREADY PAST its ownership guards
+      // (captured the lock info before release ran) and lands its atomic
+      // rewrite slowly — the exact straggler shape; absent the drain it
+      // writes after the unlink.
+      const { join: joinPath } = await import('node:path')
+      const capturedInfo = { ...storage.writerLockInfo }
+      const lockPath = joinPath(dir, 'locks', '_writer.lock')
+      const slowTick = (async () => {
+        await new Promise((r) => setTimeout(r, 100))
+        await storage.writeFileAtomic(
+          lockPath,
+          JSON.stringify({ ...capturedInfo, lastHeartbeat: new Date().toISOString() })
+        )
+      })()
+      storage.writerHeartbeatInFlight = slowTick.catch(() => {})
+
+      await writer.close() // → releaseWriterLock must drain slowTick first
+      await slowTick.catch(() => {}) // both paths fully settled either way
+      writer = null
+
+      const { existsSync } = await import('node:fs')
+      const { join } = await import('node:path')
+      expect(existsSync(join(dir, 'locks', '_writer.lock'))).toBe(false)
+
+      // And the directory is immediately claimable — no stale-TTL wait.
+      const next = new Brainy({ requireSubtype: false, storage: { type: 'filesystem', path: dir } })
+      await expect(next.init()).resolves.toBeUndefined()
+      await next.close()
+    })
+
     it('allows a second in-process writer with a warning (same PID)', async () => {
       // Two Brainy instances in the same Node process: not the dangerous
       // cross-process case. Should succeed (with a console warning).

tests/regression/metadata-index-cleanup.unit.test.ts

@@ -205,10 +205,10 @@ describe('Metadata index cleanup after remove / removeMany', () => {
       }
     })
 
-    it('handles empty ids array gracefully', async () => {
+    it('refuses an empty ids array loudly (a silent no-op is not "graceful")', async () => {
-      const result = await brain.removeMany({ ids: [] })
+      // 8.8.2: an empty selector used to resolve successfully having deleted
-      expect(result.successful).toHaveLength(0)
+      // NOTHING — the caller believed the delete happened. Now it throws.
-      expect(result.failed).toHaveLength(0)
+      await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
     })
 
     it('handles large batch (> 1 chunk) without leaving stale index entries', async () => {

tests/transaction/TransactionManager.unit.test.ts

@@ -10,6 +10,7 @@
 
 import { describe, it, expect, beforeEach } from 'vitest'
 import { TransactionManager } from '../../src/transaction/TransactionManager.js'
+import { transactTimeoutBudget } from '../../src/transaction/Transaction.js'
 import { TransactionError } from '../../src/transaction/errors.js'
 
 describe('TransactionManager', () => {
@@ -328,4 +329,45 @@ describe('TransactionManager', () => {
       expect(stats1).toEqual(stats2) // Same values
     })
   })
+
+  describe('Timeout budget + telemetry', () => {
+    it('transactTimeoutBudget: explicit override wins; default scales with batch size', () => {
+      expect(transactTimeoutBudget(1)).toBe(30_000) // small batches keep the 30s floor
+      expect(transactTimeoutBudget(15)).toBe(30_000) // the old flat cap's break-even point
+      expect(transactTimeoutBudget(100)).toBe(200_000) // 100 ops × 2s — bulk gets an honest budget
+      expect(transactTimeoutBudget(1000, 5_000)).toBe(5_000) // caller override is untouched
+    })
+
+    it('a tripped budget rolls back and names the operation, progress, and budget', async () => {
+      const rolledBack: string[] = []
+
+      const failing = manager.executeTransaction(
+        async (tx) => {
+          tx.addOperation({
+            name: 'slow-first-op',
+            execute: async () => {
+              await new Promise((r) => setTimeout(r, 30))
+              return async () => {
+                rolledBack.push('slow-first-op')
+              }
+            }
+          })
+          tx.addOperation({
+            name: 'never-reached',
+            execute: async () => undefined
+          })
+        },
+        { timeout: 5 } // the first op's 30ms sleep guarantees the pre-op-2 check trips
+      )
+
+      await expect(failing).rejects.toThrow(TransactionError)
+      const err = await failing.catch((e) => e)
+      expect(err.name).toBe('TransactionTimeoutError')
+      expect(err.message).toContain('operation 1/2') // which op, of how many
+      expect(err.message).toContain("('never-reached')") // its name
+      expect(err.message).toContain('budget 5ms') // the budget that tripped
+      expect(err.message).toContain('rolled back') // the retryability statement
+      expect(rolledBack).toEqual(['slow-first-op']) // the applied op was undone
+    })
+  })
 })

tests/unit/brainy/batch-operations.test.ts

@@ -533,8 +533,10 @@ describe('Brainy Batch Operations', () => {
       expect(result.successful).toHaveLength(0)
       
       await brain.updateMany({ items: [] })
-      await brain.removeMany({ ids: [] })
+      // removeMany is the exception (8.8.2): an empty id list is a refused
-      // Should not throw
+      // selector, not an empty batch — deleting "nothing" silently was the
+      // bug class (a positional/bare-array call looked identical).
+      await expect(brain.removeMany({ ids: [] })).rejects.toThrow(/ids: \[\]/)
     })
     
     it('should validate batch size limits', async () => {

tests/unit/brainy/warm.test.ts

@@ -0,0 +1,310 @@
+/**
+ * @module tests/unit/brainy/warm
+ * @description Coverage for `brain.warm()` / `warmOnOpen` / the provider
+ * `warm?()` contract (the cold-restart readiness fix): first operations after
+ * a cold restart should run at steady-state cost instead of paying
+ * demand-load latency on the critical path.
+ *
+ * Uses a real, in-process fake plugin provider implementing the actual
+ * `VectorIndexProvider` contract from `src/plugin.ts` — the real seam a
+ * native provider (e.g. a disk-native accelerator) plugs into. The real
+ * built-in JS metadata index and graph adjacency index run against real
+ * (filesystem or in-memory) storage with pre-existing data, so their
+ * hydration paths are exercised for real, not mocked.
+ */
+import { describe, it, expect, afterEach } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { Brainy } from '../../../src/brainy.js'
+import { NounType, VerbType } from '../../../src/types/graphTypes.js'
+import type { VectorIndexProvider } from '../../../src/plugin.js'
+import type { VectorDocument, Vector } from '../../../src/coreTypes.js'
+import { MetadataIndexManager } from '../../../src/utils/metadataIndex.js'
+import { GraphAdjacencyIndex } from '../../../src/graph/graphAdjacencyIndex.js'
+
+const tmpDirs: string[] = []
+function mkTmp(): string {
+  const d = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-warm-'))
+  tmpDirs.push(d)
+  return d
+}
+afterEach(() => {
+  for (const d of tmpDirs.splice(0)) fs.rmSync(d, { recursive: true, force: true })
+})
+
+// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions
+// (src/utils/paramValidation.ts) — match it so `add()` doesn't reject test data.
+const DIM = 384
+const V = (seed = 1): number[] => Array.from({ length: DIM }, (_, i) => Math.sin(seed + i))
+
+/**
+ * A real (not mocked) VectorIndexProvider implementation, backed by a plain
+ * Map, that optionally implements `warm()` — the exact seam
+ * `AddToVectorIndexOperation` / `brain.warm()` call through.
+ */
+class FakeVectorProvider implements VectorIndexProvider {
+  readonly name = 'fake-vector-provider'
+  readonly items = new Map<string, Vector>()
+  warmCalls = 0
+  searchCalls: Array<{ k?: number }> = []
+  // Only present on the instance when the constructor is told to — mirrors a
+  // real provider that may or may not implement the optional hook. Assigned
+  // in the constructor BODY (not as a field initializer): under native ES
+  // class fields, field initializers run before constructor-body statements
+  // — including the parameter-property assignment — so referencing a
+  // parameter property from a field initializer would see it as still
+  // `undefined`.
+  warm?: () => Promise<void>
+
+  constructor(hasWarm: boolean) {
+    if (hasWarm) {
+      this.warm = async (): Promise<void> => {
+        this.warmCalls++
+      }
+    }
+  }
+
+  async addItem(item: VectorDocument): Promise<string> {
+    this.items.set(item.id, item.vector)
+    return item.id
+  }
+  async removeItem(id: string): Promise<boolean> {
+    return this.items.delete(id)
+  }
+  async search(_queryVector: Vector, k?: number): Promise<Array<[string, number]>> {
+    this.searchCalls.push({ k })
+    return [...this.items.keys()].slice(0, k ?? 10).map((id) => [id, 0])
+  }
+  size(): number {
+    return this.items.size
+  }
+  clear(): void {
+    this.items.clear()
+  }
+  async rebuild(): Promise<void> {}
+  async flush(): Promise<number> {
+    return 0
+  }
+  getPersistMode(): 'immediate' | 'deferred' {
+    return 'deferred'
+  }
+}
+
+/** Registers `provider` under the `'vector'` plugin key, before `init()`. */
+function useFakeVectorProvider(brain: Brainy<any>, provider: FakeVectorProvider): void {
+  brain.use({
+    name: 'fake-vector-provider',
+    activate: async (ctx: any) => {
+      ctx.registerProvider('vector', () => provider)
+      return true
+    }
+  })
+}
+
+describe('brain.warm()', () => {
+  it('(a) calls the vector provider\'s warm() when present and reports "warmed"', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      silent: true
+    })
+    const provider = new FakeVectorProvider(true)
+    useFakeVectorProvider(brain, provider)
+    await brain.init()
+    await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
+
+    const report = await brain.warm()
+
+    expect(provider.warmCalls).toBe(1)
+    expect(provider.searchCalls.length).toBe(0) // warm() ran — no probe fallback
+    expect(report.vector.outcome).toBe('warmed')
+    await brain.close()
+  })
+
+  it('(b) falls back to a probe search() when warm() is absent and reports "probed", never "warmed"', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      silent: true
+    })
+    const provider = new FakeVectorProvider(false)
+    useFakeVectorProvider(brain, provider)
+    await brain.init()
+    await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
+    await brain.add({ data: 'b', type: NounType.Thing, vector: V(2) })
+
+    const report = await brain.warm()
+
+    expect(provider.warmCalls).toBe(0) // no warm() on this provider
+    expect(provider.searchCalls.length).toBe(1) // the probe ran
+    expect(provider.searchCalls[0].k).toBe(Math.min(100, provider.size())) // k = min(100, size)
+    expect(report.vector.outcome).toBe('probed')
+    expect(report.vector.outcome).not.toBe('warmed') // never conflated
+    await brain.close()
+  })
+
+  it('vector: reports "unavailable" when there is nothing to probe (empty index, unknown dimension)', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      silent: true
+    })
+    // A disk-native provider MAY honestly report 0 resident entries while
+    // durable data exists on disk (the same posture documented on
+    // `VectorIndexProvider.isReady` — "an mmap/disk-native index may
+    // legitimately report 0 resident entries"). warm()'s probe fallback
+    // reads `size()`, so this is the real trigger for "nothing to probe":
+    // never a k=0 search, an honest skip.
+    const provider = new FakeVectorProvider(false)
+    provider.size = () => 0
+    useFakeVectorProvider(brain, provider)
+    await brain.init() // the VFS root bootstrap write sets `dimensions`, but size() still reports 0
+
+    const report = await brain.warm()
+
+    expect(provider.searchCalls.length).toBe(0) // nothing probed
+    expect(report.vector.outcome).toBe('unavailable')
+    await brain.close()
+  })
+
+  it('(c) metadata + graph hydration paths actually execute against filesystem storage with pre-existing data', async () => {
+    const dir = mkTmp()
+
+    // Build a brain with real data (including a field NOT in the metadata
+    // index's own common-fields warm subset — 'wave' — so hydrateAll()'s
+    // FULL hydration is distinguishable from init()'s partial warmCache()),
+    // and real graph edges, then close it (persisting everything).
+    const seed = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'filesystem', path: dir },
+      silent: true
+    })
+    await seed.init()
+    const ids: string[] = []
+    for (let i = 0; i < 6; i++) {
+      ids.push(
+        await seed.add({
+          data: `entity ${i}`,
+          type: NounType.Thing,
+          metadata: { wave: i % 3 },
+          vector: V(i + 1)
+        })
+      )
+    }
+    for (let i = 0; i + 1 < ids.length; i++) {
+      await seed.relate({ from: ids[i], to: ids[i + 1], type: VerbType.RelatedTo })
+    }
+    await seed.close()
+
+    // Cold-reopen a FRESH instance and spy on the real hydration seams before
+    // calling warm(), so we assert they actually ran (not just that the
+    // report claims they did).
+    const metaHydrateCalls: number[] = []
+    const loadedFields: string[] = []
+    const graphInitCalls: number[] = []
+
+    const origHydrateAll = MetadataIndexManager.prototype.hydrateAll
+    const origLoadSparseIndex = (MetadataIndexManager.prototype as any).loadSparseIndex
+    const origGraphInit = GraphAdjacencyIndex.prototype.init
+
+    MetadataIndexManager.prototype.hydrateAll = async function (...args: any[]) {
+      metaHydrateCalls.push(1)
+      return origHydrateAll.apply(this, args as any)
+    }
+    ;(MetadataIndexManager.prototype as any).loadSparseIndex = async function (
+      field: string,
+      ...args: any[]
+    ) {
+      loadedFields.push(field)
+      return origLoadSparseIndex.apply(this, [field, ...args] as any)
+    }
+    GraphAdjacencyIndex.prototype.init = async function (...args: any[]) {
+      graphInitCalls.push(1)
+      return origGraphInit.apply(this, args as any)
+    }
+
+    try {
+      const brain = new Brainy({
+        requireSubtype: false,
+        storage: { type: 'filesystem', path: dir },
+        silent: true
+      })
+      await brain.init()
+
+      const report = await brain.warm()
+
+      expect(metaHydrateCalls.length).toBe(1) // hydrateAll() actually ran
+      expect(loadedFields).toContain('wave') // a NON-common field was loaded — full hydration, not the heuristic subset
+      expect(report.metadata.outcome).toBe('warmed')
+
+      expect(graphInitCalls.length).toBeGreaterThanOrEqual(1) // graph's full-load seam ran (once at brain init, once via warm())
+      expect(report.graph.outcome).toBe('warmed')
+
+      // Correctness survives: the hydrated data still answers queries.
+      const byWhere = await brain.find({ type: NounType.Thing, where: { wave: 1 } })
+      expect(byWhere.length).toBe(2) // waves 1,4 of 0..5
+
+      await brain.close()
+    } finally {
+      MetadataIndexManager.prototype.hydrateAll = origHydrateAll
+      ;(MetadataIndexManager.prototype as any).loadSparseIndex = origLoadSparseIndex
+      GraphAdjacencyIndex.prototype.init = origGraphInit
+    }
+  })
+
+  it('(d) warmOnOpen: true runs warm() during init() — observable via the fake provider', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      warmOnOpen: true,
+      silent: true
+    })
+    const provider = new FakeVectorProvider(true)
+    useFakeVectorProvider(brain, provider)
+
+    // No explicit brain.warm() call — warmOnOpen must have run it as part of init().
+    await brain.init()
+
+    expect(provider.warmCalls).toBe(1)
+    await brain.close()
+  })
+
+  it('warmOnOpen defaults to false — init() does NOT run warm() unless opted in', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      silent: true
+    })
+    const provider = new FakeVectorProvider(true)
+    useFakeVectorProvider(brain, provider)
+
+    await brain.init()
+
+    expect(provider.warmCalls).toBe(0)
+    await brain.close()
+  })
+
+  it('(e) WarmReport shape: outcome literal + durationMs per surface, plus totalDurationMs', async () => {
+    const brain = new Brainy({
+      requireSubtype: false,
+      storage: { type: 'memory' },
+      silent: true
+    })
+    const provider = new FakeVectorProvider(true)
+    useFakeVectorProvider(brain, provider)
+    await brain.init()
+    await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
+
+    const report = await brain.warm()
+
+    for (const surface of ['vector', 'metadata', 'graph'] as const) {
+      expect(['warmed', 'probed', 'unavailable']).toContain(report[surface].outcome)
+      expect(typeof report[surface].durationMs).toBe('number')
+      expect(report[surface].durationMs).toBeGreaterThanOrEqual(0)
+    }
+    expect(typeof report.totalDurationMs).toBe('number')
+    expect(report.totalDurationMs).toBeGreaterThanOrEqual(0)
+    await brain.close()
+  })
+})

tests/unit/db/fact-log.test.ts

@@ -186,4 +186,52 @@ describe('fact log — round-trip, framing, reconcile, rotation, scan', () => {
     await log.sync()
     expect(log.segmentPaths()).toEqual([]) // only a tail exists — nothing sealed
   })
+
+  describe('scanFacts liveness contract (Stage-2 D1)', () => {
+    it('a wedged store fails LOUDLY within the first-batch bound — never a silent hang', async () => {
+      // Force a sealed segment (tiny rotateBytes) so the scan must READ from
+      // storage, then wedge that read: the exact production shape (a
+      // backlogged brain whose segment read never returned).
+      const mem: any = new MemoryStorage()
+      await mem.init()
+      const wedgeable = new FactLog(mem, { rotateBytes: 1 })
+      await wedgeable.open(0)
+      await wedgeable.append(fact(1))
+      await wedgeable.append(fact(2)) // second append rotates → seg 1 sealed
+      await wedgeable.sync()
+
+      const realRead = mem.readRawBytes.bind(mem)
+      mem.readRawBytes = (p: string) =>
+        p.includes('facts/seg-') ? new Promise(() => {}) : realRead(p) // hangs forever
+
+      const scan = wedgeable.scanFacts({ firstBatchTimeoutMs: 200 })
+      const started = Date.now()
+      await expect(scan.batches().next()).rejects.toThrow(/no first batch within 200ms/)
+      expect(Date.now() - started).toBeLessThan(5_000) // bound held, not a hang
+    })
+
+    it('a healthy scan is unaffected — first batch well inside the bound, all facts delivered', async () => {
+      for (let g = 1; g <= 5; g++) await log.append(fact(g))
+      await log.sync()
+      const scan = log.scanFacts({ batchSize: 2 })
+      const all: CommitFact[] = []
+      for await (const b of scan.batches()) all.push(...b.facts)
+      expect(all.map((f) => f.generation)).toEqual([1, 2, 3, 4, 5])
+      expect(scan.summary().factsYielded).toBe(5)
+    })
+
+    it('consumer think-time between pulls never counts against the producer', async () => {
+      for (let g = 1; g <= 4; g++) await log.append(fact(g))
+      await log.sync()
+      // Bound tighter than the consumer's pause: only the FIRST pull is
+      // raced, so a slow consumer after batch 1 must not trip the deadline.
+      const gen = log.scanFacts({ batchSize: 2, firstBatchTimeoutMs: 150 }).batches()
+      const first = await gen.next()
+      expect(first.done).toBe(false)
+      await new Promise((r) => setTimeout(r, 400)) // dawdle past the bound
+      const second = await gen.next()
+      expect(second.done).toBe(false)
+      expect((await gen.next()).done).toBe(true)
+    })
+  })
 })

tests/unit/db/generation-segments.test.ts

@@ -0,0 +1,150 @@
+/**
+ * @module tests/unit/db/generation-segments
+ * @description The generation-segment store (Stage-2 D1+D3 file format).
+ * Laws: (1) fold → read round-trips deltas and records byte-faithfully via
+ * sidecar point-reads; (2) the manifest is the ONLY discovery path — reopen
+ * reads one file, never a listing; (3) a lost/corrupt sidecar rebuilds from
+ * its segment loudly, a damaged SEGMENT fails loudly (never silent wrong
+ * data); (4) D3 reclaim drops whole segments only and bumps compactedBelow;
+ * (5) the packed digest is deterministic across reopen; (6) immutability —
+ * fold refuses overlap with sealed ranges.
+ */
+import { describe, it, expect, beforeEach } from 'vitest'
+import { MemoryStorage } from '../../../src/storage/adapters/memoryStorage.js'
+import {
+  GenerationSegmentStore,
+  SEGMENTS_PREFIX,
+  type FoldGeneration
+} from '../../../src/db/generationSegments.js'
+
+const UUID = (n: number): string => `00000000-0000-4000-8000-${String(n).padStart(12, '0')}`
+
+const gen = (g: number, recordCount = 2): FoldGeneration => ({
+  generation: g,
+  timestamp: 1_700_000_000_000 + g,
+  delta: { generation: g, nouns: [UUID(g)], verbs: [], bytes: 123 + g },
+  records: Array.from({ length: recordCount }, (_, i) => ({
+    kind: (i % 2 === 0 ? 'noun' : 'verb') as 'noun' | 'verb',
+    id: UUID(g * 100 + i),
+    record: { metadata: { noun: 'document', v: g }, vector: { v: [g, i] } }
+  }))
+})
+
+describe('db/GenerationSegmentStore — the D1+D3 packed tier', () => {
+  let storage: MemoryStorage
+  let store: GenerationSegmentStore
+
+  beforeEach(async () => {
+    storage = new MemoryStorage()
+    await storage.init()
+    store = new GenerationSegmentStore(storage as any)
+    await store.open()
+  })
+
+  it('fold → read round-trips deltas and records via sidecar point-reads', async () => {
+    const meta = await store.fold([gen(1), gen(2), gen(3)])
+    expect(meta).toMatchObject({ firstGeneration: 1, lastGeneration: 3, frames: 3 })
+    expect(meta.checksum).toBeGreaterThan(0)
+
+    expect(store.hasGeneration(2)).toBe(true)
+    expect(store.hasGeneration(4)).toBe(false)
+
+    const d2 = await store.readDelta(2)
+    expect(d2?.delta).toEqual({ generation: 2, nouns: [UUID(2)], verbs: [], bytes: 125 })
+    expect(d2?.timestamp).toBe(1_700_000_000_002)
+
+    const records = await store.readRecords(3)
+    expect(records).toHaveLength(2)
+    expect(records![0]).toEqual({
+      kind: 'noun',
+      id: UUID(300),
+      record: { metadata: { noun: 'document', v: 3 }, vector: { v: [3, 0] } }
+    })
+    // Point read by id, both kinds.
+    expect(await store.readRecord(3, 'verb', UUID(301))).toEqual({
+      metadata: { noun: 'document', v: 3 },
+      vector: { v: [3, 1] }
+    })
+    expect(await store.readRecord(3, 'noun', UUID(999))).toBeNull()
+  })
+
+  it('reopen discovers everything from the manifest alone — no listing', async () => {
+    await store.fold([gen(1), gen(2)])
+    await store.fold([gen(3), gen(4)])
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(reopened.segments()).toHaveLength(2)
+    expect(reopened.hasGeneration(4)).toBe(true)
+    expect((await reopened.readDelta(1))?.timestamp).toBe(1_700_000_000_001)
+  })
+
+  it('a lost sidecar rebuilds from its segment; a damaged segment fails LOUDLY', async () => {
+    const meta = await store.fold([gen(1), gen(2)])
+    const idxPath = `${SEGMENTS_PREFIX}/seg-${String(1).padStart(20, '0')}.idx`
+    await storage.deleteRawObject(idxPath)
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    // Rebuild path: still serves correct data.
+    expect((await reopened.readRecords(2))!).toHaveLength(2)
+
+    // Now damage the SEGMENT itself: flip a payload byte → CRC mismatch, loud.
+    const segPath = `${SEGMENTS_PREFIX}/${meta.file}`
+    const bytes = (await storage.readRawBytes(segPath))!
+    bytes[bytes.length - 3] ^= 0xff
+    await storage.writeRawBytes(segPath, bytes)
+    const damaged = new GenerationSegmentStore(storage as any)
+    await damaged.open()
+    ;(damaged as any).sidecars.clear()
+    await storage.deleteRawObject(idxPath) // force the sequential rebuild over damaged bytes
+    await expect(damaged.readRecords(2)).rejects.toThrow(/CRC mismatch|damaged/)
+  })
+
+  it('D3 reclaim drops whole segments only and bumps compactedBelow', async () => {
+    await store.fold([gen(1), gen(2)])
+    await store.fold([gen(3), gen(4)])
+    await store.fold([gen(5), gen(6)])
+
+    // Horizon mid-segment-2 (below 4): only segment 1 is FULLY below → drops.
+    const r1 = await store.dropSegmentsBelow(4)
+    expect(r1).toEqual({ dropped: 1, compactedBelow: 3 })
+    expect(store.hasGeneration(1)).toBe(false)
+    expect(store.hasGeneration(3)).toBe(true) // partial segment survives whole
+
+    // Bytes actually gone.
+    expect(await storage.readRawBytes(`${SEGMENTS_PREFIX}/seg-${String(1).padStart(20, '0')}.bgs`)).toBeNull()
+
+    // Horizon past everything: the rest drop; compactedBelow is durable.
+    const r2 = await store.dropSegmentsBelow(7)
+    expect(r2.dropped).toBe(2)
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(reopened.compactedBelow()).toBe(7)
+    expect(reopened.segments()).toHaveLength(0)
+  })
+
+  it('the packed digest is deterministic across reopen and changes with history', async () => {
+    await store.fold([gen(1), gen(2), gen(3)])
+    const atSeal = await store.digestThroughPacked(3)
+    const midSegment = await store.digestThroughPacked(2)
+    expect(atSeal).not.toBeNull()
+    expect(midSegment).not.toBeNull()
+    expect(midSegment).not.toBe(atSeal)
+
+    const reopened = new GenerationSegmentStore(storage as any)
+    await reopened.open()
+    expect(await reopened.digestThroughPacked(3)).toBe(atSeal)
+    expect(await reopened.digestThroughPacked(2)).toBe(midSegment)
+
+    await reopened.fold([gen(4)])
+    expect(await reopened.digestThroughPacked(4)).not.toBe(atSeal)
+  })
+
+  it('sealed segments are immutable — fold refuses overlap, requires ascending input', async () => {
+    await store.fold([gen(1), gen(2)])
+    await expect(store.fold([gen(2), gen(3)])).rejects.toThrow(/overlaps the packed tier/)
+    await expect(store.fold([gen(4), gen(4)])).rejects.toThrow(/strictly ascending/)
+    await expect(store.fold([])).rejects.toThrow(/at least one generation/)
+  })
+})

tests/unit/db/generationStore.test.ts

@@ -488,5 +488,95 @@ describe('db/GenerationStore', () => {
       expect(result.removedGenerations).toBe(2)
       store.release(2)
     })
+
+    it('timeBudgetMs bounds a pass; the next pass resumes the same prefix', async () => {
+      await manyGens(4)
+      // A spent budget (0ms) stops before reclaiming anything — an early stop
+      // is a consistent prefix, never a partial generation.
+      const bounded = await store.compact({ timeBudgetMs: 0 })
+      expect(bounded.removedGenerations).toBe(0)
+      expect(bounded.horizon).toBe(0)
+      // The next (unbounded) pass picks up exactly where the bounded one
+      // stopped and completes the same work.
+      const resumed = await store.compact()
+      expect(resumed.removedGenerations).toBe(4)
+      expect(resumed.horizon).toBe(4)
+    })
+  })
+
+  // ==========================================================================
+  describe('history-bytes running total (the O(1) retention check)', () => {
+    /** A fresh walk with the cache dropped — ground truth for the invariant. */
+    async function groundTruthBytes(): Promise<number> {
+      ;(store as any).historyBytesTotal = null
+      return store.historyBytes()
+    }
+
+    it('is seeded once, then maintained through commits WITHOUT re-walks', async () => {
+      await commitWrite(ID_A, 1)
+      await commitWrite(ID_A, 2)
+      const seeded = await store.historyBytes()
+      expect(seeded).toBe(await groundTruthBytes())
+
+      // From here every read must come from the running total, not a walk:
+      // getDelta re-reads are the walk's cost — commits must not trigger any.
+      const getDeltaSpy = vi.spyOn(store as any, 'getDelta')
+      await commitWrite(ID_B, 1)
+      const afterCommit = await store.historyBytes()
+      expect(getDeltaSpy).not.toHaveBeenCalled()
+      getDeltaSpy.mockRestore()
+      expect(afterCommit).toBe(await groundTruthBytes())
+    })
+
+    it('stays exact through single-op group commits and compaction', async () => {
+      await commitWrite(ID_A, 1)
+      await store.historyBytes() // seed
+      // Single-op path: buffered generations flushed as one group commit.
+      await store.commitSingleOp({
+        touched: { nouns: [ID_B] },
+        execute: async () => {
+          await storage.saveNounMetadata(ID_B, metadataFixture(1))
+        }
+      })
+      await store.flushPendingSingleOps()
+      expect(await store.historyBytes()).toBe(await groundTruthBytes())
+
+      await store.historyBytes() // re-seed after ground-truth reset
+      await store.compact({ maxGenerations: 1 })
+      expect(await store.historyBytes()).toBe(await groundTruthBytes())
+    })
+
+    it('historyStats reports counts, bytes, range, and horizon read-only', async () => {
+      await commitWrite(ID_A, 1)
+      await commitWrite(ID_B, 1)
+      const stats = await store.historyStats()
+      expect(stats.generations).toBe(2)
+      expect(stats.bytes).toBe(await store.historyBytes())
+      expect(stats.oldestGeneration).toBe(1)
+      expect(stats.newestGeneration).toBe(2)
+      expect(stats.oldestTimestamp).toBeLessThanOrEqual(stats.newestTimestamp!)
+      expect(stats.horizon).toBe(0)
+      // Read-only: nothing was reclaimed by asking.
+      expect(store.committedGeneration()).toBe(2)
+
+      await store.compact({ maxGenerations: 1 })
+      const after = await store.historyStats()
+      expect(after.generations).toBe(1)
+      expect(after.oldestGeneration).toBe(2)
+      expect(after.horizon).toBe(1)
+    })
+
+    it('empty history reports null range and zero bytes', async () => {
+      const stats = await store.historyStats()
+      expect(stats).toMatchObject({
+        generations: 0,
+        bytes: 0,
+        oldestGeneration: null,
+        newestGeneration: null,
+        oldestTimestamp: null,
+        newestTimestamp: null,
+        horizon: 0
+      })
+    })
   })
 })

tests/unit/storage/verb-cursor-pagination.test.ts

@@ -95,9 +95,15 @@ describe('verb cursor pagination (graph-perf #2)', () => {
     expect(new Set(cursorSeen)).toEqual(new Set(offsetSeen))
   })
 
-  it('a foreign/malformed cursor falls back gracefully (no throw, starts from the beginning)', async () => {
+  it('a foreign/malformed cursor FAILS LOUDLY — never a silent restart from page 1', async () => {
-    const page = await storage.getVerbs({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
+    // The old behavior (decode-null → silent offset-0 fallback) re-served page 1
-    expect(page.items.length).toBe(5)
+    // forever to any while(hasMore) walker: an unbounded CPU loop with no log
-    expect(page.hasMore).toBe(true)
+    // line. An undecodable resume token now refuses the walk instead.
+    await expect(
+      storage.getVerbs({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
+    ).rejects.toThrow('invalid pagination cursor')
+    await expect(
+      storage.getNouns({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
+    ).rejects.toThrow('invalid pagination cursor')
   })
 })

tests/unit/transaction/budget-commit-completed-work.test.ts

@@ -0,0 +1,149 @@
+/**
+ * @module tests/unit/transaction/budget-commit-completed-work
+ * @description Regression coverage for the "commit-completed-work" transaction
+ * budget semantics: the budget gates STARTING the next operation; it never
+ * converts already-completed work into a rollback. Concretely:
+ *
+ * - A single-op transaction can never time out post-hoc — its one operation
+ *   either runs (and the transaction commits, however long it took) or it
+ *   never gets to start (which cannot happen: there is no operation before it
+ *   to have overrun the budget).
+ * - A multi-op transaction whose op `i` overruns the budget stops op `i+1`
+ *   from starting: everything applied so far rolls back atomically and a
+ *   `TransactionTimeoutError` is thrown — the mid-batch zero-loss guarantee is
+ *   unchanged.
+ * - `transactTimeoutBudget`'s scaling floor (`max(floorMs, opCount * 2000)`)
+ *   is overridable per call, the seam `BrainyConfig.transactionBudgetFloorMs`
+ *   feeds at the brainy.ts read sites.
+ *
+ * Uses a real class implementing the `Operation` interface (not an anonymous
+ * object literal, not a mock of Transaction internals) so the exercised path
+ * is exactly what a real caller's operation looks like.
+ */
+import { describe, it, expect } from 'vitest'
+import { Transaction, transactTimeoutBudget } from '../../../src/transaction/Transaction.js'
+import type { Operation, RollbackAction } from '../../../src/transaction/types.js'
+import { TransactionTimeoutError } from '../../../src/transaction/errors.js'
+
+const sleep = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms))
+
+/**
+ * A real `Operation` implementation that sleeps for `delayMs` before applying
+ * a write to `log`, and returns a rollback action that removes it again. Used
+ * to deterministically make one operation "slow" relative to a transaction's
+ * budget without touching any Transaction internals.
+ */
+class SlowOperation implements Operation {
+  readonly name: string
+  executed = false
+  rolledBack = false
+
+  constructor(
+    private readonly log: string[],
+    label: string,
+    private readonly delayMs: number
+  ) {
+    this.name = label
+  }
+
+  async execute(): Promise<RollbackAction | undefined> {
+    this.executed = true
+    if (this.delayMs > 0) await sleep(this.delayMs)
+    this.log.push(this.name)
+    return async () => {
+      this.rolledBack = true
+      const idx = this.log.indexOf(this.name)
+      if (idx >= 0) this.log.splice(idx, 1)
+    }
+  }
+}
+
+describe('Transaction budget — commit-completed-work semantics', () => {
+  it('(a) single-op transaction whose op overruns the budget COMMITS — no rollback, no throw', async () => {
+    const log: string[] = []
+    const op = new SlowOperation(log, 'slow-single-op', 40)
+
+    // Budget (5ms) is far smaller than the op's 40ms — under the OLD
+    // (post-completion-check) semantics this would have thrown and rolled
+    // back after the op finished. Under the new semantics there is no
+    // operation after it to gate, so it commits.
+    const tx = new Transaction({ timeout: 5 })
+    tx.addOperation(op)
+
+    await expect(tx.execute()).resolves.toBeUndefined()
+
+    expect(tx.getState()).toBe('committed')
+    expect(op.executed).toBe(true)
+    expect(op.rolledBack).toBe(false)
+    expect(log).toEqual(['slow-single-op'])
+  })
+
+  it('(b) two-op transaction: op0 overruns the budget → op1 never starts, op0 rolls back, throws TransactionTimeoutError', async () => {
+    const log: string[] = []
+    const op0 = new SlowOperation(log, 'op0-overruns', 40)
+    const op1 = new SlowOperation(log, 'op1-never-starts', 0)
+
+    const tx = new Transaction({ timeout: 5 })
+    tx.addOperation(op0)
+    tx.addOperation(op1)
+
+    const error = await tx.execute().catch((e) => e)
+
+    expect(error).toBeInstanceOf(TransactionTimeoutError)
+    expect(op0.executed).toBe(true)
+    expect(op0.rolledBack).toBe(true)
+    expect(op1.executed).toBe(false)
+    expect(op1.rolledBack).toBe(false)
+    expect(log).toEqual([]) // op0's write was undone; op1 never wrote
+    expect(tx.getState()).toBe('rolled_back')
+  })
+
+  it('a single-op transaction never checks the budget before its only operation starts', async () => {
+    // Budget of 0ms: under the old "check before every op including 0" code
+    // this could theoretically trip before op 0 even started (if any
+    // measurable time elapsed between startTime capture and the check). The
+    // documented contract is stronger: operation 0 ALWAYS gets to start.
+    const log: string[] = []
+    const op = new SlowOperation(log, 'only-op', 5)
+
+    const tx = new Transaction({ timeout: 0 })
+    tx.addOperation(op)
+
+    await expect(tx.execute()).resolves.toBeUndefined()
+    expect(tx.getState()).toBe('committed')
+    expect(op.executed).toBe(true)
+  })
+
+  it('(c) transactTimeoutBudget: an explicit floorMs overrides the 30s default floor', () => {
+    // Below the per-op scaling term, a raised floor wins.
+    expect(transactTimeoutBudget(1, undefined, 5_000)).toBe(5_000) // 1 op × 2000 = 2000 < 5000 floor
+    expect(transactTimeoutBudget(1)).toBe(30_000) // unchanged default when floorMs omitted
+
+    // Above the floor, opCount * 2000 still wins over a smaller floor.
+    expect(transactTimeoutBudget(10, undefined, 5_000)).toBe(20_000) // 10 × 2000 = 20000 > 5000 floor
+
+    // A full `override` always wins, regardless of floorMs.
+    expect(transactTimeoutBudget(10, 999, 5_000)).toBe(999)
+  })
+
+  it('(c) a configured floor changes real Transaction behavior end-to-end via TransactionManager-style options', async () => {
+    // Simulates how brainy.ts wires `this.config.transactionBudgetFloorMs`
+    // into `transactTimeoutBudget()`: opCount 0 isolates the floor term
+    // (0 × 2000 = 0), so a lowered floor makes a normally-generous budget
+    // fail-fast for a real 2-op Transaction.
+    const lowFloorBudget = transactTimeoutBudget(0, undefined, 10) // 10ms floor
+    expect(lowFloorBudget).toBe(10)
+
+    const log: string[] = []
+    const op0 = new SlowOperation(log, 'op0', 30)
+    const op1 = new SlowOperation(log, 'op1-gated-by-lowered-floor', 0)
+
+    const tx = new Transaction({ timeout: lowFloorBudget })
+    tx.addOperation(op0)
+    tx.addOperation(op1)
+
+    const error = await tx.execute().catch((e) => e)
+    expect(error).toBeInstanceOf(TransactionTimeoutError)
+    expect(op1.executed).toBe(false)
+  })
+})

tests/unit/transaction/vectorIndexOperations-rename.test.ts

@@ -0,0 +1,176 @@
+/**
+ * @module tests/unit/transaction/vectorIndexOperations-rename
+ * @description Coverage for the backend-neutral vector-index transaction
+ * operations (formerly `AddToHNSWOperation` / `RemoveFromHNSWOperation`).
+ * These op-name strings surface directly in consumer-visible transaction
+ * journals and timings — a fossil "HNSW" name misdirects an operator running
+ * a native (non-HNSW) vector provider. Verifies:
+ *   1. rollback wiring stayed byte-identical under the rename,
+ *   2. the emitted `name` stamps the active backend — `'hnsw-js'` for
+ *      Brainy's own JS index, a provider's own required `name` when it
+ *      self-identifies, and the tolerant-loud `'unknown-provider'` literal
+ *      (plus one console.warn) when a runtime instance lacks `name`
+ *      altogether (an older native provider compiled against the previous,
+ *      optional `providerId` contract) — never a silently-wrong guess.
+ */
+import { describe, it, expect, vi, afterEach } from 'vitest'
+import {
+  AddToVectorIndexOperation,
+  RemoveFromVectorIndexOperation,
+  BatchAddToVectorIndexOperation
+} from '../../../src/transaction/operations/IndexOperations.js'
+import type { VectorIndexProvider } from '../../../src/plugin.js'
+import type { VectorDocument, Vector } from '../../../src/coreTypes.js'
+import { JsHnswVectorIndex } from '../../../src/hnsw/hnswIndex.js'
+
+/**
+ * A minimal, real (not mocked) VectorIndexProvider implementation backed by
+ * a plain Map — exercises the exact contract the operations call through,
+ * without any Brainy internals.
+ */
+class FakeVectorProvider implements VectorIndexProvider {
+  readonly items = new Map<string, Vector>()
+  constructor(readonly name: string) {}
+
+  async addItem(item: VectorDocument): Promise<string> {
+    this.items.set(item.id, item.vector)
+    return item.id
+  }
+  async removeItem(id: string): Promise<boolean> {
+    return this.items.delete(id)
+  }
+  async getItem(id: string): Promise<VectorDocument | undefined> {
+    const vector = this.items.get(id)
+    return vector ? { id, vector } : undefined
+  }
+  async search(): Promise<Array<[string, number]>> {
+    return []
+  }
+  size(): number {
+    return this.items.size
+  }
+  clear(): void {
+    this.items.clear()
+  }
+  async rebuild(): Promise<void> {}
+  async flush(): Promise<number> {
+    return 0
+  }
+  getPersistMode(): 'immediate' | 'deferred' {
+    return 'immediate'
+  }
+}
+
+describe('Vector index transaction operations — backend-neutral rename', () => {
+  describe('rollback wiring (byte-identical behavior under the rename)', () => {
+    it('AddToVectorIndexOperation rollback removes a newly-added item', async () => {
+      const provider = new FakeVectorProvider('fake-provider')
+      const op = new AddToVectorIndexOperation(provider, 'id-1', [1, 2, 3])
+
+      const rollback = await op.execute()
+      expect(provider.items.has('id-1')).toBe(true)
+
+      await rollback()
+      expect(provider.items.has('id-1')).toBe(false)
+    })
+
+    it('AddToVectorIndexOperation rollback is a no-op when the item pre-existed (update semantics)', async () => {
+      const provider = new FakeVectorProvider('fake-provider')
+      await provider.addItem({ id: 'id-1', vector: [9, 9, 9] })
+
+      const op = new AddToVectorIndexOperation(provider, 'id-1', [1, 2, 3])
+      const rollback = await op.execute()
+      expect(provider.items.get('id-1')).toEqual([1, 2, 3])
+
+      await rollback()
+      // Pre-existing item is NOT removed by rollback — it stays (the update
+      // itself is not undone by this op; that's RemoveFromVectorIndexOperation's job).
+      expect(provider.items.has('id-1')).toBe(true)
+    })
+
+    it('RemoveFromVectorIndexOperation rollback re-adds the item with its original vector', async () => {
+      const provider = new FakeVectorProvider('fake-provider')
+      await provider.addItem({ id: 'id-1', vector: [4, 5, 6] })
+
+      const op = new RemoveFromVectorIndexOperation(provider, 'id-1', [4, 5, 6])
+      const rollback = await op.execute()
+      expect(provider.items.has('id-1')).toBe(false)
+
+      await rollback()
+      expect(provider.items.get('id-1')).toEqual([4, 5, 6])
+    })
+
+    it('BatchAddToVectorIndexOperation rolls back every item in reverse order on undo', async () => {
+      const provider = new FakeVectorProvider('fake-provider')
+      const op = new BatchAddToVectorIndexOperation(provider, [
+        { id: 'a', vector: [1] },
+        { id: 'b', vector: [2] },
+        { id: 'c', vector: [3] }
+      ])
+
+      const rollback = await op.execute()
+      expect([...provider.items.keys()].sort()).toEqual(['a', 'b', 'c'])
+
+      await rollback()
+      expect(provider.items.size).toBe(0)
+    })
+  })
+
+  describe('backend stamping in the emitted op name', () => {
+    afterEach(() => {
+      vi.restoreAllMocks()
+    })
+
+    it('stamps the real JS HNSW index as hnsw-js (self-identifying name)', () => {
+      const index = new JsHnswVectorIndex()
+      const addOp = new AddToVectorIndexOperation(index, 'id-1', [1, 2, 3])
+      const removeOp = new RemoveFromVectorIndexOperation(index, 'id-1', [1, 2, 3])
+
+      expect(addOp.name).toBe('AddToVectorIndex(hnsw-js)')
+      expect(removeOp.name).toBe('RemoveFromVectorIndex(hnsw-js)')
+    })
+
+    it('stamps a self-identifying provider\'s own name, never "HNSW"', () => {
+      const provider = new FakeVectorProvider('acme-diskann')
+      const addOp = new AddToVectorIndexOperation(provider, 'id-1', [1, 2, 3])
+      const removeOp = new RemoveFromVectorIndexOperation(provider, 'id-1', [1, 2, 3])
+      const batchOp = new BatchAddToVectorIndexOperation(provider, [{ id: 'a', vector: [1] }])
+
+      expect(addOp.name).toBe('AddToVectorIndex(acme-diskann)')
+      expect(removeOp.name).toBe('RemoveFromVectorIndex(acme-diskann)')
+      expect(batchOp.name).toBe('BatchAddToVectorIndex(acme-diskann)')
+      expect(addOp.name).not.toContain('HNSW')
+      expect(removeOp.name).not.toContain('HNSW')
+    })
+
+    it('tolerant-loud: stamps "unknown-provider" and warns exactly once when a runtime provider lacks the required `name` (an older native provider compiled against the previous optional `providerId` contract)', () => {
+      const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
+      // Simulate a native provider compiled before `name` was required — the
+      // TypeScript contract requires `name`, but `as unknown as` mirrors what
+      // actually reaches this code at runtime from an un-rebuilt native addon.
+      const legacyProvider = {
+        addItem: async (item: VectorDocument) => item.id,
+        removeItem: async () => true,
+        search: async () => [],
+        size: () => 0,
+        clear: () => {},
+        rebuild: async () => {},
+        flush: async () => 0,
+        getPersistMode: () => 'immediate' as const
+      } as unknown as VectorIndexProvider
+
+      const addOp = new AddToVectorIndexOperation(legacyProvider, 'id-1', [1, 2, 3])
+      const removeOp = new RemoveFromVectorIndexOperation(legacyProvider, 'id-1', [1, 2, 3])
+
+      expect(addOp.name).toBe('AddToVectorIndex(unknown-provider)')
+      expect(removeOp.name).toBe('RemoveFromVectorIndex(unknown-provider)')
+      // Never silently falls back to the JS engine's name for a provider it
+      // knows nothing about — that's the exact fossil-naming bug being fixed.
+      expect(addOp.name).not.toContain('hnsw-js')
+      // Loud, not silent — but exactly once per provider instance, not once
+      // per op stamped against it.
+      expect(warnSpy).toHaveBeenCalledTimes(1)
+      expect(warnSpy.mock.calls[0][0]).toContain('name')
+    })
+  })
+})

tests/unit/utils/osLimits.test.ts

@@ -0,0 +1,76 @@
+/**
+ * @module tests/unit/utils/osLimits
+ * @description OS-limit detection for pool-scale use. Laws:
+ * (1) the /proc/self/limits parser reads soft/hard NOFILE exactly, including
+ *     'unlimited'; (2) assessment warns ONLY below the pool floors and NEVER
+ *     on an unreadable (null) limit — no measurement, no claim; (3) the full
+ *     check composes both sources and survives unreadable /proc silently.
+ */
+import { describe, it, expect } from 'vitest'
+import {
+  parseProcLimits,
+  assessOsLimits,
+  checkOsLimits,
+  NOFILE_POOL_FLOOR,
+  MAX_MAP_COUNT_POOL_FLOOR
+} from '../../../src/utils/osLimits.js'
+
+const SAMPLE_LIMITS = [
+  'Limit                     Soft Limit           Hard Limit           Units',
+  'Max cpu time              unlimited            unlimited            seconds',
+  'Max open files            1024                 1048576              files',
+  'Max locked memory         8388608              8388608              bytes'
+].join('\n')
+
+describe('osLimits — detect + warn at pool scale', () => {
+  it('parses soft/hard NOFILE from /proc/self/limits, including unlimited', () => {
+    expect(parseProcLimits(SAMPLE_LIMITS)).toEqual({ soft: 1024, hard: 1048576 })
+    expect(
+      parseProcLimits('Max open files            unlimited            unlimited            files')
+    ).toEqual({ soft: Infinity, hard: Infinity })
+    expect(parseProcLimits('no such row here')).toEqual({ soft: null, hard: null })
+  })
+
+  it('warns below the floors, stays quiet at or above them', () => {
+    const low = assessOsLimits({ nofileSoft: 1024, nofileHard: 1048576, maxMapCount: 65530 })
+    expect(low).toHaveLength(2)
+    expect(low[0]).toContain('RLIMIT_NOFILE soft limit is 1024')
+    expect(low[0]).toContain(`ulimit -n ${NOFILE_POOL_FLOOR}`)
+    expect(low[0]).toContain('raise the soft limit only') // hard already allows it
+    expect(low[1]).toContain('vm.max_map_count is 65530')
+    expect(low[1]).toContain(`vm.max_map_count=${MAX_MAP_COUNT_POOL_FLOOR}`)
+
+    expect(
+      assessOsLimits({
+        nofileSoft: NOFILE_POOL_FLOOR,
+        nofileHard: Infinity,
+        maxMapCount: MAX_MAP_COUNT_POOL_FLOOR
+      })
+    ).toEqual([])
+  })
+
+  it('an unreadable limit makes NO claim — nulls never warn', () => {
+    expect(assessOsLimits({ nofileSoft: null, nofileHard: null, maxMapCount: null })).toEqual([])
+  })
+
+  it('checkOsLimits composes both sources and survives unreadable /proc silently', async () => {
+    const report = await checkOsLimits(async (p) => {
+      if (p === '/proc/self/limits') return SAMPLE_LIMITS
+      if (p === '/proc/sys/vm/max_map_count') return '65530\n'
+      throw new Error('unexpected path')
+    })
+    expect(report.nofileSoft).toBe(1024)
+    expect(report.maxMapCount).toBe(65530)
+    expect(report.warnings).toHaveLength(2)
+
+    const offLinux = await checkOsLimits(async () => {
+      throw Object.assign(new Error('ENOENT'), { code: 'ENOENT' })
+    })
+    expect(offLinux).toEqual({
+      nofileSoft: null,
+      nofileHard: null,
+      maxMapCount: null,
+      warnings: []
+    })
+  })
+})