-
-
-
- Three database paradigms. One API. Zero configuration.
- The in-process knowledge database for TypeScript — vector search, graph traversal,
- and metadata filtering unified in a single query.
+
- 
- 
- 
- 
- 
- 
-
- Quick start · - One query · - Features · - Scale with Cor · - Docs -
+**Three database paradigms. One API. Zero configuration.** ---- - -Built because we were tired of stitching a vector store to a graph database to a document store — and spending weeks on plumbing before writing a line of business logic. Brainy indexes every fact **three ways at once** and lets one call query them together: - -| You write | Brainy indexes it as | You query it with | -|---|---|---| -| `data: 'Ada wrote the first program'` | a **384-dim vector** (local embedding — no API key) | `find({ query: 'computing pioneers' })` | -| `metadata: { field: 'CS', year: 1843 }` | **structured fields** (O(1) exact, O(log n) range) | `find({ where: { year: { lessThan: 1900 } } })` | -| `relate({ from: ada, to: babbage })` | a **typed, directed graph edge** | `find({ connected: { to: babbage, depth: 2 } })` | - -It runs **inside your process** — no server, no Docker, nothing to operate — and persists to plain files you can snapshot with a hard link. +Built because we were tired of stitching together Pinecone + Neo4j + MongoDB and spending weeks on configuration before writing a single line of business logic. Brainy unifies vector search, graph traversal, and metadata filtering so you don't have to choose. **New here?** → **[What is Brainy? — plain-language overview, no jargon](docs/eli5.md)** -## Quick start +--- + +## Install ```bash -bun add @soulcraft/brainy # Bun ≥ 1.1 — recommended -npm install @soulcraft/brainy # Node.js ≥ 22 +npm install @soulcraft/brainy ``` +## Quick Start + ```javascript import { Brainy, NounType, VerbType } from '@soulcraft/brainy' -const brain = new Brainy() // in-memory; one line swaps to disk +const brain = new Brainy() await brain.init() -// Text auto-embeds locally; metadata auto-indexes -const react = await brain.add({ +// Add knowledge — text auto-embeds, metadata auto-indexes +const reactId = await brain.add({ data: 'React is a JavaScript library for building user interfaces', type: NounType.Concept, - subtype: 'library', metadata: { category: 'frontend', year: 2013 } }) -const next = await brain.add({ - data: 'Next.js is a React framework with server-side rendering', +const nextId = await brain.add({ + data: 'Next.js framework for React with server-side rendering', type: NounType.Concept, - subtype: 'framework', - metadata: { category: 'frontend', year: 2016 } + metadata: { category: 'framework', year: 2016 } }) -await brain.relate({ from: next, to: react, type: VerbType.DependsOn, subtype: 'runtime' }) +// Create a relationship +await brain.relate({ from: nextId, to: reactId, type: VerbType.BuiltOn }) + +// Query all three paradigms at once +const results = await brain.find({ + query: 'modern frontend frameworks', // Vector similarity + where: { year: { greaterThan: 2015 } }, // Metadata filtering + connected: { to: reactId, depth: 2 } // Graph traversal +}) ``` -## One query, three engines +**[Full API Reference](docs/api/README.md)** | **[soulcraft.com/docs](https://soulcraft.com/docs)** + +--- + +## Three Indexes, One Query + +Every piece of knowledge lives in three indexes simultaneously: + +- **`data`** → **Vector index** — Content for semantic search. Strings auto-embed into 384-dim vectors. Queried with `find({ query: '...' })`. +- **`metadata`** → **Metadata index** — Structured fields for filtering. O(1) lookups. Queried with `find({ where: { ... } })`. +- **`relate()`** → **Graph index** — Typed, directed relationships between entities. Traversed with `find({ connected: { ... } })`. + +```javascript +// Data → vector index (semantic search) +const articleId = await brain.add({ + data: 'A deep dive into transformer architectures', + type: NounType.Document, + metadata: { author: 'Dr. Chen', year: 2024, tags: ['AI'] } // → metadata index +}) + +// Relationships → graph index +await brain.relate({ from: authorId, to: articleId, type: VerbType.Authored }) + +// Query all three at once +brain.find({ + query: 'attention mechanisms', // Vector similarity + where: { year: { greaterThan: 2023 } }, // Metadata filter + connected: { from: authorId, depth: 1 } // Graph traversal +}) +``` + +**[Data Model Reference](docs/DATA_MODEL.md)** | **[Query Operators](docs/QUERY_OPERATORS.md)** + +--- + +## Features + +### Triple Intelligence + +Vector search + graph traversal + metadata filtering in every query. No stitching services together — one `find()` call combines all three. ```javascript const results = await brain.find({ - query: 'modern frontend frameworks', // vector — what it means - where: { year: { greaterThan: 2015 } }, // metadata — what it is - connected: { to: react, depth: 2 } // graph — what it touches + query: 'machine learning', + where: { department: 'engineering', level: 'senior' }, + connected: { from: teamLeadId, via: VerbType.WorksWith, depth: 2 } }) ``` -Every clause is optional; any combination composes. Under the hood Brainy plans the query across an HNSW vector index, a roaring-bitmap field index, and an adjacency graph index — and re-validates every result against your predicate before returning it, so a corrupt index can never hand you a wrong answer. +### Hybrid Search -## Feature tour - -### The database is a value - -Pin it, rewind it, fork it. Snapshot isolation without a server. +Automatically combines keyword (text) and semantic (vector) search. No configuration needed. ```javascript -const db = brain.now() // pin current state — O(1) - -await brain.transact([ // atomic all-or-nothing, CAS-guarded - { op: 'update', id: order, metadata: { status: 'paid' } }, - { op: 'relate', from: invoice, to: order, type: VerbType.References, subtype: 'billing' } -], { ifAtGeneration: db.generation }) - -await db.get(order) // still 'pending' — pinned forever -await brain.get(order) // 'paid' — live - -const lastWeek = await brain.asOf(Date.now() - 7 * 86_400_000) // full query surface, past state -const whatIf = await db.with([{ op: 'remove', id: order }]) // speculative — never touches disk -await brain.now().persist('/backups/today') // instant hard-link snapshot +await brain.find({ query: 'David Smith' }) // Auto: text + semantic +await brain.find({ query: 'AI concepts', searchMode: 'semantic' }) // Semantic only +await brain.find({ query: 'exact id', searchMode: 'text' }) // Text only ``` -**[Consistency model](docs/concepts/consistency-model.md)** · **[Snapshots & time travel](docs/guides/snapshots-and-time-travel.md)** +### Query Operators -### Local embeddings — no API keys - -Strings embed on-device with a bundled MiniLM model (WASM). Semantic search works offline, in CI, and on air-gapped machines, at zero cost per call. Hybrid keyword + semantic ranking is the default: +Filter metadata with equality, comparison, array, existence, pattern, and logical operators: ```javascript -await brain.find({ query: 'David Smith' }) // auto: text + semantic -await brain.find({ query: 'AI concepts', searchMode: 'semantic' }) // semantic only +await brain.find({ + where: { + status: 'active', // Exact match + score: { greaterThan: 90 }, // Comparison + tags: { contains: 'ai' }, // Array + anyOf: [{ role: 'admin' }, { role: 'owner' }] // Logical OR + } +}) ``` -### A typed graph, not a bag of edges +**[Query Operators Reference](docs/QUERY_OPERATORS.md)** — all operators with indexed/in-memory matrix -42 entity types × 127 relationship types form a shared vocabulary for any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), yours. Your own taxonomy layers on with `subtype`, enforced at write time: +### Graph Relationships + +Typed, directed edges between entities. Traverse connections at any depth. ```javascript -await brain.add({ data: 'Avery Brooks', type: NounType.Person, subtype: 'employee' }) +await brain.relate({ from: personId, to: projectId, type: VerbType.WorksOn }) -brain.counts.bySubtype(NounType.Person) // O(1) — { employee: 12, customer: 847 } -brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true }) +const results = await brain.find({ + connected: { from: personId, via: VerbType.WorksOn, depth: 3 } +}) ``` -**[Type system](docs/architecture/noun-verb-taxonomy.md)** · **[Subtypes & facets](docs/guides/subtypes-and-facets.md)** +### Git-Style Branching -### Graph analytics built in +Fork your entire database in <100ms. Snowflake-style copy-on-write. ```javascript -await brain.graph.rank() // which entities matter most (centrality) -await brain.graph.communities() // natural clusters -await brain.graph.path(a, b) // how two things connect -await brain.graph.subgraph([seed], { depth: 2 }) // bounded neighborhood → { nodes, edges } -await brain.graph.export() // whole graph, one O(N+E) streaming pass +const experiment = await brain.fork('test-migration') +await experiment.add({ data: 'test data', type: NounType.Concept }) +await experiment.commit({ message: 'Add test data', author: 'dev@co.com' }) +await brain.checkout('test-migration') + +// Time-travel: query at any past commit +const snapshot = await brain.asOf(commitId) +const pastResults = await snapshot.find({ query: 'historical data' }) +await snapshot.close() ``` -### Write-time aggregations +**[Branching Documentation](docs/features/instant-fork.md)** -`SUM` / `COUNT` / `AVG` / `MIN` / `MAX` with `GROUP BY` and time windows, maintained incrementally on every write — reads are O(1) lookups, not scans. **[Aggregation guide](docs/guides/aggregation.md)** +### Entity Versioning -### Import anything +Save, restore, and compare entity snapshots. + +```javascript +const userId = await brain.add({ data: 'Alice', type: NounType.Person }) +await brain.versions.save(userId, { tag: 'v1.0' }) + +await brain.update(userId, { data: 'Alice Smith' }) +await brain.versions.save(userId, { tag: 'v2.0' }) + +const diff = await brain.versions.compare(userId, 1, 2) +await brain.versions.restore(userId, 1) +``` + +### Virtual Filesystem + +File operations with semantic search built in. + +```javascript +const vfs = brain.vfs + +await vfs.writeFile('/docs/readme.md', 'Project documentation') +const content = await vfs.readFile('/docs/readme.md') +const tree = await vfs.getTreeStructure('/docs', { maxDepth: 3 }) + +// Semantic file search +const matches = await vfs.search('React components with hooks') +``` + +**[VFS Quick Start](docs/vfs/QUICK_START.md)** | **[Common Patterns](docs/vfs/COMMON_PATTERNS.md)** + +### Import Anything + +CSV, Excel, PDF, URLs — auto-detected format, auto-classified entities. ```javascript await brain.import('customers.csv') -await brain.import('sales.xlsx') // every sheet -await brain.import('research-paper.pdf') // tables extracted +await brain.import('sales-data.xlsx', { excelSheets: ['Q1', 'Q2'] }) +await brain.import('research-paper.pdf', { pdfExtractTables: true }) await brain.import('https://api.example.com/data.json') ``` -Entities auto-classify on the way in; `brain.extractEntities(text)` exposes the same NER ensemble directly. **[Import guide](docs/guides/import-anything.md)** +**[Import Guide](docs/guides/import-anything.md)** -### A filesystem that understands content +### Entity Extraction + +AI-powered named entity recognition with 4-signal ensemble scoring. ```javascript -await brain.vfs.writeFile('/docs/readme.md', 'Project documentation') -await brain.vfs.search('React components with hooks') // semantic file search +const entities = await brain.extractEntities('John Smith founded Acme Corp in New York') +// [ +// { text: 'John Smith', type: NounType.Person, confidence: 0.95 }, +// { text: 'Acme Corp', type: NounType.Organization, confidence: 0.92 }, +// { text: 'New York', type: NounType.Location, confidence: 0.88 } +// ] ``` -**[VFS quick start](docs/vfs/QUICK_START.md)** +**[Neural Extraction Guide](docs/neural-extraction.md)** -### Operations-grade by default +### Plugin System -- **Single-writer, many-reader** — an exclusive lock protects the data directory; `Brainy.openReadOnly()` and the `brainy inspect` CLI examine a live brain from another process, safely. -- **Self-upgrading data files** — a 7.x brain opens under 8.x and migrates itself behind an observable lock (`getIndexStatus().migration`), with an automatic pre-upgrade backup. No migration scripts. -- **No silent wrong answers** — cold-open guards self-heal or throw typed errors (`MetadataIndexNotReadyError`, `GraphIndexNotReadyError`); they never return `[]` for data that exists. - -**[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)** - -## From laptop to hundreds of millions - -Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**: - -```bash -npm install @soulcraft/cor -``` +Optional native acceleration via `@soulcraft/cortex` — SIMD distance calculations, CRoaring bitmaps, Candle ML embeddings. ```javascript -const brain = new Brainy({ storage: { type: 'filesystem', path: './data' } }) -await brain.init() // @soulcraft/cor detected — same code, native engines underneath +const brain = new Brainy({ plugins: ['@soulcraft/cortex'] }) +await brain.init() ``` -Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate. +Plugins are opt-in. Brainy never auto-imports packages unless listed in `plugins`. -Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both. +**[Plugin Documentation](docs/PLUGINS.md)** -## Performance +--- -- JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41). -- Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan. -- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)** +## Type System -## Use cases +42 noun types and 127 verb types form a universal knowledge protocol: -**AI agent memory** — persistent semantic recall with relationship tracking · **Knowledge bases** — auto-linking and meaning-aware navigation · **Semantic search** over codebases, documents, media · **Enterprise data** — CRM, catalogs, institutional memory · **Games & simulations** — worlds and characters that remember. +``` +42 Nouns × 127 Verbs = 5,334 base relationship combinations +``` + +Model any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), education (`Student → completes → Course`), or your own. + +### Subtypes — sub-classification within a NounType *or* VerbType + +Both noun types and verb types are intentionally coarse. Use the top-level `subtype` field to sub-classify entities AND relationships within a type — flat string, no hierarchy, your choice of vocabulary: + +```javascript +// Nouns: sub-classify entities +await brain.add({ + data: 'Avery Brooks — runs the AI lab', + type: NounType.Person, + subtype: 'employee' // 'customer', 'vendor', 'contractor', … +}) + +// Verbs: sub-classify relationships +await brain.relate({ + from: ceoId, + to: vpId, + type: VerbType.ReportsTo, + subtype: 'direct' // 'dotted-line', 'matrix', … +}) + +// Filter on the fast path — column-store hit, not metadata fallback: +const employees = await brain.find({ type: NounType.Person, subtype: 'employee' }) +const directReports = await brain.getRelations({ from: ceoId, subtype: 'direct' }) + +// O(1) counts via the persisted rollups: +brain.counts.bySubtype(NounType.Person) +// → { employee: 12, customer: 847, vendor: 34 } + +brain.counts.byRelationshipSubtype(VerbType.ReportsTo) +// → { direct: 12, 'dotted-line': 3 } +``` + +**Enforce the pairing.** Register a vocabulary per type or turn on brain-wide strict mode to ensure every entity AND relationship has both `type` AND `subtype`: + +```javascript +// Per-type rule with vocabulary +brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true }) + +// Or brain-wide strict mode +const brain = new Brainy({ requireSubtype: true }) +``` + +For other facets you want counted (`status`, `source`, `role`), register them with `brain.trackField(name)`. Renaming an existing convention to `subtype`? Use `brain.migrateField({from, to, entityKind: 'both'})` to walk nouns AND verbs in one pass. Full guide: **[Subtypes & Facets](docs/guides/subtypes-and-facets.md)**. + +**[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** | **[Stage 3 Canonical Reference](docs/STAGE3-CANONICAL-TAXONOMY.md)** + +--- + +## Storage: Memory to Cloud + +The same API at every scale. Change one config line to go from prototype to production. + +### Development — Zero Config + +```javascript +const brain = new Brainy() +``` + +### Production — Filesystem with Compression + +```javascript +const brain = new Brainy({ + storage: { type: 'filesystem', path: './data', compression: true } +}) +``` + +### Cloud — S3, GCS, Azure, Cloudflare R2 + +```javascript +const brain = new Brainy({ + storage: { + type: 's3', + s3Storage: { bucketName: 'my-knowledge-base', region: 'us-east-1' } + } +}) +``` + +Performance benchmarks and capacity planning in **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)**. + +**[Cloud Deployment Guide](docs/deployment/CLOUD_DEPLOYMENT_GUIDE.md)** | **[Capacity Planning](docs/operations/capacity-planning.md)** + +--- + +## Use Cases + +- **AI agents** — Persistent memory with semantic recall and relationship tracking +- **Knowledge bases** — Auto-linking, semantic search, relationship-aware navigation +- **Semantic search** — Find by meaning across codebases, documents, or media +- **Enterprise knowledge** — CRM, product catalogs, institutional memory +- **Interactive experiences** — Game worlds, NPCs, and characters that remember +- **Content platforms** — Similarity-based discovery, intelligent tagging + +--- ## Documentation -| Start | Core | Going deeper | -|---|---|---| -| [Brainy explained simply](docs/eli5.md) | [API reference](docs/api/README.md) | [Architecture overview](docs/architecture/overview.md) | -| [Installation](docs/guides/installation.md) | [Data model](docs/DATA_MODEL.md) | [Consistency model](docs/concepts/consistency-model.md) | -| [Natural-language queries](docs/guides/natural-language.md) | [Query operators](docs/QUERY_OPERATORS.md) | [Multi-process model](docs/concepts/multi-process.md) | -| | [Find system](docs/FIND_SYSTEM.md) | [Scaling](docs/SCALING.md) | +### Start Here + +- **[Brainy explained simply](docs/eli5.md)** — Plain-language overview, no jargon, no code + +### Core + +- **[API Reference](docs/api/README.md)** — Every method with parameters, returns, and examples +- **[Data Model](docs/DATA_MODEL.md)** — Entity structure, data vs metadata +- **[Query Operators](docs/QUERY_OPERATORS.md)** — All BFO operators with examples +- **[Find System](docs/FIND_SYSTEM.md)** — Natural language find() and hybrid search + +### Architecture + +- **[Architecture Overview](docs/architecture/overview.md)** — System design and components +- **[Triple Intelligence](docs/architecture/triple-intelligence.md)** — Vector + graph + metadata unified query +- **[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** — Universal type system +- **[Data Storage Architecture](docs/architecture/data-storage-architecture.md)** — Type-aware indexing and HNSW + +### Virtual Filesystem + +- **[VFS Quick Start](docs/vfs/QUICK_START.md)** — Build file explorers that never crash +- **[VFS Core](docs/vfs/VFS_CORE.md)** — Full VFS API reference +- **[Semantic VFS](docs/vfs/SEMANTIC_VFS.md)** — AI-powered file navigation + +### Guides + +- **[Import Anything](docs/guides/import-anything.md)** — CSV, Excel, PDF, URLs +- **[Framework Integration](docs/guides/framework-integration.md)** — React, Vue, Angular, Svelte +- **[Natural Language Queries](docs/guides/natural-language.md)** — Master the find() method + +### Operations + +- **[Cloud Deployment](docs/deployment/CLOUD_DEPLOYMENT_GUIDE.md)** — AWS, GCS, Azure +- **[Capacity Planning](docs/operations/capacity-planning.md)** — Memory, storage, and scaling +- **[Performance](docs/PERFORMANCE.md)** — Benchmarks and architecture details +- Cost Optimization: **[AWS S3](docs/operations/cost-optimization-aws-s3.md)** | **[GCS](docs/operations/cost-optimization-gcs.md)** | **[Azure](docs/operations/cost-optimization-azure.md)** | **[R2](docs/operations/cost-optimization-cloudflare-r2.md)** + +--- ## Requirements -**Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use. +**Bun 1.0+** (recommended) or **Node.js 22 LTS** -## Contributing & license +```bash +bun install @soulcraft/brainy # Bun — best performance +npm install @soulcraft/brainy # Node.js — fully supported +``` -Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors. +> **Deprecation Notice:** Browser support (OPFS, Web Workers, WASM embeddings) is deprecated in v7.10.0 and will be removed in v8.0.0. Brainy v8+ will be server-only. + +## Single-Writer Model + +Brainy is **single-writer, many-reader** on filesystem storage. One writer +holds an exclusive lock on the data directory; any number of readers can +inspect it concurrently. Opening a second writer throws with the PID of the +existing one. + +```typescript +// Live application — writer mode is the default +const brain = new Brainy({ storage: { type: 'filesystem', rootDirectory: '/data/brain' } }) +await brain.init() + +// Out-of-band diagnostics from a separate process — safe to run while the +// writer is live +const reader = await Brainy.openReadOnly({ + storage: { type: 'filesystem', rootDirectory: '/data/brain' } +}) +await reader.requestFlush({ timeoutMs: 5000 }) +const stats = await reader.stats() +``` + +For incident debugging, use the `brainy inspect` CLI: + +```bash +brainy inspect stats /data/brain +brainy inspect find /data/brain --where '{"entityType":"booking"}' +brainy inspect explain /data/brain --where '{"entityType":"booking"}' +brainy inspect health /data/brain +``` + +See [the multi-process model](docs/concepts/multi-process.md) and the +[inspection guide](docs/guides/inspection.md) for the full story, including +stale-lock detection, the cross-process flush RPC, and what's not yet +enforced on cloud storage backends. + +## Contributing + +We welcome contributions! See **[CONTRIBUTING.md](CONTRIBUTING.md)** for guidelines. + +## License + +MIT © Brainy Contributors diff --git a/RELEASES.md b/RELEASES.md index 7799c6f6..0ab71c45 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -8,1596 +8,168 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so - Debugging data, query, or storage behaviour - A new Brainy feature is available that you want to adopt -## Removed APIs — 7.x → 8.x (the complete ledger) +--- -Every public API removed at the 8.0 major, with its sanctioned replacement. If your code -still calls a left-column name on 8.x it throws (or the config key is rejected) — the -replacement is always a one-line change. (Standing contract from 8.9.0 forward: removals -happen only at majors, after ≥1 minor of loud runtime deprecation naming the replacement.) +## v7.33.2 — 2026-06-24 -| Removed (7.x) | Replacement (8.x) | -|---|---| -| `brain.search(query, k)` | `find({ query })` — semantic; `find({ query, searchMode })` for hybrid | -| `brain.getRelations({...})` | `related(id, opts)` for adjacency; `find({ connected: {...} })` for scoped traversal | -| `brain.neural()` clustering | `find({ vector })` + aggregation `GROUP BY` | -| `Db.search()` | `db.find({ vector })` | -| Pre-8.0 storage path aliases (`directory`, `basePath`, …) | one `storage.path` key (old aliases throw) | -| Reserved keys inside `metadata` bags (silently remapped in 7.x) | top-level params (`subtype`, `visibility`, `confidence`, `weight`, …) — reserved-in-bag throws | -| 7.x COW branches layout (`branches/main/`) | generational MVCC (`asOf()`, `now()`, `db.persist(path)`) — on-disk migration is automatic at first 8.x open | +**Affected products:** any consumer using graph traversals — `find({ connected: { from, via } })`, +`neighbors()`, `related()` — with a **native graph index** on **`mmap-filesystem`** (or any adapter +whose cold-open adjacency load can fail), where a process restart could leave graph queries returning +`[]` despite persisted edges. Drop-in; no API or data changes. -The fork/snapshot family (`brain.snapshot()`, `createSnapshot()`, `restoreSnapshot()`) -is sometimes cited as a 7.x removal — those methods never existed on 7.x; the 8.0 Db API -(`asOf`/`persist`/`restore({confirm})`) is their first real implementation. +### Fix — graph adjacency no longer silently returns `[]` on a cold open + +On a fresh brain open, a native graph provider can load its relationship **count** (the manifest) but +fail to load the source→target **adjacency** itself — so `find({ connected })` / `neighbors()` / +`related()` returned an empty array **indistinguishable from "no edges"**, and stayed empty after +warm-up. Because it was `[]` and not an error, callers could not tell real data was missing. + +Brainy now runs a one-time consistency check on the first graph read: if the index reports that +relationships exist but a known persisted edge's source resolves to **no** neighbors, it **rebuilds +the adjacency from storage**. If even a rebuild cannot read the edge, it throws a loud, catchable +`GraphIndexNotReadyError` instead of serving `[]` as truth. The probe is O(1) (one edge + one neighbor +lookup), cached per brain, and a **no-op once the adjacency is live** — so consumers can return to +clean graph queries instead of in-memory-filter workarounds. + +> The underlying cold-open load is a native-provider concern (being addressed upstream); this Brainy +> change makes the failure **self-healing and loud** rather than silent. --- -## v8.9.0 — 2026-07-19 (flush is durability-only: history maintenance moves to close()) +## v7.33.1 — 2026-06-22 -The write path stops paying maintenance costs — the last structural piece of the -flush-storm class (a production deployment measured single writes blocked 25–191s behind -history reclaim running inline on flush under memory pressure): +**Affected products:** any consumer on `filesystem` / `mmap-filesystem` storage with **more than one +page (> 500) of entities** that defines an **aggregate** over an already-populated brain (the common +reopen-then-`defineAggregate()`/`queryAggregate()` case). **Critical fix** — a permanent post-init +CPU loop. Drop-in; no API or data changes. **Upgrade from 7.32.1 / 7.33.0 immediately.** -- **`flush()` never compacts history.** It persists the current window's deltas and - nothing else — its cost no longer depends on history backlog or retention mode, in any - configuration. **`close()` is the auto-compaction site** (time-bounded per pass, ~5s; - an early stop is a consistent prefix and the next pass resumes). -- **`compactHistory()` gains `timeBudgetMs`** — bound your own maintenance windows; the - same resumable-prefix guarantee applies. -- **The documented trade**: a long-lived writer that never closes accumulates history - until its next explicit `compactHistory()`. Predictable writes, explicit maintenance. - If you run bounded retention on an always-on service, schedule a periodic - `compactHistory({ ...caps, timeBudgetMs })` in your maintenance window. -- **New public doc: `docs/performance-envelopes.md`** — measured per-op envelopes - (p50/p95 at stated scales, hardware, and backend, with the measuring script cited). - Refresh rule going forward: any release touching a measured path re-runs that op's - benchmark and updates the envelope in the same release. -- **New in this file: the Removed APIs 7.x→8.x table** (top of this document) — every - removal with its sanctioned replacement, one place, per the engine-currency contract. - Standing from here: removals only at majors, after ≥1 minor of loud runtime deprecation. +### Fix — `getNouns()` cursor pagination no longer re-scans the first page forever (permanent CPU loop) -## v8.8.2 — 2026-07-19 (one field-resolution law: reserved-field aggregates stop drifting) +`storage.getNouns({ pagination: { cursor } })` paginated by **cursor** never advanced: the shard-scan +adapter is offset-based and **ignored** the cursor, so every cursor call re-returned the first page. +On its own that was harmless — until v7.32.1 correctly made `totalCount` the true dataset total, which +makes `hasMore` stay `true` until a caller has paged through everything. A caller that paginates by +cursor (`cursor = page.nextCursor`) — notably **aggregate backfill** (`defineAggregate()` / +`queryAggregate()` over a pre-populated store) — therefore looped **forever**, re-walking the entire +entity shard tree on every iteration. With more than one page of entities this pegged 1–2 CPU cores +permanently on the JS main thread — with zero queries or traffic — and never settled. (Before v7.32.1 +the very same loop terminated after one page, silently backfilling only the first 500 entities — an +incomplete aggregate.) -Four fixes from a consumer conformance audit, all rooted in the same disease — two field-resolution -regimes where there must be one: +`getNouns()` now treats the cursor as an **opaque, advancing offset token**, so cursor pagination +behaves exactly like offset pagination — it advances and terminates — and an aggregate backfill now +streams the **whole** corpus exactly once (no longer truncated at 500, no longer looping). The backfill +loop was additionally hardened to pure offset pagination as defense in depth. Regression that +reproduces the infinite loop and fails fast on any re-scan: +`tests/unit/storage/getNouns-cursor-pagination.test.ts`. -- **Aggregates grouped by a RESERVED field (`subtype`, `visibility`, …) now decrement on - delete.** The delete/update hooks fed the aggregation engine a partial entity view (type, - service, data, metadata only), so a reserved-field `groupBy` resolved to a nonexistent group - on the way DOWN — counts drifted upward forever after any delete, and updates that moved an - entity between reserved-field groups double-counted it. The hooks now pass the full-fidelity - entity view (every reserved field top-level, the same shape the add path uses). If your - deployment derives stats from reserved-field aggregates, re-define those aggregates once - after upgrading (a changed definition triggers one rescan) or run them fresh — the drifted - persisted counts do not self-heal retroactively. -- **Aggregation `source.where` on reserved fields now filters** instead of silently matching - nothing: the matcher resolves fields through the same resolver `groupBy` uses (top-level - standard fields + custom metadata), so `where: { subtype: 'note' }` means what it says. -- **`removeMany()` refuses empty/invalid selectors loudly.** A bare array passed positionally - (`removeMany([id])` instead of `removeMany({ ids: [id] })`), an empty params object, or - `ids: []` used to resolve successfully having deleted nothing. All three now throw. -- **`find()` accepts both where-key spellings.** Metadata is flattened at index time - (`metadata.entry.title` indexes as `entry.title`); a `metadata.`-prefixed where key now - falls back to its flattened spelling when the prefixed one isn't indexed — the - "unindexed field(s), returning []" confusion for storage-shaped spellings is gone. (A - literal nested custom key named `metadata` still wins when indexed as spelled.) +--- -## v8.8.1 — 2026-07-18 (flush no longer walks the whole generation history + the import dedup off-switch is now honest) +## v7.32.1 — 2026-06-17 -### The flush-storm fix (production incident, reported by a long-running deployment) +**Affected products:** consumers on `filesystem` / `mmap-filesystem` storage whose logs showed +`🔄 Small dataset (1 items) - rebuilding all indexes…` on cold start, or noise from a +`mmap-vector backend not wired` line on every init. Two fixes. Drop-in; no API or data changes. -Under the default adaptive retention, **every `flush()` re-walked the entire committed -generation history** to compute total history bytes for the budget check — O(all -generations) with disk re-reads past the 4,096-entry delta-cache bound. On a brain with -70,000+ accumulated generations that turned every write into a full-tail scan (60-100s -writes), even though the budget (free-RAM-based) never tripped and nothing was ever -reclaimed. Fixed: +### Fix — `getNouns().totalCount` reports the true total, not the page size -- `historyBytes()` now maintains a **running total**: seeded by one walk on first use, - then updated incrementally at every commit and reclaim — the adaptive retention check - on every flush is O(1). Invariant regression-pinned (running total ≡ fresh walk through - both commit paths and compaction). -- New **`brain.historyStats()`** (read-only, exported `HistoryStats`): generation count, - total on-disk bytes, generation/timestamp range, compaction horizon, retention mode, - and the effective adaptive budget — the one-call fleet-audit for sizing retention - exposure per brain. -- Interim guidance for keep-everything deployments already affected: `retention: 'all'` - skips the adaptive accounting entirely (and is the correct policy if you never want - history reclaimed). The accumulated files are harmless at rest; this release removes - the per-write cost of their existence. +`storage.getNouns({ pagination: { limit } })` returned `totalCount` equal to the **page size**, not +the dataset total: the type-first shard scan early-terminates at `offset + limit` for memory +efficiency, and the page-collected length was returned as the total. So +`getNouns({ pagination: { limit: 1 } })` reported `totalCount: 1` for **any** non-empty brain. -### The import dedup off-switch (lifecycle honesty) +The index-rebuild gate uses exactly this call to size the corpus, so a cold start that needs a +rebuild logged `Small dataset (1 items) - rebuilding all indexes…` regardless of the real entity +count (a production deployment saw this for an ~8,800-entity brain — the rebuild then ran from +scratch instead of loading the persisted vector snapshot). -The post-import background deduplication pass (a merge-DELETE writer that runs ~5 minutes -after an import, merging entities judged duplicates by id / name / vector similarity) had -three lifecycle defects, all fixed: +`getNounsWithPagination` now reports the authoritative O(1) noun counter (maintained on every +add/delete and rehydrated from `counts.json` on init) as the unfiltered `totalCount`, and computes +`hasMore` from it. Filtered scans are unchanged (collected length, a lower bound). Layout-independent +(applies equally to branch/COW layouts). Regression: `tests/unit/storage/getNouns-totalCount.test.ts`. -- **`enableDeduplication: false` now actually disables it.** The background pass was - scheduled unconditionally — an import that explicitly opted out could still have - entities auto-removed 5 minutes later. The flag now gates BOTH the inline merge and - the background pass (regression-pinned). -- **One deduplicator per brain, owned by the brain.** Each `import()` call constructed its - own coordinator + deduplicator, so the "debounced" timer never actually debounced across - imports (N imports = N delete timers). The brain now owns a single instance — the - debounce genuinely spans imports — and `close()` cancels pending work, so a delete pass - can never fire against a closed brain. -- **The 5-minute timer is unref'd** — a pending pass no longer holds the process open - (the exit-hang class; this timer had escaped the earlier sweep). +### Log — benign "mmap-vector backend not wired" downgraded to debug -Retention note for keep-everything deployments: with `enableDeduplication: false` on -import calls and `retention: 'all'` in config, no engine path removes records -automatically. +When a native vector provider replaces the JS HNSW index (it owns its own vector storage and exposes +no `setVectorBackend` hook), brainy logged `mmap-vector backend not wired … per-entity reads in use` +on **every** init. This is expected and benign in the native-index model — not a fault, and not by +itself an indication of per-entity reads — but it appeared on every warm and was repeatedly mistaken +for a cold-start cause. It is now a debug-level line (surface it with `BRAINY_LOG_LEVEL=debug`). -## v8.8.0 — 2026-07-17 (OS-limit detection for pool-scale deployments) +> Note: this release fixes the misleading *count/log*. The remaining cold-start symptom on the +> native line (rebuilding instead of loading the persisted vector snapshot) is resolved when the +> native provider loads its snapshot at construction so the index reports a non-zero size before +> brainy's rebuild gate — already the model in the next major (8.0 + native 3.0). -Small minor: brains now detect the two OS limits that bite at pool scale and warn **before** -the incident instead of during it. +--- -- At open (once per process, Linux-only, measurement-only), Brainy reads `RLIMIT_NOFILE` - (soft/hard, from `/proc/self/limits`) and `vm.max_map_count`, and warns loudly when either - sits below the pool-scale floors (soft NOFILE < 65 536; max_map_count < 262 144) — with the - exact raise commands (`ulimit -n` / `LimitNOFILE=` / `sysctl vm.max_map_count`). On stock - defaults the failure otherwise arrives as `EMFILE` or a failed mmap deep inside an index - open, long after the real cause stopped being visible. An unreadable limit produces **no** - warning — no measurement, no claim (non-Linux platforms stay silent). -- Exported for ops doors: `checkOsLimits()` returns the full `OsLimitsReport` - (values + warnings) programmatically, with the floors exported as constants. +## v7.32.0 — 2026-06-16 -## v8.7.1 — 2026-07-17 (writer-lock acquisition is race-proof + machine-readable through init) +**Affected products:** anyone who needs a **portable graph backup**, a partial export, or a +**7.x → 8.0 migration path**. Purely additive — no API removed, no data change. Drop-in. -Two hardenings of the multi-process writer lock (the `locks/_writer.lock` lease that makes a -second writer on the same brain directory fail loudly): +### New — portable graph `export()` / `import()` on `brain.data()` -- **Lock acquisition claims atomically.** The acquire path used read-then-write, leaving a - window where two processes racing an *absent* lock could both "succeed" — and the loser - kept running unlocked, silently. The claim is now an atomic create-exclusive write - (`O_EXCL`): exactly one racer wins; the loser re-evaluates and either fails loudly with - the winner's details or performs a verified stale-takeover. Bounded retries; contention - beyond them fails loudly rather than degrading into a lockless open. -- **`BRAINY_WRITER_LOCKED` survives `init()`.** The conflict error documents a - machine-readable contract (`err.code`, `err.lockInfo` with the holder's pid/host/ - heartbeat), but init's error wrapping silently stripped both, leaving consumers a message - to regex against. The error now passes through unwrapped. +`brain.data()` gains a portable backup/restore pair that serializes part or all of a brain +to a single versioned JSON document (`BackupData`) and restores it. This is the format that +embeds inside portable artifacts (e.g. a serialized subgraph carried by a higher-level file +type) and the recommended way to move a graph between environments or **upgrade an existing +7.x brain into an 8.0 brain**. -Measured while verifying (for operators sizing audits): `brain.auditGraph()` at a -production-consumer scale of ~2,600 relationships / 800 entities costs ~0.1 s warm and -~0.5 s cold, with exact scar counting across reopen. +```typescript +const data = await brain.data() -## v8.7.0 — 2026-07-17 (bulk-transact ergonomics: scaled budgets + timeout telemetry) - -The bulk-import ergonomics release, from a consumer's measured production incident (a serial -import on network-attached storage at ~2 s/op met a flat 30 s transaction budget): - -- **The transact apply budget now scales with the batch** — `max(30 s, opCount × 2 s)` — or - is exactly what you pass as the new `TransactOptions.timeoutMs`. A flat 30 s cap silently - limited honest bulk work to ~15 operations on slow disks while looking generous for small - batches. Internal batch paths (e.g. `removeMany` chunks) get the same scaling. -- **`TransactionTimeoutError` is a diagnosis, not just a failure**: it now reports the - operation it stopped at as `i/N` with the operation's name, elapsed vs budgeted time, and - states the batch rolled back atomically and is retryable. Its `context` carries the same - fields programmatically. -- **The transact envelope is documented** — batch sizing, budget math, chunking with - `ifAbsent` idempotency, and the precompute pattern (`embedBatch` + per-op `vector`) that - keeps model inference out of the commit path. Guide: `docs/guides/optimistic-concurrency.md`. -- Note: `brain.embed()` / `brain.embedBatch()` (the precompute APIs) already ship — public, - with native-provider passthrough, verified end-to-end (batch and single paths produce - bit-identical vectors; a vector-supplied `add` is fully searchable). Honest measurement: - on the default WASM engine, batch throughput ≈ sequential (~160 ms/text) — the precompute - win is keeping inference out of the budgeted commit path, not raw embedding speed. - -## v8.6.0 — 2026-07-17 (brain.auditGraph — the graph-truth verification instrument) - -A minor release adding one new public API, from the fleet's graph-trust program: a read-only -audit that **proves whether relationship reads return stored truth** on a given brain. - -- **`brain.auditGraph(options?)`** walks every canonical relationship record, queries the same - read path applications use (`related()` / VFS `readdir`) with all visibility tiers included, - and classifies every discrepancy into its failure family: `missingFromReads` (records the - read path omits — a stale adjacency index), `danglingEndpoints` (relationships whose endpoint - entity no longer exists — the historical partial-delete scar class), and `readOnlyVerbIds` - (read-path edges with no stored record — ghosts). Design-hidden internal/system edges are - counted separately so intentional hiding is never misclassified as loss. Counts are exact; - example lists cap at `maxExamples` with an explicit `truncatedExamples` flag; the result is - narrated loudly on incoherence. Mutates nothing — safe on a live brain. -- The operational pairing: audit → if incoherent, `repairIndex()` → audit again. A `coherent` - report after the repair is the verified all-clear. Run it after any engine upgrade, restore, - or migration. Guide: `docs/guides/inspection.md`. -- Also: `getNounIds` pagination now refuses an undecodable resume cursor loudly (the same - contract `getNouns`/`getVerbs` gained in 8.5.2 — the third and final walk brought under it). -- Types exported: `GraphAuditReport`, `GraphAuditDiscrepancy`. - -## v8.5.2 — 2026-07-17 (aggregation backfill: exception-safe, generation-verified, and loud) - -Hardening patch from a migration incident (a byte-copied store on new hardware; the service -entered a silent full-CPU loop at boot). Four changes, all in the aggregation engine's -backfill/adoption path: - -- **Backfill walks are exception-safe and non-destructive.** A rescan now builds into a - staging map and swaps in atomically on completion; a mid-walk failure drops the staging map, - keeps the previous live state serving, and surfaces the storage error to the failing query. - Previously the walk wiped live state *before* a scan that could throw, never cleared the - pending flag on failure, and re-ran a full walk on every subsequent query — a silent - wipe/walk/throw loop at the caller's retry rate. -- **Failed walks are latched.** After a walk fails, retries within a 30-second cooldown rethrow - the recorded error instantly instead of re-walking — a tight caller-side retry loop now costs - one loud error per query, never a full store walk per query. -- **Adoption is generation-verified.** Persisted aggregation state is stamped with the store's - committed generation at flush; reopen adoption requires the stamp to equal the current - watermark. Stale state (unclean shutdown) or over-counting state (a fact-log truncation on a - copied store pulled the watermark back) triggers exactly one loud rescan — never a silent - adopt. Pre-8.5.2 state on generation-aware stores rescans once after upgrade, then is stamped. -- **The path narrates.** Adoption decisions, no-adoptable-state outcomes, walk start/finish - (entity count + duration), and walk failures all log by default; a non-advancing storage - pagination cursor aborts the walk loudly instead of looping forever. - -Plus three guards from a full audit of every loop in the open/init path: - -- **Invalid pagination cursors fail loudly.** A supplied-but-undecodable resume token to - `getNouns`/`getVerbs` used to silently restart the walk at offset 0 — to a `while(hasMore)` - caller that re-serves page 1 forever (an unbounded silent CPU loop). It now throws with a - clear message instead. -- **The graph cold-load verb walk has a stall guard.** `hasMore=true` with a missing or - non-advancing cursor aborts loudly instead of re-reading the same page forever. -- **A derived index AHEAD of the store is named at open.** Brainy already surfaced a provider - generation *behind* the committed watermark; the *ahead* direction (the signature of a - byte-copy of a live service, or a log truncation during crash recovery) now logs a loud - warning explaining what happened and that `brain.repairIndex()` forces a heal — instead of - passing unnamed into whatever the derived index does next. - -## v8.5.1 — 2026-07-17 (aggregation state survives restarts + the query-cap ratchet removed) - -Patch release from a production incident (aggregate/count paths taking 40–90 s on an idle box -while vector search stayed fast, and every `find({ limit: 5000 })` suddenly failing against an -"auto-configured query limit of 1000"). Three fixes, one cosmetic: - -- **Aggregation state is actually adopted on reopen.** The boot pattern `defineAggregate()` → - query raced the engine's async state load: the synchronous define always won, flagged a - backfill, and the first query then wiped the just-loaded persisted state and re-walked the - entire store — every restart, forever. Reopening with an unchanged definition now adopts the - persisted state directly (zero scans); a backfill runs only on a real definition change, a - missing/failed state load, or a write that landed before adoption (exactness wins). Apps that - rely on persisted definitions without re-defining at boot also no longer race a spurious - "Aggregate not defined". -- **Backfills are single-flight and batched.** Concurrent queries on a cold aggregate used to - each wipe the others' partial state and start their own full store walk — under steady query - arrival the store never converged (the 40–90 s loop). Now all concurrent queries share one - walk, and one walk fills every aggregate pending backfill (M aggregates ≠ M scans). -- **The query-cap "learning" ratchet is removed.** `maxLimit` is a memory-protection bound, but - a hidden tuner shrank it 20 % per recorded query while the lifetime-average query time - exceeded 1 s — down to a floor of 1000, below the documented 10 000 auto floor, with no - practical recovery, and the resulting error blamed "available free memory" (stale basis - label). The cap now comes from its construction-time basis (or your explicit - `maxQueryLimit` / `reservedQueryMemory`) alone and never changes at runtime; query timing is - recorded for diagnostics only. -- **Cosmetic:** the engine's own persistence keys (`__aggregation_*`, `brainy:entityIdMapper`) - no longer log `[Storage] Unknown key format` at boot — they were always routed correctly; - they're now recognized before the warning fires. - -Operationally: if a host was bitten, upgrading and restarting is the whole fix — no repair -ritual needed. Setting `maxQueryLimit` explicitly remains the valve that bypasses auto-detection -entirely. - -## v8.5.0 — 2026-07-15 (provider access to the fact log + the shared stamp verifier) - -Small additive follow-up to 8.4.0, from the native accelerator's first consumption pass: - -- **Index providers can now reach the fact log through the storage adapter** — new optional - capability `storage.scanFacts()` / `storage.factLogHeadGeneration()` / `storage.factSegmentPaths()`, - wired by the host brain at init as a closure over its live log. Providers hold only `storage` and - must never construct their own fact-log reader (the log's open path is writer-side); `null` means - "no fact log here — use the enumeration walk." -- **The family-stamp verifier is shared** via `@soulcraft/brainy/internals` - (`readFamilyStamp` / `writeFamilyStamp` / `verifyFamilyStamp` + types) so first-party native - providers run literally the same verification function, never a synchronized copy. -- **Rollup stamp invariants accept strings** (`number | string`) — content fingerprints such as a - per-tree SHA-256 are valid invariant values; a type mismatch reads as incoherence, never a pass. -- **`storage.committedGeneration()`** — the committed watermark as a capability, so a provider - compares its stamp's `sourceGeneration` against the store's truth without parsing the private - manifest format. -- **Two durability/stability contracts pinned in the suite:** fsync-before-ack (holds for - `transact()` today; the single-op path is pinned as the documented future target — group commit - becomes latency batching, never durability skipping) and scan-stability-under-rotation (a scan - snapshot yields exactly its facts — no gaps, duplicates, or bleed-in — while segments rotate - beneath it). - -No behavior change for applications; all additions. - -## v8.4.0 — 2026-07-15 (the generation fact log — a sequential, self-verifying commit stream) - -A minor release, fully backward-compatible (all additions; no behavior change for existing APIs). -This is infrastructure: it changes nothing about how you query today, and lays the substrate that -makes index heals and incremental catch-up sequential-read problems instead of directory walks. - -- **Every committed write now also appends a "fact" — an after-image commit record.** Alongside the - existing before-image history, each committed generation (single-op and `transact()` alike) appends - what each touched entity/relationship *became* — or a body-less tombstone for a removal — to an - append-only, checksummed segment log under `_generations/facts/`. Crash-safe by construction: a - torn tail is detected and ignored; on open the log is reconciled to committed truth, so an absent - generation always means "never committed." `transact()` facts are durable when `transact()` - returns; single-op facts ride the same group-commit flush as their history. - -- **New: `brain.scanFacts()`** — stream committed facts in commit order, in batches, with heal-grade - telemetry (total scope up front; per-batch generation range, byte size, and segment; a summary - cross-check at the end; loud abort on any gap — never a silent skip). **`brain.factSegmentPaths()`** - hands zero-copy consumers the immutable sealed segment files directly. New exported types: - `CommitFact`, `FactOp`, `FactScanBatch`, `FactScanHandle`. Facts accumulate from the first write - after upgrading — pre-existing history is not retroactively converted (enumeration remains the - fallback for old data). - -- **New: the entity-tree family stamp.** At every flush/close, brainy stamps which committed - generation the canonical entity files reflect plus the rollup invariants (entity/relationship - counts) that verify the tree whole. At open, coherence is a comparison — a genuine divergence is - loud and names the failing invariant; `repairIndex()` recounts from canonical and re-stamps. New - exports: `readFamilyStamp`, `verifyFamilyStamp`, `ENTITY_TREE_STAMP_PATH`, `FamilyStamp` types. - -- **Storage adapters** gain optional binary raw-byte primitives (`appendRawBytes`, `readRawBytes`, - `writeRawBytes`, `rawByteSize`) — feature-detected; the filesystem and memory adapters implement - them; an adapter without them simply hosts no fact log. The fact-log namespace is registered as a - protected family: no sweeper or GC can delete under it. - -No API breaks. 24 new tests; the full commit-path regression suite is green. - -## v8.3.3 — 2026-07-15 (rename moves the containment edge — no ghost in the old directory) - -One production-reported fix plus a repair path, completing the delete/move hygiene arc (8.3.1 fixed -deletes, 8.3.2 fixed counters, this fixes moves). - -- **A cross-directory `vfs.rename()` now MOVES the containment edge instead of accumulating one per - parent.** The old parent's `Contains` edge was never removed on a move, leaving the entity a child - of **both** directories: `readdir(oldDir)` kept listing it after the move, re-creating the old path - showed the same name twice, and any tree-walking consumer (sync engines, file browsers) saw the - file in two places. The old edge is now removed by edge id, resolved from the graph's own adjacency - — a removal never requires reading the thing being removed. Bonus fix in the same seam: a move **to - the root** now gets its containment edge (it was previously skipped, orphaning the file out of - `readdir('/')`). - -- **`repairIndex()` now also reconciles VFS containment** (new `vfs.repairContainment()`): every VFS - entity's containment edges are checked against its canonical `metadata.path` — stale old-parent - ghosts and duplicate edges are removed, a missing expected edge is restored, and user - knowledge-graph edges are never touched (only `vfs-contains` edges are candidates). Loud per - repair. Stores that performed cross-directory renames under ≤8.3.2 should run `brain.repairIndex()` - once after upgrading — the same single ritual now heals orphan directories, counters, **and** - containment edges. - -- Also ships a permanent lens-consistency regression suite (combined type+subtype vs subtype-only vs - canonical ground truth, id-for-id, warm and after a cold reopen), ported from the field - investigation that closed the historical lens-drop report. - -No API changes beyond the new optional `vfs.repairContainment()` (also invoked by `repairIndex()`). - -## v8.3.2 — 2026-07-14 (honest counters — the recount + removal-without-re-reading) - -Completes 8.3.1's delete-hygiene story at the counter layer, from a production proof chain reported -by a downstream deployment: persisted entity totals were permanently **inflated** — deletes whose -count decrement was silently skipped — and because paginated `totalCount` serves -`Math.max(persistedTotal, scanned)`, the inflated number always won and **no disk cleanup could ever -lower it**. - -- **A removal's count decrement no longer requires re-reading the record being removed.** The - decrement was sourced from re-reading the entity's metadata inside the delete; if that read - returned `null` (a replace race, or a ghost left by a pre-8.3.1 partial delete) the decrement was - silently skipped while the paired add had counted — minting drift on every write→delete→re-create - cycle. The caller's pre-delete read now rides through the whole delete path - (`remove()`/`removeMany()` → the delete operation → `deleteNoun`/`deleteVerb` → - `deleteNounMetadata`/`deleteVerbMetadata`, both sides symmetric): a null internal read falls back - to the known prior record instead of skipping. The `StorageAdapter` signatures gain an optional - `priorMetadata` parameter (additive; existing adapters unaffected). - -- **`repairIndex()` is the sanctioned counter recount — unconditional, and it actually persists.** - `rebuildTypeCounts()` previously rebuilt only the type-statistics arrays and computed the total - *just to log it* — the persisted scalar (`counts.json`) survived every "rebuild" untouched, so an - already-inflated brain could never be corrected. One canonical walk now rebuilds **every** counter - rollup — scalar totals, per-type maps, and type statistics — and persists them, and `repairIndex()` - runs it unconditionally (not only when orphan directories are found: counters can be inflated over - perfectly clean shelves). Brains with delete history should run `brain.repairIndex()` once after - upgrading; the correction survives reopen. - -No API breaks (optional-parameter additions only). Regression tests cover the drift cycle, the -null-read decrement fallback, and the persisted recount across reopen. - -## v8.3.1 — 2026-07-14 (full-removal deletes + family-scoped migration gate) - -Two production-reported fixes in the write/index spine, plus an operator repair path. No API changes; -all behavior changes make previously-wrong states honest. - -- **Deleting an entity now removes it completely — no more "ghost" leftovers on disk.** A canonical - noun delete removed the metadata (content) leg but left the entity's `vectors.json` and its `