feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure

- brain.fillSubtypes(rules): idempotent subtype back-fill for pre-8.0 data.
  One rule per NounType/VerbType (literal default or per-entry function);
  fills only entries still missing a subtype through the public update()/
  updateRelation() paths; returns { scanned, filled, skipped, errors, byType }.
  Full unit suite in tests/unit/brainy/fill-subtypes.test.ts.
- Fix getNouns/getVerbs pagination hasMore (peek one past the window) —
  was permanently false, silently truncating every multi-page walk.
- find({ near }) without near.id now throws a teaching error instead of an
  opaque storage sharding failure; CLI --threshold without --near applies a
  plain score floor.
- CLI init/close audit: every one-shot command init()s, close()s, and exits
  explicitly; delete the unmaintained interactive REPL; replace the cloud-era
  storage subcommands with status/batch-delete; new types/validate commands.
- requireSubtype JSDoc now documents the 8.0 default-on contract; audit()
  recommendation points at fillSubtypes.
- Docs: data-storage-architecture rewritten to the real 8.0 on-disk layout;
  README storage section reflects filesystem+memory and snapshots; eli5 and
  SEMANTIC_VFS /as-of/ semantics corrected; internal tracker IDs and
  .strategy references scrubbed from published files.
This commit is contained in:
David Snelling 2026-06-11 10:42:34 -07:00
parent 9b0f4acd5b
commit c44678390e
30 changed files with 1517 additions and 3226 deletions

View file

@ -278,11 +278,16 @@ brain.counts.byRelationshipSubtype(VerbType.ReportsTo)
**Enforce the pairing.** Register a vocabulary per type or turn on brain-wide strict mode to ensure every entity AND relationship has both `type` AND `subtype`:
```javascript
// Per-type rule with vocabulary
// Per-type rule with a closed vocabulary
brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true })
// Or brain-wide strict mode
const brain = new Brainy({ requireSubtype: true })
// 8.0 default: every write requires a subtype. Exempt genuine catch-all types…
const brain = new Brainy({ requireSubtype: { except: [NounType.Thing] } })
// …or opt out while migrating pre-8.0 data, then audit and back-fill:
const legacy = new Brainy({ requireSubtype: false })
await legacy.audit() // gaps, grouped by type
await legacy.fillSubtypes({ [NounType.Person]: 'unspecified' }) // close them
```
For other facets you want counted (`status`, `source`, `role`), register them with `brain.trackField(name)`. Renaming an existing convention to `subtype`? Use `brain.migrateField({from, to, entityKind: 'both'})` to walk nouns AND verbs in one pass. Full guide: **[Subtypes & Facets](docs/guides/subtypes-and-facets.md)**.
@ -291,7 +296,7 @@ For other facets you want counted (`status`, `source`, `role`), register them wi
---
## Storage: Memory to Cloud
## Storage: Memory and Filesystem
The same API at every scale. Change one config line to go from prototype to production.
@ -301,28 +306,31 @@ The same API at every scale. Change one config line to go from prototype to prod
const brain = new Brainy()
```
### Production — Filesystem with Compression
### Production — Filesystem (gzip compression on by default)
```javascript
const brain = new Brainy({
storage: { type: 'filesystem', path: './data', compression: true }
storage: { type: 'filesystem', rootDirectory: './data' }
})
```
### Cloud — S3, GCS, Azure, Cloudflare R2
### Backups and Portability — Snapshots
```javascript
const brain = new Brainy({
storage: {
type: 's3',
s3Storage: { bucketName: 'my-knowledge-base', region: 'us-east-1' }
}
})
const db = brain.now()
await db.persist('/backups/2026-06-11') // instant hard-link snapshot
await db.release()
const snapshot = await Brainy.load('/backups/2026-06-11') // read-only Db
const hits = await snapshot.search('quarterly invoices')
await snapshot.release()
```
A snapshot directory is self-contained — copy it to another machine, open it with `Brainy.load()`, or restore it wholesale with `brain.restore(path, { confirm: true })`.
Performance benchmarks and capacity planning in **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)**.
**[Cloud Deployment Guide](docs/deployment/CLOUD_DEPLOYMENT_GUIDE.md)** | **[Capacity Planning](docs/operations/capacity-planning.md)**
**[Capacity Planning](docs/operations/capacity-planning.md)**
---
@ -371,10 +379,8 @@ Performance benchmarks and capacity planning in **[docs/PERFORMANCE.md](docs/PER
### Operations
- **[Cloud Deployment](docs/deployment/CLOUD_DEPLOYMENT_GUIDE.md)** — AWS, GCS, Azure
- **[Capacity Planning](docs/operations/capacity-planning.md)** — Memory, storage, and scaling
- **[Performance](docs/PERFORMANCE.md)** — Benchmarks and architecture details
- Cost Optimization: **[AWS S3](docs/operations/cost-optimization-aws-s3.md)** | **[GCS](docs/operations/cost-optimization-gcs.md)** | **[Azure](docs/operations/cost-optimization-azure.md)** | **[R2](docs/operations/cost-optimization-cloudflare-r2.md)**
---
@ -387,7 +393,7 @@ bun install @soulcraft/brainy # Bun — best performance
npm install @soulcraft/brainy # Node.js — fully supported
```
> **Deprecation Notice:** Browser support (OPFS, Web Workers, WASM embeddings) is deprecated in v7.10.0 and will be removed in v8.0.0. Brainy v8+ will be server-only.
> Brainy 8.0 is server-only. Browser support (OPFS storage, Web Workers, in-browser WASM embeddings) was removed in 8.0 — the 7.x line remains available on npm if you need it.
## Single-Writer Model