feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure

- brain.fillSubtypes(rules): idempotent subtype back-fill for pre-8.0 data.
  One rule per NounType/VerbType (literal default or per-entry function);
  fills only entries still missing a subtype through the public update()/
  updateRelation() paths; returns { scanned, filled, skipped, errors, byType }.
  Full unit suite in tests/unit/brainy/fill-subtypes.test.ts.
- Fix getNouns/getVerbs pagination hasMore (peek one past the window) —
  was permanently false, silently truncating every multi-page walk.
- find({ near }) without near.id now throws a teaching error instead of an
  opaque storage sharding failure; CLI --threshold without --near applies a
  plain score floor.
- CLI init/close audit: every one-shot command init()s, close()s, and exits
  explicitly; delete the unmaintained interactive REPL; replace the cloud-era
  storage subcommands with status/batch-delete; new types/validate commands.
- requireSubtype JSDoc now documents the 8.0 default-on contract; audit()
  recommendation points at fillSubtypes.
- Docs: data-storage-architecture rewritten to the real 8.0 on-disk layout;
  README storage section reflects filesystem+memory and snapshots; eli5 and
  SEMANTIC_VFS /as-of/ semantics corrected; internal tracker IDs and
  .strategy references scrubbed from published files.
This commit is contained in:
David Snelling 2026-06-11 10:42:34 -07:00
parent 9b0f4acd5b
commit c44678390e
30 changed files with 1517 additions and 3226 deletions

View file

@ -167,8 +167,12 @@ this rename; only the API surface moved.
Escape hatches: `requireSubtype: false` (last-resort opt-out for legacy
data) or `requireSubtype: { except: [NounType.Thing, ...] }` (per-type
allowlist). Per-type rules registered via `brain.requireSubtype(type, opts)`
still compose. `brain.audit()` reports entries missing a subtype and
`brain.migrateField()` backfills them.
still compose. `brain.audit()` reports entries missing a subtype and the new
`brain.fillSubtypes(rules)` migration helper backfills them — one rule per
NounType/VerbType (literal default or per-entry function), applied only to
entries still missing a subtype, returning
`{ scanned, filled, skipped, errors, byType }`. Idempotent: re-running fills
nothing, so a crashed run is resumed by running it again.
- **Verb ids are Brainy-generated UUIDs — by contract.** `relate()` now throws
a teaching error if a caller passes an `id` field
(`src/utils/paramValidation.ts:526`). In 7.x a supplied id was silently
@ -270,8 +274,10 @@ The full cost model is in
`getHistory` / `streamHistory` / `versions` / `data()` per the table above.
6. **Subtypes:** if your data predates subtype discipline, start with
`requireSubtype: false`, run `brain.audit()`, backfill with
`brain.migrateField()`, then remove the opt-out so the 8.0 default
enforcement protects you going forward.
`brain.fillSubtypes(rules)` (one rule per type — a literal default or a
function deriving the subtype from each entry), re-run `audit()` until
`total === 0`, then remove the opt-out so the 8.0 default enforcement
protects you going forward.
7. **CLI scripts:** `fork` / `branch` / `checkout` / `migrate`
`snapshot <path>` / `restore <path>` / `history` / `generation`.
8. **Plugin authors:** apply the contract renames and the BigInt graph
@ -553,8 +559,7 @@ alongside every other metadata field on the existing write paths.
`_rev`, `ifRev`, `RevisionConflictError`, and `ifAbsent` all survive the 8.0 Db
redesign unchanged. 8.0 layers `brain.transact(tx, { ifAtGeneration })` for
whole-tx CAS on top of the same per-entity mechanism — per-entity for single-record
patterns, generation-based for "did the world move under me." See
`.strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md` § C-6 (internal).
patterns, generation-based for "did the world move under me."
---
@ -672,8 +677,7 @@ in `validateFindParams()`, both fire before any storage/index/Cortex call. Corte
The new `docs/guides/find-limits.md` calls out that 8.0's Datomic-style `Db.find()`
may tighten per-call limits; that's a Brainy 8.0 / Cortex 3.0 coordination point
documented in `.strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md` (open question #4 covers
the rollout staging, which now also applies to query limits).
whose rollout staging now also covers query limits.
### What consumers should do
@ -830,8 +834,7 @@ Brainy before any native call.
- **Native `audit()` proxy.** For billion-scale brains, `audit()` walks every entity (O(N)).
A native implementation reading from a "null-subtype" bitmap in the column store would be
O(buckets). Listed as the 6th open question in the
[Brainy 8.0 spec doc](`/media/dpsifr/storage/home/Projects/brainy/.strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md`).
O(buckets).
- **Strict-mode parity test.** Cortex should mirror Brainy's new
`tests/integration/strict-mode-self-test.test.ts` against their native paths to catch any
latent bug where native writes bypass JS validation.