feat: subtype top-level field + trackField + migrateField

Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).

Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
  BaseStorage, mirrored after nounCountsByType with the same self-heal
  rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path

Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
  cardinality + per-NounType breakdown stats. Backed by the aggregation
  engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update

Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
  streams every entity, copies the value from one path to another, and
  (unless readBoth) clears the source. Supports top-level standard fields,
  metadata.X, and data.X paths. Idempotent — safe to re-run.

Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
  quick-start all treat subtype as a core primitive with anonymous example
  vocabularies (employee/customer/invoice/milestone).

Tests
- 26 new integration tests covering write/read/update/delete round-trips,
  counts rollup decrement + re-route on mutation, trackField + byField
  with and without perType, vocabulary whitelist enforcement, and
  migrateField for metadata.X → subtype and data.X → subtype paths
  including readBoth deprecation-window semantics.

Unit suite: 1468/1468 passing. Type-check + build clean.
This commit is contained in:
David Snelling 2026-06-04 17:24:36 -07:00
parent e47fea0917
commit 2cdf70ee0f
14 changed files with 1698 additions and 21 deletions

View file

@ -186,6 +186,7 @@ When you add an entity, Brainy stores these standard fields in the metadata obje
| Field | Set By | Description |
|-------|--------|-------------|
| `noun` | System | Entity type (NounType enum value) |
| `subtype` | User | Per-NounType sub-classification (e.g. `'employee'`, `'invoice'`, `'milestone'`). Flat string, no hierarchy. Indexed on the fast path and rolled into per-NounType statistics. |
| `data` | System | The raw `data` value (stored opaquely) |
| `createdAt` | System | Creation timestamp |
| `updatedAt` | System | Last update timestamp |
@ -196,6 +197,30 @@ When you add an entity, Brainy stores these standard fields in the metadata obje
On read, these standard fields are extracted to top-level Entity properties. The `metadata` field on the returned Entity contains **only your custom fields**.
### Subtype — sub-classification within a NounType
`type` (NounType) is a stable 42-value enum. `subtype` is the consumer-chosen string vocabulary *within* a type:
```typescript
// A Person who is an employee:
await brain.add({
data: 'Avery Brooks — runs the AI lab',
type: NounType.Person,
subtype: 'employee',
metadata: { department: 'ai-lab' }
})
// A Document that is an invoice:
await brain.add({
data: 'INV-2026-001',
type: NounType.Document,
subtype: 'invoice',
metadata: { amount: 1500 }
})
```
`subtype` lives at the **top level** — NOT inside `metadata`, NOT inside `data`. That's how `find({ type, subtype })` routes through the standard-field fast path (column-store hit) instead of the metadata fallback. See **[Subtypes & Facets](./guides/subtypes-and-facets.md)** for the full guide including `trackField()` and `migrateField()`.
---
## See Also